OpenSIPS Script rework
There are two types of problems with the current OpenSIPS scripting language, which relate to both its structure and functionality:
Trying to address the above issues, we suggest as starting points for the Public Meeting two new formats for the configuration script:
1) a simple re-structuring to solve the problems, but trying to keep a similarity with the actual structure. The grammar will enforce three main sections in the cfg file, in a certain order:
example # script starts with the global param section debug = 4 log_name = "opensips_proxy.log" listen = 1.1.1.1 mpath = "/tmp/test/" # after global params we have the section dealing with modules loadmodule [tm] { fr_timer = 4 timer_partitions = 2 } loadmodule [avpops] { db_url = "mysql://user:passwd@host/database" use_domain = 1 } # last section is exclusively for routing routes route {...} failure_route [resume] {...} local_route {...}
2) a complete re-structuring to align to a standard config file format, such as the INI format . The entire configuration will be split into two files: a ".ini" file for global parameters, module configuration and routing files, and a second file containing just the routing logic:
Example of "opensips.ini" file
[global] debug = 4 log_name = "oss.log" listen = 1.1.1.1 mpath = "/tmp/test/" routing_file = "./opensips.route" routing_socket = wss:127.0.0.1:8977 [tm] fr_timer = 4 timer_partitions = 2 [avpops] db_url = "mysql://user:passwd@host/database" use_domain = 1
Example of opensips.route file
route {...} failure_route [resume] {...} local_route {...}
Both above formats address the following issues:
As a side note, we are also taking into consideration a mid/long term feature request which will allow OpenSIPS to delegate its entire routing logic to scripts written in different languages or even to remote applications (via network protocol).
18:02 ---| bogdan_vs has changed the topic to: OpenSIPS Public Meeting - the script rework 18:02 <@ bogdan_vs>| Hi all 18:02 < FlavioEGonc>| Hi, 18:02 < razvanc>| we've just come back from the long summer vacation 18:02 < razvanc>| and now its time for a new public meeting 18:02 <@ bogdan_vs>| indeed... 18:03 < razvanc>| where we'll discuss some script changes we want to make 18:03 <@ bogdan_vs>| fresh forces, fresh people, fresh ideas ;) 18:04 <@ bogdan_vs>| we posted last week some ideas about these changes 18:04 <@ bogdan_vs>| mainly what are the reasons behind the change and what we are looking to solve with these changes 18:04 <@ bogdan_vs>| and most important, some potential solutions, as a starting point 18:05 <@ bogdan_vs>| http://www.opensips.org/Community/IRCmeeting20150930 18:06 <@ bogdan_vs>| Anyone who read that and willing to contribute to the topic ? 18:06 < FlavioEGonc>| I like the idea of having the core parameters in a separate file. These are usually tied to the server. 18:07 < FlavioEGonc>| It would also interesting to have macro variables in the core file maing the routing script independent of hardware information. 18:08 < FlavioEGonc>| Similar to what we do now with M4 18:09 < razvanc>| FlavioEGoncalves: so you'd like to see some preprocessor capabilities directly in OpenSIPS? 18:10 < FlavioEGonc>| Yes it should help, and it is indeed perceived as a differential for developers in usability 18:10 < razvanc>| we've discussed this topic a time ago, and decided we'd better go with M4, since it's a more generic language and offers a lot more features that what we would have added 18:11 < FlavioEGonc>| Anything capable to separate user and harwdare information from the script logic. M4 is good and I use it now, but it is an additional component. 18:12 < razvanc>| that's true 18:12 < razvanc>| I find the file separation a very nice feature 18:12 < razvanc>| although we can do that now 18:12 < razvanc>| I've never used it that much 18:13 < FlavioEGonc>| Me too, if the ini had some MACRO variables (similar to M4) it could solve the separation problem. 18:14 < razvanc>| so you were thinking we could define some variables in the global section of the ini file, then use it in the route file? 18:14 < janthon1>| neteng 18:14 < FlavioEGonc>| Yes, they would be replaced during the script load and read-only. 18:15 < FlavioEGonc>| In summary #define 18:18 < razvanc>| so reading the two options, I presume you like the second one :) 18:18 < razvanc>| using the .ini file 18:18 < razvanc>| .ini syntax 18:20 < CIDname>| if I may say... 18:20 <@ bogdan_vs>| what about script reloading ? 18:20 < CIDname>| the ini syntax seems more of a departure 18:20 <@ bogdan_vs>| what is the feeling in regards to such a feature ? 18:20 < CIDname>| from the current 18:21 < razvanc>| CIDname: so you'd rather keep something more familiar to what we have? 18:21 < CIDname>| I would razvan 18:21 < CIDname>| especially since the object is to make it "simpler" 18:21 <@ bogdan_vs>| indeed, the ini is a radical change, option 2) is more familiar 18:21 < razvanc>| if we were to choose the 2nd option, we'd actually have two syntaxes 18:21 < CIDname>| now on the script reloading 18:22 < CIDname>| you mean... being able to reload just routing section? 18:22 < razvanc>| CIDname: yes, reloading the entire OpenSIPS would be near impossible 18:22 < bogdan_vs> | razvanc: true, but gives a better separation between configuration (params) and routing (script) 18:22 < CIDname>| well we can take example from many other softwares 18:22 < CIDname>| like fs and * 18:23 < CIDname>| the ability to reload only certain aspects of the core based on what little things you change is ideal 18:23 < CIDname>| for instance 18:23 < CIDname>| we already have something similar for modules 18:23 < CIDname>| authentication on IP 18:23 < CIDname>| add to the db 18:23 < CIDname>| reload the table 18:23 < vlad-paiu>| the ini structure would be just for the global params & module params. the routing logic would remain intact and in a separate file. this would make it cleaner from my point of view, and you'd also know better what exactly you are reloading, just the routing logic file 18:24 < CIDname>| from a functionality perspective, I don't disagree at all 18:24 < CIDname>| it's obvious its a better way 18:24 < FlavioEGonc>| Initially I liked the ini structure, but what's the bennefit? 18:24 < CIDname>| i'm just pointing out, that there would be complexity added to the end user 18:24 < vlad-paiu>| CIDname : reloading the script would just be reloading the actual script.. all the loaded data ( eg. permissions, drouting ) would remain untouched 18:25 < CIDname>| yeah I get that 18:25 < CIDname>| I was just backing up why I thought it was good 18:25 < CIDname>| in fs and * 18:25 < CIDname>| you don't need to do a full reload 18:25 < CIDname>| only the parts you touched 18:25 < vlad-paiu>| FlavioEGoncalves : well, it's a standard. and the script would be simpler and easier to read, I think 18:26 < FlavioEGonc>| There is a problem with changes in the format, incompatibility with previous versions and effort to migrate scripts. is it worth changing? 18:26 < CIDname>| ooo 18:26 < vlad-paiu>| Instead of loadmodule "tm.so" and modparam("tm", "fr_timeout", 10) you would have just [tm] fr_timeout=10 18:26 < CIDname>| flavio makes a good point 18:26 <@ bogdan_vs>| File based separation can be done in both options 18:27 <@ bogdan_vs>| CIDname / FlavioEGoncalves : we do not want to make the live harder for users, but we cannot simply sit back and do no progress for the sake of changes-fear 18:27 < CIDname>| agree 100% 18:28 < CIDname>| sooner or later radical changes have to be made 18:28 <@ bogdan_vs>| the idea of changes is not something cosmetic 18:28 < CIDname>| I think if it's well documented 18:28 <@ bogdan_vs>| there are some good reasons leading to that 18:28 < CIDname>| we would be ok 18:28 < razvanc>| also, changing the structure was more like a need, to support other features 18:28 < razvanc>| like auto loading, script reload 18:28 <@ bogdan_vs>| we have those reasons into the wiki page 18:28 < razvanc>| exactly ^ 18:29 <@ bogdan_vs>| CIDname: we can even consider a small translator tool for the scripts 18:29 <@ bogdan_vs>| as it is about syntax only 18:29 < CIDname>| anything would help bogdan 18:29 < vlad-paiu>| migrating the script should be trivial, either way we go ( ini or similar to now ) 18:29 <@ bogdan_vs>| can be automatically done by a small tool 18:29 < CIDname>| we just don't want to add another pothole in the road 18:29 < vlad-paiu>| bogdan_vs : right 18:29 <@ bogdan_vs>| well, the script format proved its limitations.... 18:30 < FlavioEGonc>| Ok, if the bennefit is bigger than the effort to migrate it makes sense. 18:30 < CIDname>| you guys have or are working on a menu config right? 18:30 < razvanc>| CIDname: vlad-paiu have already done the menuconfig 18:30 <@ bogdan_vs>| we can keep adding shiny piece of glass to it to make it nicer, but the big problems will still be there 18:30 < CIDname>| if we bundle into that, could be more seemless for the transition 18:31 < CIDname>| sp I think we all agree and support the change needs and reasons 18:31 <@ bogdan_vs>| at the end of the day, aside aligning the script to the actual needs (for new functionalities), the new grammar will be more friendly with the users 18:32 < CIDname>| we need to make a commitment to make it as easy on the users as possible 18:33 < FlavioEGonc>| I read the reasons on the wiki page, the reload, other languages and external routing makes sense. 18:34 <@ bogdan_vs>| FlavioEGoncalves: yes, in medium, long term we need to have a better separation between engine and logic (read here "script") 18:34 <@ bogdan_vs>| in the future, this logic may be outside OpenSIPS 18:34 < FlavioEGonc>| If you are working with menuconfig, please fix the issue in the script generator (no DB_URL in the uri module). The scripts generated don't load. 18:34 <@ bogdan_vs>| you can control your OpenSIPS from other applications, in other languages 18:35 < CIDname>| don't make me masturbate all over this irc 18:35 <@ bogdan_vs>| FlavioEGoncalves: bug report (y) 18:35 < CIDname>| thats gonna be so freaking cool 18:35 < FlavioEGonc>| Ok 18:35 < CIDname>| he went to get an umbrella 18:35 < CIDname>| lol 18:35 <@ bogdan_vs>| should we make a public poll for the format ? 18:36 < CIDname>| no 18:36 < CIDname>| executive decision 18:36 <@ bogdan_vs>| as it is a 100% end-user oriented, which version is considered more friendly should be selected by users 18:36 <@ bogdan_vs>| imho 18:37 < razvanc>| I think the question would be whether we can make the new structure similar to the current one or not 18:37 < razvanc>| because if we will introduce a new syntax people will have to learn, we'd better go with .ini, because most people already know it 18:37 <@ bogdan_vs>| option 1) is closer (as format) to the existing one 18:37 < liviuc>| Flavio: it's already fixed; see this: https://github.com/OpenSIPS/opensips/commit/724d16d99f9ae46c 18:38 <@ bogdan_vs>| liviuc KO to FlavioEGoncalves :) 18:39 <@ bogdan_vs>| razvan: what scares people is not the grammar for setting the param, but rather the actual routing script 18:39 <@ bogdan_vs>| which will stay the same anyhow 18:39 <@ bogdan_vs>| I do not see a problem in migrating from modparam(x,y,z) to y=z 18:39 <@ bogdan_vs>| it is more about comfort and logic 18:40 <@ bogdan_vs>| and as time as we go upwards with them, it should be no resistance from the users 18:40 < razvanc>| well, I agree 18:40 < CIDname>| I do too 18:41 <@ bogdan_vs>| the difference between 1) and 2) is more esthetically 18:41 < CIDname>| option 1 is the closest to normal 18:41 < CIDname>| I don't think there will be a user that would willingly say.. "hey change it to 2" 18:41 <@ bogdan_vs>| has nothing to do with easiness or simplicity 18:42 < razvanc>| what I was saying is that I'd rather keep the syntax similar to the routes 18:42 <@ bogdan_vs>| as liviuc said at some point (out of frustration ??) - "let's have them both" :P 18:42 < CIDname>| well 18:42 < CIDname>| that would nicely solve the problem :) 18:42 <@ bogdan_vs>| hey....^^^ I'm joking 18:43 < razvanc>| for example, loadmodule "avpops" {} should be loadmodule[avpops] {} 18:43 < CIDname>| <blank stare> 18:43 < razvanc>| consistent with the routes, and so on 18:43 < liviuc>| that resembles a route statement actually 18:43 < liviuc>| right 18:43 <@ bogdan_vs>| it may be confusing 18:44 <@ bogdan_vs>| :| 18:44 < liviuc>| I think 10 different keywords which all end with "_route" should be more confusing 18:44 < liviuc>| and yet everybody loves OpenSIPS 18:44 < razvanc>| gg liviuc :D 18:45 < CIDname>| well played 18:45 < razvanc>| and it won't be confusing at all if we use different files :) 18:45 <@ bogdan_vs>| ok, but the most important is that we agree in pushing these changes 18:45 <@ bogdan_vs>| unfortunately lirakis couldn;t attend today.... 18:46 <@ bogdan_vs>| but he provided some input yesterday on the channel 18:47 < liviuc>| regarding the people who'd prefer N routing files, we can keep the current "import_file" directive 18:47 < liviuc>| so they can stick all kinds of files within the main script: global param files, module files, route files 18:48 < liviuc>| making an additional "routing_file" parameter seems like a burden 18:48 < FlavioEGonc>| Liviu, the import_file has an issue, it is difficult to debug later, because of the numbering. 18:48 < liviuc>| question is: is it fixable? :) 18:48 <@ bogdan_vs>| <FlavioEGoncalves> :what do you mean ? 18:48 < FlavioEGonc>| I never used but I'm assuming that numbering would be of both files together. 18:48 < liviuc>| I mean, in Python you can "import" wherever you want! 18:49 < FlavioEGonc>| So error in line 300 wouldn't be in the 300th line of the file, as I said I never used the import directive, but I would like to have a match between the line with the problem and the line of the file. 18:50 <@ bogdan_vs>| line numbering ? 18:50 <@ bogdan_vs>| it should be relative to the file 18:51 <@ bogdan_vs>| ok, so bottom line 18:51 <@ bogdan_vs>| we do the changes (at least we start planning them) 18:51 <@ bogdan_vs>| one by one 18:52 <@ bogdan_vs>| the remaining thing is which format to go for .... 18:52 < jarrod>| i liked the concept of repl for show commands (fifo) and changing configuration parameters similar to junipers junos 18:52 <@ bogdan_vs>| for the globals ? 18:53 < razvanc>| jarrod: are you talking about something like osipsconsole? 18:53 < jarrod>| honestly ive never used it, but could be 18:53 <@ bogdan_vs>| that's why, internally, we will migrate all the global params to something similar the modules have (a structure) 18:54 < razvanc>| that would be nice :) 18:55 < CIDname>| for syntax I would say we choose 1 18:56 < CIDname>| assuming there are no future limitation that we would encounter 18:56 < razvanc>| bogdan_vs: are you thinking about whether we should put the global parameters in the file in a different section, or use it as it is in the example? 18:57 < bogdan_vs> | razvanc: it is not about the script, but rather in the code - instead of hardcoding and having all the params in grammar, we can have something dynamic as for the param modules 18:57 < razvanc>| bogdan_vs: that's what I said it would be nice 18:57 <@ bogdan_vs>| so you do not have to change grammar (and 5 files) each time a core param is added or removed 18:57 < razvanc>| :D 18:57 < razvanc>| but I was now asking about the script file 18:57 < razvanc>| in section 1 18:57 <@ bogdan_vs>| oh... 18:58 < razvanc>| should we use a global section for the core parameters? 18:58 <@ bogdan_vs>| maybe leaving without block 18:58 <@ bogdan_vs>| that would be my first love impression :) 18:59 < razvanc>| I bet its me, I like everything to be consistent :) 18:59 < CIDname>| bet you wash your hands like 25 times a day too 18:59 <@ bogdan_vs>| I would suggest to make a poll to see what versions (1 versus 2) is more liked by users 19:00 < razvanc>| CIDname: 27 to be precise :) 19:00 < CIDname>| LMAO! 19:00 <@ bogdan_vs>| it may provide useful info for making a final decision 19:00 < CIDname>| ok lets do the poll 19:00 < CIDname>| so we can verify that I'm right ;) 19:00 < razvanc>| bogdan_vs: I agree, this should be the community's decision 19:01 <@ bogdan_vs>| decision or not, but it can be valuable info :) 19:01 < razvanc>| yup 19:01 < CIDname>| I would leave out as to "why" we're doing it 19:01 <@ bogdan_vs>| right 19:01 < CIDname>| well the long description of why 19:02 <@ bogdan_vs>| that was in the Today's menu 19:02 < razvanc>| yup, we'll follow up on the mainling list with a poll 19:02 < FlavioEGonc>| I like the idea of having blocks such as [global] it is easier to understand and more compact. 19:03 <@ bogdan_vs>| ok, we will re-work the 2 examples, based on what was discussed here 19:04 < liviuc>| I was initially against global { ... }. Now I'm starting to like it 19:04 <@ bogdan_vs>| and we will follow up with an email on the lists, inviting people to the poll (even if a pool will be better) 19:04 < razvanc>| yup, we'll sum up everything we discussed, and perhaps propose a third version in the poll 19:04 < FlavioEGonc>| Ok, 19:05 <@ bogdan_vs>| ok, anything else left to be said ? 19:05 < liviuc>| Also, we really need to look into the line numbering issue for import/include directives 19:06 <@ bogdan_vs>| liviuc: open a bug report ;) 19:06 < FlavioEGonc>| It is not a bug, 19:06 < FlavioEGonc>| Just an inconvenient 19:07 <@ bogdan_vs>| a fuzzy difference :) 19:07 < CIDname>| bugs can be invonceniences 19:07 < razvanc>| FlavioEGoncalves: if you can't match the file+line with the exact location, I think it is a bug 19:07 < CIDname>| i'm dyslexic 19:07 < FlavioEGonc>| Ok, Ok, 19:08 < CIDname>| your inconvenince has just been upgraded 19:08 < razvanc>| :D 19:08 <@ bogdan_vs>| ok, I guess we are done for today 19:08 < CIDname>| noooooooo 19:08 < CIDname>| let's stay and party! 19:09 < CIDname>| sigh 19:09 < CIDname>| everyone has work to do it seems 19:09 <@ bogdan_vs>| CIDname - you bring booz and girls ;) 19:09 < razvanc>| :)) 19:09 < CIDname>| man it wouldn't be a party without hookers & beer 19:09 <@ bogdan_vs>| thank you for participating and especially for the valuable input to the topic 19:09 < razvanc>| I guess I'll have to strip this conversation from the meeting's summary :) 19:09 <@ bogdan_vs>| and keep your eyes on the follow ups ;) 19:10 < CIDname>| razvan: why? 19:10 <@ bogdan_vs>| btw, keep an eye on the OpenSIPS week in Austin ! 19:10 < razvanc>| let's say they do not allign with today meeting's topic 19:10 < CIDname>| that's right! 19:11 < CIDname>| who's coming to Austin?! 19:11 <@ bogdan_vs>| we have a Summit planned for the beginning of November 19:11 < CIDname>| I'll have hookers & beer 19:11 < CIDname>| raz: (seeing you're editing anyway) lmao 19:11 < FlavioEGonc>| I'll be there. 19:11 < tuxx->| hookers & beer in austin? 19:11 < tuxx->| when? :D 19:11 <@ bogdan_vs>| and the first in-class Bootcamp - 3 days of Advanced OpenSIPS 19:12 <@ bogdan_vs>| who gets the OpenSIPS certification -> hookers & beer in austin :) 19:12 < CIDname>| Nov 9 & 10th is the summit 19:12 < CIDname>| and 11- 13 an advanced hookers & beer class 19:12 < CIDname>| errr I mean advanced opensips training 19:12 < tuxx->| :D 19:12 < CIDname>| Seriously though... 19:13 < CIDname>| this will be the best summit ever 19:13 < CIDname>| cool town 19:13 < CIDname>| cool speakers 19:13 < CIDname>| cool topics 19:13 < CIDname>| http://www.opensips.org/Community/Summit-2015Austin 19:13 ---| bogdan_vs has changed the topic to: OpenSIPS Week - Summit & Training - 9th - 13th of November, Austin, TX