Login | Register

Community

Community -> PublicMeetings -> 27th of August 2014

Topics

Development Directions for OpenSIPS 1.12

  • Fraud detection: a new module that allows you to configure different calling profiles and control calls according to different constraints
  • Quality Routing: create a new module that routes messages based on the quality of the gateways
  • Message compression: provide different methods for reducing the size of the SIP messages
  • Asynchronous processing: provide a method to allow the SIP messages processing to resume after I/O tasks (DB, DNS queries) in other processes

Conclusions

  • Message compression
    • compact header names - interesting topic since it is standard and already supported by user agents.
    • body + non-critical headers are gzip'ed and added as new body: interesting feature, but needs further discussions
    • to be continued on the mailing list (link)
  • Fraud detection
    • interesting topic
    • should be able to have a feature to filter dialogs based on severity
    • there should be three different states: normal, warning, critical
    • to be continued on the mailing list (link)
  • Quality-based routing
    • would be nice to have the statistics and decision parts decoupled
    • to be continued on the mailing list (link)
  • Asynchronous processing
    • have a reactor for each process
    • remove the processing from the timer processes
    • the workers will process the time-based tasks
    • to be continued on the mailing list (link)
  • Message handling
    • increase CSEQ for auth is only non-dialog based - it should also be done dialog based?
    • create a topology hiding module and move all the features there
    • topology hiding should work for both INVITE and non-INVITE dialogs
  • Other features
    • asynchronous TLS operations
    • trigger events when dialog states are changed
    • support for multiple events endpoints

IRC Logs

18:00 <@  bogdan_vs>| Hi all !!
18:00 <@  bogdan_vs>| I'm happy  to declare the OpenSIPS meeting started :)
18:00 <    brettnem>| woot!
18:01 <     lirakis>| yay
18:01 <@  bogdan_vs>| during the meeting please let's stick to the meeting related topics and keep other topic for later
18:01 <@  bogdan_vs>| Today's topic is about the features in 1.12 OpenSIPS (the next release)
18:02 <    brettnem>| link for the lazy: http://www.opensips.org/Community/IRCmeeting20140827
18:02 <@  bogdan_vs>| good brettnem :)
18:02 <    Samael28>| Ok. Is it discussion or just press-release? )
18:02 <@  bogdan_vs>| discussion :)
18:02 <     lirakis>| Hi Bogdan, this is Eric from OnSIP.  I know John and I asked you about RTPengine support at cluecon, and I believe you said it would be in 1.12.  I wanted to ask about that again since I dont see it on the topics.
18:03 <@  bogdan_vs>| lirakis : it is not on the topic as it is already done :)
18:03 <     lirakis>| ok! ill have to do a pull and check it out
18:04 <@  bogdan_vs>| https://github.com/OpenSIPS/opensips/tree/master/modules/rtpengine
18:04 <    Samael28>| My request is up to make OpenSIPS mechanism for transparent proxy with registration feature. Means on one side is no registration, on other - it is.
18:04 <@  bogdan_vs>| it is only for 1.12 (current trunk), obviously :)
18:05 <     lirakis>| bogdan_vs, great! thanks very much, i am def. intereste to chekc it out
18:05 <     lirakis>| bogdan_vs, i am also interested in message compression
18:05 <@  bogdan_vs>| on the link brettnem gave, we have the topics we already want to have (but we are not limited to that)
18:05 <     lirakis>| what are the proposed methods for doing that?
18:06 <@  bogdan_vs>| lirakis: we have 2 directions for that
18:06 <@  bogdan_vs>| (1) SIP-wise compacting
18:06 <@  bogdan_vs>| (2) text compression
18:06 <    brettnem>| Perhaps we should just hit those discussion points one by one, gauge interest and implementation methodology?
18:07 <       sekil>| what is the method nowadays to synchronize T_fr_inv_timeout and timer_avp var from cpl-c?
18:07 <   vlad-paiu>| Samael28 : you can already have OpenSIPS register to remote end-points, via the uac_registrant module - see http://www.opensips.org/html/docs/modules/1.11.x/uac_registrant.html
18:07 <@  bogdan_vs>| (1) - header names -> short version ; multiple body hdrs under same hdr ; removing some headers (cleanup)
18:08 <@  bogdan_vs>| (2) body + non-critical hdrs are gzip'ed and added as new body
18:08 <    Samael28>| (1) is good, but only if all SIP traffic passes through proxy
18:08 <    brettnem>| bogdan_vs: I've noticed sometimes when receiving compact headers and where the proxy makes header changes that I get a mix of compact and regular headers in the same message
18:09 <@  bogdan_vs>| brettnem: yeah, right now OpenSIPS adds hdrs only as long format
18:09 <@  bogdan_vs>| but this will be addressed
18:09 <    Samael28>| Also, is gzipped body supports on every equipment?
18:09 <    brettnem>| yeah, I've seen compatibility issues with the mixed types
18:09 <@  bogdan_vs>| Samael28: that will be opensips-2-opensips mainly
18:09 <     lirakis>| hmm - yeah i am interested in compression that does not require UA support
18:09 <    brettnem>| like all compact except Contact.. or maybe all vias are short but the last one added
18:10 <@  bogdan_vs>| in complex system, a message may pass 3-5 instances (SBC, balancers, proxy, presence, etc)
18:10 <     lirakis>| i mean, we are specifically seeing lots of size issues with webrtc stuff now b/c of the huge bodies
18:10 <@  bogdan_vs>| and internally you can route the compressed version
18:10 <    brettnem>| I suspect compressed headers is *only* opensips-opensips.. there isn't a standard that defines compressed headers is there?
18:10 <     lirakis>| brettnem, yeah that is my issue
18:10 <     lirakis>| we dont have a problem opensips-opensips
18:10 <@  bogdan_vs>| brettnem: not I'm aware of...this does not mean it does not
18:10 <     lirakis>| b/c we set our mtu's to be HUGE
18:11 <    Samael28>| What the purpose of shortening? It will reduce bandwidth, but increase CPU
18:11 <    brettnem>| I wonder if it would be worth the effort to propose a standard
18:11 <@  bogdan_vs>| if someone is aware :)....let me know
18:11 <     lirakis>| Samael28, udp fragmentation
18:11 <@  bogdan_vs>| I will give a beer for that ;)
18:11 <    brettnem>| Avoiding packet fragmentation would be really nice
18:11 <     razvanc> | Samael28: simply reduce the size of the SIP messages to avoid udp fragmentation
18:11 <    Samael28>| lirakis, But what about TCP fir signalling?
18:12 <     lirakis>| Samael28, that is not a "solution" for people who use UDP
18:12 <    Samael28>| lirakis, but if it's only to opensips-opensips it can be used
18:13 <@  bogdan_vs>| this msg compression is already work in progress...
18:13 <    Samael28>| btw, it's will be nice to have feature to switch tcp\udp
18:13 <     lirakis>| Samael28, as i said - udp frag is not an issue opensips to opensips for us.  b/c we control the network and can set huge mtu's so things odnt fragment
18:13 <@  bogdan_vs>| at lest addressing (1) and (2)
18:13 <     lirakis>| Samael28, you can already do that, and we do
18:13 <    brettnem>| I can't really image a situation where it's a good idea unless you are doing something with a ton of headers.. webrtc might be a candidate, but other than that, the tradeoffs you normally get with compression I don't think add a whole lot of benefit here..
18:14 <    brettnem>| ie: I don't think you'll get more IO from compression, bandwidth is cheap and it takes a lot of signaling to use of a significant amount of bandwidth
18:14 <     lirakis>| yeah - i dont see significant value in doing method (2) with gzip of the body
18:14 <@  bogdan_vs>| brettnem : presence ? :)
18:14 <    brettnem>| well
18:14 <    brettnem>| presence is typically an end device notification.. typically, yes?
18:14 <     lirakis>| unless the user agents supports it
18:14 <    brettnem>| and that device isn't going to support what you are proposing
18:14 <@  bogdan_vs>| sometime you may have really large bodies - even for media, if you have audio and video, etc
18:15 <    Samael28>| webrtc mostly relies on backend processing, mainly most of body is not processed by opensips, but freeswitch for ex
18:15 <    brettnem>| yes, I've seen RTP offers so big the packet gets fragmented
18:15 <     lirakis>| yeah - we are doing TONS of webrtc, and the bodies are huge
18:15 <@  bogdan_vs>| brettnem: once again, such compression is good for piping through your platform messages (where parts of them you do not need to understand)
18:15 <    brettnem>| I think the *only* advantage of compression is avoiding packet fragmentation
18:15 <     lirakis>| but having them compressed between proxies is not valueable to us
18:15 <     lirakis>| i agree with brettnem
18:16 <@  bogdan_vs>| it may be because of parsing ;)
18:16 <     lirakis>| unless the useragent supports decompression
18:16 <     lirakis>| i dont see value
18:16 <     lirakis>| i want to reduce packet size end to end
18:16 <     lirakis>| not just between proxies
18:16 <@  bogdan_vs>| more than 40% CPU time (used by opensips) is becuase of parsing and building SIP messages
18:16 <    Samael28>| lirakis, it's about to your ua settings
18:16 <    brettnem>| there are some cases where compression gains processing IO (like storage engines because of IO bottlenecks).. but in this case, you won't see that kind of performance improvement
18:16 <@  bogdan_vs>| if you have a message with firstline, one VIA, Callid, cseq, FROM + TO and body
18:16 <     lirakis>| Samael28, ... im not sure you are getting the point of this discussion
18:17 <@  bogdan_vs>|  that will be really fast on handling (parsing and routing)
18:17 <    brettnem>| bogdan_vs: you'll trade off a simple parse with a decompress + parse
18:17 <     lirakis>| bogdan_vs, i definitely like the compressing of multi line headers into single line
18:17 <   vlad-paiu>| lirakis: are you aware of any standards / RFCs that mandate such end-to-end compression - and if any UAs support things like this ?
18:17 <     lirakis>| vlad-paiu, i know apple does gz compression on message bodies - but I know of no RFC relating to it off the top of my head
18:17 <    brettnem>| if there are a lot of "non-essential" headers that you can ignore, there might be benefit I suppose, but it also might end up being a lot of dev work with no real benefit
18:18 <@  bogdan_vs>| brettnem: you need to do compress/decompress only at the edge of the platform
18:18 <     lirakis>| i would generalyl be against removing "non-essential" headers
18:18 <@  bogdan_vs>| inside the platform, you may visit couple of SIP entities
18:18 <     lirakis>| i dont want opensips to start acting like a broken ALG
18:18 <     lirakis>| and stripping out arbitrary headers
18:18 <    brettnem>| btw, I'd just like to throw this out.. If we do anything with "non-essential" headers, I think we need to *explicitly* define the essential headers in the docs somewhere..
18:18 <     lirakis>| ^
18:19 <    brettnem>| lirakis: I think the point is allowing the SIP parser in OpenSIPs to *ignore* non-essentials
18:19 <@  bogdan_vs>| it is not stripping -> just add them into the compressed body
18:19 <@  bogdan_vs>| it is up to you what to be compressed or removed
18:19 <    brettnem>| to save parse time.. i.e.: here's 100 "extra" headers the proxy doesn't need.. compress them, and allow opensips to pass it transparently instead of parsing head header
18:20 <    Samael28>| And what is considered to be "non-essentials"? I'm using SIP headers to pass some variables from end-to-end
18:20 <    brettnem>| I don't see this as a function opensips needs to support.. you can already add a compressed body, right?
18:20 <      liviuc>| the compression module may document some built-in allowed headers, and the user may overwrite these in the script
18:21 <    brettnem>| or are we talking about adding a *SIP* header like X-CompressedData: <binary data>
18:21 <@  bogdan_vs>| OK, let's do this way - as dissecting each feature may take time, let's try to identify and we can do more in depth on mailing list
18:21 <@  bogdan_vs>| so "compression" look interesting, but needs a lot of discussion - so we can open a thread for that on mailing list, to sort it out
18:21 <@  bogdan_vs>| what do you thing guys ?
18:21 <    brettnem>| For *compact* header support, I'd like to see the platform be able to normalize if the packet is compact or not.. Even if adding or changing headers (contact, from, to, etc)
18:22 <@  bogdan_vs>| should we do it this way, to be able to cover more topics / features here ?
18:22 <     lirakis>| bogdan_vs, i agree - i think there needs to be a broader discussion
18:22 <     lirakis>| bogdan_vs, its definitely interesting though
18:22 <@  bogdan_vs>| that's my points
18:22 <    brettnem>| Compression is interesting, but the implementation is highly dependent on the business need and supported endpoints
18:22 <@  bogdan_vs>| well, you cannot make everybody happy :D
18:22 <@  bogdan_vs>| but I see cases something like that will be useful
18:23 <    brettnem>| right so we need to come up with a generic way to.... "Transport compressed metadata across a SIP proxy"
18:23 <@  bogdan_vs>| ok, so this is a good topic - agreed
18:23 <    brettnem>| sure
18:23 <     lirakis>| bogdan_vs, yes - i was agreeing with you, i think it should be discussed on the mailling list
18:23 <@  bogdan_vs>| we will summarize on emial and carry on there
18:23 <@  bogdan_vs>| perfect, done with it for the moment :)
18:23 <    brettnem>| fwiw, it's not high on my priority need. I think it would be interesting to see other people's priorities.
18:24 <     lirakis>| its not HIGH priority to me either, but an interesting topic
18:24 <    brettnem>| Header *compaction* however is a bit higher on my need since normalizing signaling always leads to better endpoint compatibility
18:24 <     lirakis>| i would agree, header compaction is higher
18:24 <@  bogdan_vs>| brettnem, well let's see first what's on the table, before talking about priorities
18:24 <    brettnem>| bogdan_vs: sure :)
18:24 <@  bogdan_vs>| yes, compacting has the big plus as it should be UAC supported
18:25 <@  bogdan_vs>| ok, next....
18:25 <@  bogdan_vs>| Fraud detection ?
18:25 <     lirakis>| yeah
18:25 <    Samael28>| Yes
18:26 <     lirakis>| so how is fraud detection really different than pike
18:26 <@  bogdan_vs>| it is...
18:26 <     lirakis>| is it basically pike, but looking at the ruri instead of ip
18:26 <    Samael28>| I've implement it via own, analyze SDP bodies and iptables rules + fail2ban
18:26 <@  bogdan_vs>| we want to do fraud detection based on dialing patterns
18:26 <     lirakis>| right
18:27 <@  bogdan_vs>| a pattern means - where you dial (as destinations), how often, parallel calls
18:27 <@  bogdan_vs>| to see if the user behaves inside the pattern or not
18:27 <     lirakis>| right
18:27 <     lirakis>| ok
18:27 <@  bogdan_vs>| and you can assign a pattern to each users
18:27 <    brettnem>| this is highly implementation dependent
18:27 <@  bogdan_vs>| and get notifications when the thresholds are exceeded
18:28 <    Samael28>| user will be registered user or some predefined dest?
18:28 <@  bogdan_vs>| any kind of user...
18:28 <    Samael28>| Like ip address
18:28 <     lirakis>| registration is not related to outbound in anyway
18:28 <@  bogdan_vs>| let's say, any entity sending traffic - registered user, trunk, gw, etc
18:28 <    Samael28>| and what for inbound?
18:28 < FlavioEGonc>| There are some identifiable dialing patterns for fraud
18:29 <     lirakis>| i think the way bogdan_vs is talking about it makes sense - create your own profile
18:29 < FlavioEGonc>| simultaneous calls for the same dialed number
18:29 <     lirakis>| get an alert when usage goes out side that profile
18:29 < FlavioEGonc>| calls per minute in an international route and many others.
18:29 <    Samael28>| Yes, it will be perfect. But what to do via treshhold? Block? Send mail?
18:29 <    brettnem>| *raise event*
18:29 <     lirakis>| ^
18:29 < FlavioEGonc>| The action is up to the script writer.
18:30 <    brettnem>| Make it an event, and the action can be performed in an event route
18:30 <@  bogdan_vs>| yes, you can intercept the event in script, if you want, or have an externa app consuming it
18:30 <    brettnem>| yeah
18:30 <    brettnem>| not sure how you generically can define "dialing patterns"
18:30 <@  bogdan_vs>| for defining a pattern, we are looking at:
18:30 <    brettnem>| I'd like the idea of spam assassin for voip
18:31 <    brettnem>| like certain dialing "actions" are worth "spam points"
18:31 <@  bogdan_vs>| (1) dialed prefixes - gray destinations
18:31 <    brettnem>| and X spam points, raise event
18:31 <@  bogdan_vs>| (2) number of simulations calls per destination
18:31 <@  bogdan_vs>| (3) number of total calls per destination
18:31 <    brettnem>| so for example, each of bogdan's examples here is worth X points
18:31 <    brettnem>| and if your score is over Y points total, then the event is raised
18:31 <     lirakis>| i think raising an event is good - that way the script writer can trap it, and do whatever
18:31 <     lirakis>| yea
18:32 <    brettnem>| calling a grey destination itself isn't so bad
18:32 <    brettnem>| calling it with 100 ports might be bad
18:32 <@  bogdan_vs>| each may get evaluated to 3 states - (a) normal (b) warning (c) critical
18:32 <    brettnem>| calling it with 100 ports at 100 cps would be really bad
18:32 <@  bogdan_vs>| if at least one critical -> stop the call
18:32 <@  bogdan_vs>| if you have more than "x" warnings -> block
18:32 <@  bogdan_vs>| something like that
18:32 <     lirakis>| bogdan_vs, i would just have different events
18:33 <     lirakis>| and leave the behavior up to the script writer
18:33 <@  bogdan_vs>| yes, everything will be translated into events
18:33 <    brettnem>| would be nice to set something on the dialog to indicate the dialog's threshold.. so you could do something like "give me all dialogs that are in warning"
18:33 <@  bogdan_vs>| true
18:33 < FlavioEGonc>| It could work exactly like pike or rl_check, a function called in the script (positive or negative) the script writer decides what to do.
18:33 <     lirakis>| E_fraud_warining, E_fraud_critical
18:34 <@  bogdan_vs>| yes, + some info about the exceeded param (in profile)
18:35 <    brettnem>| yeah
18:35 <    Samael28>| great
18:35 <@  bogdan_vs>| I guess we can do the same with this topics - agreed on the big picture and have it moved on the mailing list
18:35 <@  bogdan_vs>| for further detailed discussions
18:35 <     lirakis>| bogdan_vs, agreed
18:35 <    brettnem>| I have a general question about fraud
18:36 <@  bogdan_vs>| ?
18:36 <    brettnem>| does anyone see any benefit to having a sort of repo of "bad actors"
18:36 <    brettnem>| like for example
18:36 <    brettnem>| UA => sipvicious
18:36 <    brettnem>| known bad IPs
18:36 <     lirakis>| i dunno
18:36 <    brettnem>| known bad ANIs
18:36 <     lirakis>| bad ip's is already tracked some where
18:36 <@  bogdan_vs>| Flavio had some ideas on that....
18:36 <     lirakis>| bogus UA are pretty small
18:36 <    brettnem>| I know a lot of that is subjective, I'm thinking of something like "Razor" for email
18:37 <     lirakis>| sipvicous, freindly-scanner, vaxasip
18:37 <    brettnem>| right
18:37 <     lirakis>| those are the only scanners i really know of
18:37 <    Samael28>| Made is by own... sipvicious/sipcli + http://voipbl.org/
18:37 < FlavioEGonc>| Yes, we are keeping a database of bad UAs, bad Dialed Numbers and bad IPs
18:37 <@  bogdan_vs>| FlavioEGoncalves> you were planning to build a centralize DB with info on attackers (IPs, clients, etc)
18:37 <    brettnem>| I think I also saw something along these lines in a kamailio module somewhere
18:37 <     lirakis>| FlavioEGoncalves, is that public?
18:37 < FlavioEGonc>| No it is not public,
18:38 <    brettnem>| Samael28: who is voipbl?
18:38 < FlavioEGonc>| Some sources of the dialed numbers are paid.
18:39 <    brettnem>| It might be interesting to have a community list and a module that works with the community list format
18:39 <    Samael28>| brettnem, canadian guys, have banlist of ip's
18:39 < FlavioEGonc>| But it is something that we are trying to have sponsorship to offer.
18:39 <    brettnem>| the only problem with that is that it's totally subjective and at the list maintainers discretion as to what's on it.
18:39 <     lirakis>| i agree with brettnem
18:39 <     lirakis>| i dont think there is GREAT use in a generic list
18:40 <@  bogdan_vs>| ok, parking this one for the list ?
18:40 <     lirakis>| ^
18:40 <    brettnem>| (thumbsup)
18:40 < FlavioEGonc>| Ok
18:40 <@  bogdan_vs>| cool....moving forward :)
18:40 <@  bogdan_vs>| Quality based Routing ?
18:41 <@  bogdan_vs>| This is based on Flavio's idea...
18:41 <    Samael28>| Example? Routes from QoS info?
18:41 <@  bogdan_vs>| for Dynamic Routing, to be able to order GW (inside a rule, for a prefix) based on quality
18:41 <     lirakis>| so ... if you already are using dispatcher
18:41 < FlavioEGonc>| The basic idea is to be able to extend DROUTING to allow rules regarding the sigaling quality of the call.
18:41 <@  bogdan_vs>| opensips, based on calls, can learn the quality (ASR, PDD, CCR, ACD, etc)
18:41 <    brettnem>| FlavioEGoncalves: define "quality"
18:41 <      liviuc>| "quality" i.e. meeting a set of SLA parameters
18:42 <    brettnem>| ok so
18:42 <    brettnem>| I can see a specific implementation for drouting here
18:42 <    brettnem>| however
18:42 < FlavioEGonc>| The first idea is to keep counters for ACD, PDD, ASR and CCR of the gateways
18:42 <@  bogdan_vs>| quality can be also RTP based, if we can collect something from that level
18:42 <    brettnem>| I think it would be interesting to have a module do nothing but perform the counters (ACD,PDD,ASR, CCR, etc)
18:42 <@  bogdan_vs>| like stats in BYE or info from RTPProxy or others
18:42 <    brettnem>| treated like a dynamic pipe like ratelimit
18:43 <    brettnem>| I think that would be fantastic really
18:43 <    Samael28>| Maybe better to provide some mechanism to collect MOS on destinations?
18:43 <    brettnem>| bah, easy there with MOS :)
18:43 <    brettnem>| what I'm thinking is to build those stats in pipes
18:43 <    Samael28>| Hard q, I know )
18:43 < FlavioEGonc>| Once we have this data, it would be interesting to have some parameters for do_routing such as minimum ASR, maximum PDD, minimum CCR
18:43 <    brettnem>| then in drouting, address those pipe names for each gateway/carrier
18:44 <    brettnem>| so you can dynamically create stats for any "pipe"
18:44 < FlavioEGonc>| and also minimum number of samples to start applying the rules.
18:44 <    brettnem>| and then reuse those pipe stats for something else
18:44 <    brettnem>| like the min/max stats values in drouting
18:44 <    brettnem>| FlavioEGoncalves: yes, that's really important
18:44 <    brettnem>| also maybe a weight for the different parameters.. low ASR might not count against you as much as high PDD
18:44 <    Samael28>| And what is practice case for this?
18:45 <    brettnem>| Almost everyone doing trunks needs to know these stats anyway
18:45 <@  bogdan_vs>| :D
18:45 <    brettnem>| being able to pull them right off the switch, say in a fifo is very useful
18:45 < FlavioEGonc>| Automatic gateway selection based not only in the position in the list, but also in the minimum quality required.
18:46 <    brettnem>| and then if the switch could actually reorder gateways based on "performance" you have a very powerful tool to maintain high quality built *right in* at the switch
18:46 <    brettnem>| today these tricks are played with post-processing CDRs in all sorts of magic we have to do behind the scenes :)
18:46 < FlavioEGonc>| It would be easy to an administrator in a route select a gateway based on CLI/NOCLI minimum ASR, maximum PDD, min number of samples...
18:47 <    brettnem>| I personally really like the idea, but I'd like to see it implemented in a generic way that allows the raw stats to be used on their own.. for stats collection
18:47 <    brettnem>| probably need some sort of sliding window and concept of period as well.. which will complicate the collection mechanism
18:47 <    brettnem>| ie: last 15 minute ASR
18:47 <@  bogdan_vs>| brettnem> like one module to gather statistics for some destination and another module to interpret them and push changes in DR ?
18:48 < FlavioEGonc>| <brettnem>Interesting concept.
18:48 <    brettnem>| yeah, or if you just use the collection one.. you can just pull via fifo for whatever
18:48 <    brettnem>| there could be another module.. for example that performs events when X stat reaches Y val
18:49 < FlavioEGonc>| It would be interesting also to have a function to get the string of all gateways matching the condition to further sort based on cost by an external algorithm.
18:49 <    brettnem>| SORTING
18:49 <    brettnem>| I would love some internal sorting... hash based sorting
18:49 <    brettnem>| :D :D :D
18:50 <    brettnem>| FlavioEGoncalves: seriously tho, I'd see getting the stat as one operation and then SORT being a transformation on that dataset
18:50 <    brettnem>| since "sort" in of itself is very useful
18:51 <@  bogdan_vs>| ok, agreed on the interest of this topic, moving on the mailing list for more details ?
18:51 < FlavioEGonc>| Ok,
18:51 <    brettnem>| sure
18:52 <@  bogdan_vs>| last, at lest on other side is "Async IO in cfgscript"
18:52 <@  bogdan_vs>| this a quite heavy one, at least as implementation
18:52 <@  bogdan_vs>| the idea is to allow you, from script, to perform async I/Os , like DB, REST, exec
18:52 <    brettnem>| I am *extremely* interested in this one
18:53 <@  bogdan_vs>| the resume is done in a "resume_route", like TM does for failures :)
18:53 <    brettnem>| and I think this might have the largest impact on the project as a whole.. in terms of pushing the limits of what the core can do
18:53 <@  bogdan_vs>| so, it will be based on TM (to store the context in transaction) and allow you to resume later in a different route
18:53 <     lirakis>| i do think its interesting, but i can only see things like avp_dbops being greatly benefited
18:53 <@  bogdan_vs>| it does - I'm already working on that - adding one I/O reactor per process
18:54 <    brettnem>| opensips at it's core is a router.. designed to take a call and figure out what to do with it.
18:54 <     lirakis>| we run into issues with other modules like presence etc. doing dbops that block
18:54 <    brettnem>| as we design more complicated route scenarios, the business logic to determine where to send a call blocks the children
18:54 <@  bogdan_vs>| next step - saving context in TM....last step, reasuming in script
18:54 <    brettnem>| and ultimately limits the usefulness of the product
18:54 <    brettnem>| IF we can avoid clogging up the children by allowing longer running lookups to occur in the background, we can get much more performance out of a single instance
18:55 <@  bogdan_vs>| this will have a huge impact on the internal structure of OpenSIPS - timers will need to be reworked...but these are implementation "details" :)
18:55 <@  bogdan_vs>| but overall, timers will work better as they will not be "encapsulated" in a single process
18:55 <@  bogdan_vs>| they will be distributed
18:56 <     lirakis>| bogdan_vs, sure but the details are important obv.  we are already specifically giving a dedicated timer process to everymodule with the option of a dedicated timer process
18:56 <    brettnem>| I have frequently had to troubleshoot instances of opensips that were dropping packets because of backend latency (typically databases). Variation in backend speeds made for unpredictable performance which makes everyone nervous
18:56 <   Samael28_>| May be su huge rework is left to 2.0 ver?
18:56 <    brettnem>| please don't wait for 2.0 for this :D
18:56 <@  bogdan_vs>| Samael28_> some of the work is imported from 2.0
18:57 <     lirakis>| bogdan_vs, i think that providing the dedicated timer process option to more modules would be a great thing to do
18:57 <     lirakis>| b/c that causes lots of blocking issues on the current setup
18:57 <@  bogdan_vs>| the plan for 2.0 changed a "bit" - having a completly different version (2.0) is a no go (as devel resources and acceptance)
18:57 <   Samael28_>| Means how much will be broken with suchrework inexisting conf?
18:57 <     lirakis>| i mean - it gives you a lot of this "for free"
18:57 <     lirakis>| *kind of*
18:57 <      saghul>| but if there would be a reactor running on every process, timers could be added there as well perhaps?
18:58 <@  bogdan_vs>| so the plan is to keep the "2.0" as experimental pad....and what is successful will be pushed, piece by piece, into main stream code
18:58 <      liviuc>| the async changes should leave the script backwards-compatible
18:58 <     lirakis>| ^
18:58 <      liviuc>| since the db functions will be prefixed with async_*
18:58 <     lirakis>| and all the timers
18:59 <@  bogdan_vs>| saghul> that's the plan..the timer procs wil just generate the timer jobs and have them distributed to the reactors in all the other procs
18:59 <     lirakis>| so ... can you not just have a seperate process running this async stuff, that has callbacks triggered to drop into "route_X"
18:59 <     lirakis>| and not muck with timers in general?
18:59 <     lirakis>| im confused why the timers would have to be brought into this
18:59 <      saghul>| bogdan_vs: I meant, each process' reactor could do timers as well, instead of a separated process
18:59 <    brettnem>| probably because for a specific transaction, there are a handful of timers that operate within that transaction
19:00 <@  bogdan_vs>| theoretically yes, right now there are too many things changing and we need to take them one by one...
19:00 <    brettnem>| suspending the transaction means that those timers still need to operate, just not in that transaction anymore.. what happens if you hit a SIP timer while you are suspended
19:00 <     lirakis>| if you did not bind it to the tm module though
19:00 <@  bogdan_vs>| the plan is to have only "worker" procs doing handling of SIP msg...
19:00 <@  bogdan_vs>| no handling in timer procs
19:01 <     lirakis>| and instead built it off callbacks on this new async module
19:01 <      saghul>| i see
19:01 <    brettnem>| ...kamailio does some of this async suspend/resume stuff.. <sigh> I haven't used it yet, but it might be interesting to see how they implemented it..
19:01 <     lirakis>| so ... if it is based on the tm module
19:01 <      saghul>| IIRC they have a separate process that runs a reactor
19:01 <@  bogdan_vs>| lirakis, the timers are not directly related to async....
19:01 <     lirakis>| opensips is "suspending" the transaction
19:02 <     lirakis>| but the UAC isnt
19:02 <@  bogdan_vs>| we just need to change the timer implementation in order to make async possible
19:02 <     lirakis>| so you are still going to have to do the operation within the transaction timeout
19:02 <     lirakis>| so ... why bother with suspending the transaction?
19:03 <     lirakis>| not sure if my question makes sense
19:03 <@  bogdan_vs>| on an I/O op, I will use the transaction as context...to be able to resume later when the I/O is done
19:03 <     lirakis>| right
19:03 <     lirakis>| i understand saving the transaction context
19:03 <     lirakis>| if you had a different implementation
19:03 <@  bogdan_vs>| the timers (as procs) will not execute any more tasks, but just trigger them
19:04 <     lirakis>| i guess the mechanisms im thinking about
19:04 <     lirakis>| are ... lesss refined
19:04 <     lirakis>| just using a different proc for the async, and having callbacks registered that trigger a defined route
19:05 <     lirakis>| where it restores the transaction context
19:05 <     lirakis>| and not caring about suspending the transaction
19:05 <     razvanc> | lirakis: there will not be a single specialized process
19:05 <@  bogdan_vs>| no, there is no different proc for asyng
19:05 <     lirakis>| right - i am suggesting a different implementation
19:05 <@  bogdan_vs>| that will be ahuge limitation
19:06 <     lirakis>| if you litereally had an "async_op" module
19:06 <     lirakis>| that spun up dedicated procs
19:06 <@  bogdan_vs>| it will become a bottle neck, not to mention that you will have to keep moving resume contexts between procs
19:06 <     lirakis>| hmm
19:08 <     lirakis>| yeah - i wasnt suggesting a single proc
19:08 <@  bogdan_vs>| anyhow, these are really complex internal-design related things
19:08 <     lirakis>| sure - i understand
19:08 <@  bogdan_vs>| the most important is, as functionality, there is a need and agreement on how to work
19:09 <     lirakis>| no - i know, but the internals are important to me as well, sorry for derailing if the focus is on functionality
19:10 <@  bogdan_vs>| lirakis, we can go deeper on the mailing list...
19:10 <     lirakis>| right ;)
19:10 <@  bogdan_vs>| the details are a bit difficult to explain over IM :)
19:10 <     lirakis>| yeah - i think i am just missing some thing
19:10 <     lirakis>| haha
19:12 <     razvanc> | saghul: can you elaborate a bit the async TLS requirements?
19:13 <@  bogdan_vs>| ok, in the mean while, another topic :
19:13 <@  bogdan_vs>| extending message mangling
19:14 <@  bogdan_vs>| Vlad recently added CSEQ increase for auth
19:14 <@  bogdan_vs>| this can be done dialog and non-dialog based
19:14 <   Samael28_>| Oh.... Is there any man for it? )))
19:14 <     saghul> | razvanc: pong
19:14 <@  bogdan_vs>| also topology hiding will become a separate module - it could work on top of dialog or without it
19:15 <      saghul>| right now TCP is async (connect, write)
19:15 <      saghul>| but not TLS
19:15 <@  bogdan_vs>| and we have callid changing (for SIP TH) - we will move this into the TH module
19:16 < FlavioEGonc>| Would TH be easier to implement and compatible with NAT traversal?
19:17 <@  bogdan_vs>| FlavioEGoncalves> the idea is to have TH as standalone and work for both for INVITE dialogs and non-dialog INVITE
19:17 <@  bogdan_vs>| so, a wider coverage :)
19:17 <@  bogdan_vs>| not sure what;s the NAT traversal relation
19:17 < FlavioEGonc>| Ok, right now it is easy to mess with sequential request when using TH.
19:18 < FlavioEGonc>| I had some cases of looping sequential requests,
19:18 <@  bogdan_vs>| maybe we need to put in place a small tutorial
19:19 <@  bogdan_vs>| at the end , you need to use 2 functions, but the trick is to know where :D
19:19 < FlavioEGonc>| <@bogdan_vs> True
19:20 <   vlad-paiu>| saghul : ok - will take a look at it , also adding it to priorities for 1.12
19:20 <@  bogdan_vs>| so, any wish list from other people ? :D
19:20 < FlavioEGonc>| Attended transfers in B2BUA
19:20 <@  bogdan_vs>| we are open to suggestions, do not mis the opportunity :)
19:20 <    brettnem>| FlavioEGoncalves: what was your issue with looping sequential requests? I think I saw something like that
19:21 <    brettnem>| bogdan_vs: I'd like to see events triggered on dialog state changes
19:21 <@  bogdan_vs>| FlavioEGoncalves> yes, I have that on the list already
19:22 < FlavioEGonc>| <brettnem> It is now solved, but when you receive a bad request from an UA there are some looping situations, I don't remember exactly the problem, happened three months ago.
19:22 <     razvanc> | brettnem: sure, we'll add this too on the wishlist for 1.12
19:22 <    brettnem>| FlavioEGoncalves: what was the fix for it.. I occaisionally get ACKs that loop to self (on loopback)
19:22 < FlavioEGonc>| regarding attended transfers there are actually two thing that needs to be addressed.
19:23 <    brettnem>| and.. I'd like to see rabbit natively support multiple endpoints with automatic retry and re-enable :D
19:24 < FlavioEGonc>| <brettnem>I think I ran a calidate and fix_dialog aftter match_dialog.
19:24 <     razvanc> | brettnem: as we've discussed, that's also on our list
19:24 < FlavioEGonc>| <brettnem>validate I mean
19:24 <    brettnem>| FlavioEGoncalves: did yours look like that? ACKs looping on loopback?
19:25 < FlavioEGonc>| <brettnem>Yes ACKs in loopback, 100% of cpu.
19:25 <@  bogdan_vs>| FlavioEGoncalves> send an kick-off email on the transfer topic, we can sort it out via list
19:26 <    brettnem>| FlavioEGoncalves: cool, thanks I'll give that a shot.. So you validate + fix AFTER match? hum
19:26 < FlavioEGonc>| <@bogdan_vs>Actually the current REFER scenario works for attended transfers
19:27 <@  bogdan_vs>| in the mean while, call for ideas......do not be shy
19:28 <@  bogdan_vs>| nothing more ?
19:28 <      saghul>| vlad-paiu: fantastic, thanks!
19:28 <    brettnem>| hrm
19:28 <@  bogdan_vs>| just as an heads-up - next meeting will be on webRTC


Page last modified on September 03, 2014, at 05:19 PM