About |
About.PerformanceTests-3-4-Listing HistoryHide minor edits - Show changes to markup May 10, 2023, at 01:24 PM
by
- Added lines 166-167:
The objective here was to compare the TCP and UDP engines as part of a testing scenario with a high volume of SIP traffic. As the nature of the SIP scenario was not too relevant, we went with the simplistic T1 setup, of simply performing t_relay() from UAC to UAS, effectively stress-testing the TCP engine to the limit. Added lines 170-175:
In this test, the UAC/UAS sipp instances were configured in single-connection mode, routing all calls through a single TCP connection. The actual script used for this scenario can be found here. Find the test results under the TCP Engine Tests table, where this particular test is marked as test 14.1. Added lines 178-179:
WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed. Added lines 182-183:
WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed. Added lines 186-191:
In this test, the UAC/UAS sipp instances were configured in multi-connection mode, routing all calls through multiple TCP connections. Overall, OpenSIPS was managing TCP connections in the order of thousands. The actual script used for this scenario can be found here. Find the test results under the TCP Engine Tests table, where this particular test is marked as test 14.4. Changed lines 194-198 from:
[T14.6] N Connections, parallel read: 2to:
WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed. [T14.6] N Connections, parallel read: 2WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed. May 10, 2023, at 01:16 PM
by
- Changed line 85 from:
In the test, OpenSIPS used the cachedb_local module in order to do less database queries. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help. to:
Building on the previous test, one critical change was made: we used the OpenSIPS cachedb_local module in order to fully eliminate DB queries during call setup. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help. May 10, 2023, at 01:14 PM
by
- Changed lines 51-52 from:
[T5.2] Default Script with dialog support + Topology Hidingto:
[T5.2] Default Script with dialog + Topology HidingChanged line 63 from:
[T6] Default Script with dialog support and authenticationto:
[T6] Default Script with dialog + authenticationMay 10, 2023, at 01:13 PM
by
- Added lines 8-9:
Basic Calling ScenariosChanged lines 136-176 from:
Find the test results under the Authenticated Calls table, where this particular test is marked as test 12. to:
Find the test results under the Authenticated Calls table, where this particular test is marked as test 12. Advanced Calling Scenarios[T13.1] B2B Topology HidingThe purpose of this test was to gauge the performance difference of dialog-based topology hiding vs. B2B topology hiding. The actual script used for this scenario can be found here. Find the test results under the Complex Calling Scenarios table, where this particular test is marked as test 13.1. [T13.2] B2B REFER HandlingThe purpose of this test was to assess the maximum CPS possible in the B2B REFER scenario. The actual script used for this scenario can be found here. Find the test results under the Complex Calling Scenarios table, where this particular test is marked as test 13.2. [T13.3] B2B Marketing ScenarioThe purpose of this test was to assess the maximum CPS possible in the B2B Marketing scenario. The actual script used for this scenario can be found here. Find the test results under the Complex Calling Scenarios table, where this particular test is marked as test 13.3. TCP Tests[T14.1] Single Connection, parallel read: OFF[T14.2] Single Connection, parallel read: 1[T14.3] Single Connection, parallel read: 2[T14.4] N Connections, parallel read: OFF[T14.5] N Connections, parallel read: 1[T14.6] N Connections, parallel read: 2May 10, 2023, at 01:06 PM
by
- Changed lines 116-117 from:
[T11] DB_Flatstore CDR accounting + Auth Cachingto:
[T11] DB_Flatstore CDR accountingChanged line 128 from:
[T12] DB_Flatstore CDR accountingto:
[T12] DB_Flatstore CDR accounting + Auth CachingMay 10, 2023, at 01:06 PM
by
- Changed lines 116-117 from:
[T11] DB_Flatstore accounting + Auth Cachingto:
[T11] DB_Flatstore CDR accounting + Auth CachingChanged line 128 from:
[T12] DB_Flatstore accountingto:
[T12] DB_Flatstore CDR accountingMay 10, 2023, at 01:06 PM
by
- Changed line 91 from:
[T9] Accountingto:
[T9] CDR AccountingMay 10, 2023, at 01:05 PM
by
- Changed lines 72-73 from:
[T7] Default Script with dialog + authentication (10k sub)to:
[T7] Default Script with dialog + authentication (10k sub)Changed lines 81-82 from:
[T8] Subscriber cachingto:
[T8] Subscriber cachingChanged lines 91-92 from:
[T9] Accountingto:
[T9] AccountingChanged lines 104-105 from:
[T10] CDR accounting + Auth Cachingto:
[T10] CDR accounting + Auth CachingChanged lines 116-117 from:
[T11] DB_Flatstore accounting + Auth Cachingto:
[T11] DB_Flatstore accounting + Auth CachingChanged line 128 from:
[T12] DB_Flatstore accountingto:
[T12] DB_Flatstore accountingMay 10, 2023, at 01:05 PM
by
- Changed lines 60-61 from:
Default Script with dialog support and authenticationto:
[T6] Default Script with dialog support and authenticationChanged lines 65-115 from:
The actual script used for this scenario can be found here. In this scenario we stopped the test at 6000 CPS with an average load of 26.7 % ( actual load returned by htop ) See chart, where this particular test is marked as test F. 10k subscribersThis test used the same script as the previous one, the only difference being that there were 10 000 users in the subscribers table. The purpose of this test was to see how the USRLOC module scales with the number of registered users. In this scenario we stopped the test at 6000 CPS with an average load of 30.3 % ( actual load returned by htop ) See chart, where this particular test is marked as test G. Subscriber cachingIn the test, OpenSIPS used the localcache module in order to do less database queries. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help. The actual script used for this scenario can be found here. In this scenario we stopped the test at 6000 CPS with an average load of 18 % ( actual load returned by htop ) See chart, where this particular test is marked as test H. AccountingThis test had OpenSIPS running with 10k subscribers, with authentication ( no caching ), dialog aware and doing old type of accounting ( two entries, one for INVITE and one for BYE ). The purpose of this test was to see the performance penalty introduced by having OpenSIPS do the old type of accounting. The actual script used for this scenario can be found here. In this scenario we stopped the test at 6000 CPS with an average load of 43.8 % ( actual load returned by htop ) See chart, where this particular test is marked as test I. CDR accountingIn this test, OpenSIPS was directly generating CDRs for each call, as opposed to the previous scenario. The purpose of this test was to see how the new type of accounting compares to the old one. The actual script used for this scenario can be found here. In this scenario we managed to achieve 6000 CPS with an average load of 38.7 % ( actual load returned by htop ) See chart, where this particular test is marked as test J. CDR accounting + Auth CachingIn the last test, OpenSIPS was generating CDRs just as in the previous test, but it was also caching the 10k subscribers it had in the MYSQL database. In this scenario we stopped the test at 6000 CPS with an average load of 28.1 % ( actual load returned by htop ) See chart to:
The actual script used for this scenario can be found here. Find the test results under the Authenticated Calls table, where this particular test is marked as test 6. [T7] Default Script with dialog + authentication (10k sub)This test used the same script as the previous one, the only difference being that there were 10.000 users in the subscribers table. The purpose of this test was to see how the USRLOC module scales with the number of registered users. Find the test results under the Authenticated Calls table, where this particular test is marked as test 7. [T8] Subscriber cachingIn the test, OpenSIPS used the cachedb_local module in order to do less database queries. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help. The actual script used for this scenario can be found here. Find the test results under the Authenticated Calls table, where this particular test is marked as test 8. [T9] AccountingThis test had OpenSIPS running with 10k subscribers, with authentication ( no caching ), dialog aware and doing CDR accounting. The purpose of this test was to see the performance penalty introduced by having OpenSIPS do both READ and WRITE queries to the database. The actual script used for this scenario can be found here. Find the test results under the Authenticated Calls table, where this particular test is marked as test 9. [T10] CDR accounting + Auth CachingIn this test, OpenSIPS was generating CDRs just as in the previous test, but it was also caching the 10k subscribers it had in the MYSQL database. The actual script used for this scenario can be found here. Find the test results under the Authenticated Calls table, where this particular test is marked as test 10. [T11] DB_Flatstore accounting + Auth CachingA similar test to T9, however the DB backend used to write the CDR is db_flatstore, writing them directly to disk as CSV records in an optimized fashion. The actual script used for this scenario can be found here. Find the test results under the Authenticated Calls table, where this particular test is marked as test 11. [T12] DB_Flatstore accountingThis last test completely eliminated the need for querying MySQL during call setup, as we now also pre-cached the authentication information for all 10k subscribers. The actual script used for this scenario can be found here. Find the test results under the Authenticated Calls table, where this particular test is marked as test 12. May 10, 2023, at 12:47 PM
by
- Changed lines 8-9 from:
1. Minimal Stateful SIP Proxyto:
[T1] Minimal Stateful SIP ProxyChanged lines 17-18 from:
2. Stateful proxy with loose routingto:
[T2] Stateful proxy with loose routingChanged lines 21-28 from:
The actual script used for this scenario can be found here . In this scenario we stopped the test at 12000 CPS with an average load of 20.6 % ( actual load returned by htop ) See chart, where this particular test is marked as test B. Stateful proxy with loose routing and dialog supportto:
The actual script used for this scenario can be found here . Find the test results in this table, where this particular test is marked as test 2. [T3] Stateful proxy with loose routing and dialog supportChanged lines 29-36 from:
The actual script used for this scenario can be found here . In this scenario we stopped the test at 9000 CPS with an average load of 20.5 % ( actual load returned by htop ) See chart, where this particular test is marked as test C. Default Scriptto:
The actual script used for this scenario can be found here . Find the test results in this table, where this particular test is marked as test 3. [T4] Default ScriptChanged lines 37-44 from:
The actual script used for this scenario can be found here . In this scenario we stopped the test at 9000 CPS with an average load of 17.1 % ( actual load returned by htop ) See chart, where this particular test is marked as test D. Default Script with dialog supportto:
The actual script used for this scenario can be found here . Find the test results in this table, where this particular test is marked as test 4. [T5.1] Default Script with dialog supportChanged lines 45-51 from:
The actual script used for this scenario can be found here . In this scenario we stopped the test at 9000 CPS with an average load of 22.3 % ( actual load returned by htop ) See chart, where this particular test is marked as test E. Default Script with dialog support and authenticationto:
The actual script used for this scenario can be found here . Find the test results in this table, where this particular test is marked as test 5.1. [T5.2] Default Script with dialog support + Topology HidingThis scenario added topology hiding support on top of the previous one, with Call-ID concealment. The purpose of this scenario was to determine the performance penalty introduced by the the topology_hiding module. The actual script used for this scenario can be found here . Find the test results in this table, where this particular test is marked as test 5.2. Default Script with dialog support and authenticationMay 10, 2023, at 12:44 PM
by
- Added lines 1-111:
About -> Performance Tests 3.4 -- Scenarios(:toc-float Table of Content:) Details about each test scenario. 1. Minimal Stateful SIP ProxyIn the first test, OpenSIPS behaved as a minimalistic proxy, just statefully passing messages from the UAC to the UAS (a simple t_relay()). The purpose of this test was to see what is the performance penalty introduced by making the proxy stateful. The actual script used for this scenario can be found at here . Find the test results in this table, where this particular test is marked as test 1. 2. Stateful proxy with loose routingIn the second test, OpenSIPS script implements also the "Record-Route" mechanism, recording the path in initial requests, and then making sequential requests follow the determined path. The purpose of this test was to see what is the performance penalty introduced by the mechanism of record and loose routing. The actual script used for this scenario can be found here . In this scenario we stopped the test at 12000 CPS with an average load of 20.6 % ( actual load returned by htop ) See chart, where this particular test is marked as test B. Stateful proxy with loose routing and dialog supportIn the 3rd test we additionally made OpenSIPS dialog aware. The purpose of this particular test was to determine the performance penalty introduced by the dialog module. The actual script used for this scenario can be found here . In this scenario we stopped the test at 9000 CPS with an average load of 20.5 % ( actual load returned by htop ) See chart, where this particular test is marked as test C. Default ScriptThe 4th test had OpenSIPS running with the default script (provided by OpenSIPS distros). In this scenario, OpenSIPS can act as a SIP registrar, can properly handle CANCELs and detect traffic loops. OpenSIPS routed requests based on USRLOC, but only one subscriber was used. The purpose of this test was to see what is the performance penalty of a more advanced routing logic, taking into account the fact that the script used by this scenario is an enhanced version of the script used in the 3.2 test . The actual script used for this scenario can be found here . In this scenario we stopped the test at 9000 CPS with an average load of 17.1 % ( actual load returned by htop ) See chart, where this particular test is marked as test D. Default Script with dialog supportThis scenario added dialog support on top of the previous one. The purpose of this scenario was to determine the performance penalty introduced by the the dialog module. The actual script used for this scenario can be found here . In this scenario we stopped the test at 9000 CPS with an average load of 22.3 % ( actual load returned by htop ) See chart, where this particular test is marked as test E. Default Script with dialog support and authenticationCall authentication was added on top of the previous scenario. 1000 subscribers were used, and a local MYSQL was used as the DB back-end. The purpose of this test was to see the performance penalty introduced by having the proxy authenticate users. The actual script used for this scenario can be found here. In this scenario we stopped the test at 6000 CPS with an average load of 26.7 % ( actual load returned by htop ) See chart, where this particular test is marked as test F. 10k subscribersThis test used the same script as the previous one, the only difference being that there were 10 000 users in the subscribers table. The purpose of this test was to see how the USRLOC module scales with the number of registered users. In this scenario we stopped the test at 6000 CPS with an average load of 30.3 % ( actual load returned by htop ) See chart, where this particular test is marked as test G. Subscriber cachingIn the test, OpenSIPS used the localcache module in order to do less database queries. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help. The actual script used for this scenario can be found here. In this scenario we stopped the test at 6000 CPS with an average load of 18 % ( actual load returned by htop ) See chart, where this particular test is marked as test H. AccountingThis test had OpenSIPS running with 10k subscribers, with authentication ( no caching ), dialog aware and doing old type of accounting ( two entries, one for INVITE and one for BYE ). The purpose of this test was to see the performance penalty introduced by having OpenSIPS do the old type of accounting. The actual script used for this scenario can be found here. In this scenario we stopped the test at 6000 CPS with an average load of 43.8 % ( actual load returned by htop ) See chart, where this particular test is marked as test I. CDR accountingIn this test, OpenSIPS was directly generating CDRs for each call, as opposed to the previous scenario. The purpose of this test was to see how the new type of accounting compares to the old one. The actual script used for this scenario can be found here. In this scenario we managed to achieve 6000 CPS with an average load of 38.7 % ( actual load returned by htop ) See chart, where this particular test is marked as test J. CDR accounting + Auth CachingIn the last test, OpenSIPS was generating CDRs just as in the previous test, but it was also caching the 10k subscribers it had in the MYSQL database. In this scenario we stopped the test at 6000 CPS with an average load of 28.1 % ( actual load returned by htop ) See chart |