The objective here was to compare the TCP and UDP engines as part of a testing scenario with a high volume of SIP traffic. As the nature of the SIP scenario was not too relevant, we went with the simplistic T1 setup, of simply performing ''t_relay()'' from UAC to UAS, effectively stress-testing the TCP engine to the limit.

In this test, the UAC/UAS ''sipp'' instances were configured in '''single-connection''' mode, routing all calls through a single TCP connection.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/15.1-tcp-engine.cfg | here ]].

Find the test results under the '''TCP Engine Tests''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc8 | table ]], where this particular test is marked as test '''14.1'''.

WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed.

WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed.

In this test, the UAC/UAS ''sipp'' instances were configured in '''multi-connection''' mode, routing all calls through multiple TCP connections. Overall, OpenSIPS was managing TCP connections in the order of thousands.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/15.1-tcp-engine.cfg | here ]].

Find the test results under the '''TCP Engine Tests''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc8 | table ]], where this particular test is marked as test '''14.4'''.

WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed.

!!! [T14.6] N Connections, parallel read: 2

WIP: this test uncovered a TCP connection sharing issue at high traffic volumes which is currently being addressed.

Building on the previous test, one critical change was made: we used the OpenSIPS cachedb_local module in order to ''fully'' eliminate DB queries during call setup. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help.

!!! [T5.2] Default Script with dialog + Topology Hiding

!!! [T6] Default Script with dialog + authentication

!! Basic Calling Scenarios

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''12'''.

!! Advanced Calling Scenarios

!!! [T13.1] B2B Topology Hiding

The purpose of this test was to gauge the performance difference of dialog-based topology hiding vs. B2B topology hiding.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/13.1-b2b-top-hiding.cfg | here ]].

Find the test results under the '''Complex Calling Scenarios''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc7 | table ]], where this particular test is marked as test '''13.1'''.

!!! [T13.2] B2B REFER Handling

The purpose of this test was to assess the maximum CPS possible in the B2B REFER scenario.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/13.2-b2b-refer.cfg | here ]].

Find the test results under the '''Complex Calling Scenarios''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc7 | table ]], where this particular test is marked as test '''13.2'''.

!!! [T13.3] B2B Marketing Scenario

The purpose of this test was to assess the maximum CPS possible in the B2B Marketing scenario.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/13.3-b2b-marketing.cfg | here ]].

Find the test results under the '''Complex Calling Scenarios''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc7 | table ]], where this particular test is marked as test '''13.3'''.

!! TCP Tests

!!! [T14.1] Single Connection, parallel read: OFF

!!! [T14.2] Single Connection, parallel read: 1

!!! [T14.3] Single Connection, parallel read: 2

!!! [T14.4] N Connections, parallel read: OFF

!!! [T14.5] N Connections, parallel read: 1

!!! [T14.6] N Connections, parallel read: 2

!!! [T11] DB_Flatstore CDR accounting

!!! [T12] DB_Flatstore CDR accounting + Auth Caching

!!! [T11] DB_Flatstore CDR accounting + Auth Caching

!!! [T12] DB_Flatstore CDR accounting

!!! [T9] CDR Accounting

!!! [T7] Default Script with dialog + authentication (10k sub)

!!! [T8] Subscriber caching

!!! [T9] Accounting

!!! [T10] CDR accounting + Auth Caching

!!! [T11] DB_Flatstore accounting + Auth Caching

!!! [T12] DB_Flatstore accounting

!!! [T6] Default Script with dialog support and authentication

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/6-default-dlg-auth-1k.cfg | here ]].

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''6'''.




!!!! [T7] Default Script with dialog + authentication (10k sub)

This test used the same script as the previous one, the only difference being that there were 10.000 users in the subscribers table. The purpose of this test was to see how the USRLOC module scales with the number of registered users.

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''7'''.




!!!! [T8] Subscriber caching

In the test, OpenSIPS used the cachedb_local module in order to do less database queries. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help.

The actual script used for this scenario can be found [[https://opensips.org/pub/images/stress-tests/cfg/8-default-dlg-auth-10k-cache.cfg | here ]].

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''8'''.



!!!! [T9] Accounting

This test had OpenSIPS running with 10k subscribers, with authentication ( no caching ), dialog aware and doing CDR accounting. The purpose of this test was to see the performance penalty introduced by having OpenSIPS do both READ and WRITE queries to the database.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/9-default-dlg-auth-10k-cdr.cfg | here ]].

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''9'''.






!!!! [T10] CDR accounting + Auth Caching

In this test, OpenSIPS was generating CDRs just as in the previous test, but it was also caching the 10k subscribers it had in the MYSQL database.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/10-default-dlg-auth-10k-cache-cdr.cfg | here ]].

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''10'''.





!!!! [T11] DB_Flatstore accounting + Auth Caching

A similar test to T9, however the DB backend used to write the CDR is ''db_flatstore'', writing them directly to disk as CSV records in an optimized fashion.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/11-default-dlg-auth-10k-cdrflat.cfg | here ]].

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''11'''.





!!!! [T12] DB_Flatstore accounting

This last test completely eliminated the need for querying MySQL during call setup, as we now also pre-cached the authentication information for all 10k subscribers.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/12-default-dlg-auth-10k-cache-cdrflat.cfg | here ]].

Find the test results under the '''Authenticated Calls''' [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''12'''.

!!! [T1] Minimal Stateful SIP Proxy

!!! [T2] Stateful proxy with loose routing

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/2-tm-rr.cfg | here ]] .

Find the test results in this [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''2'''.

!!! [T3] Stateful proxy with loose routing and dialog support

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/3-tm-rr-dialog.cfg | here ]] .

Find the test results in this [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''3'''.

!!! [T4] Default Script

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/4-default.cfg | here ]] .

Find the test results in this [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''4'''.

!!! [T5.1] Default Script with dialog support

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/5-default-dlg.cfg | here ]] .

Find the test results in this [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''5.1'''.

!!! [T5.2] Default Script with dialog support + Topology Hiding

This scenario added topology hiding support on top of the previous one, with Call-ID concealment. The purpose of this scenario was to determine the performance penalty introduced by the the topology_hiding module.

The actual script used for this scenario can be found [[ https://opensips.org/pub/images/stress-tests/cfg/5.2-default-dlg-th.cfg | here ]] .

Find the test results in this [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''5.2'''.




!!! Default Script with dialog support and authentication

!!!!! About -> Performance Tests 3.4 -- Scenarios
(:toc-float Table of Content:)

Details about each test scenario.

----

!!! 1. Minimal Stateful SIP Proxy

In the first test, OpenSIPS behaved as a minimalistic proxy, just statefully passing messages from the UAC to the UAS (a simple t_relay()). The purpose of this test was to see what is the performance penalty introduced by making the proxy stateful.\

The actual script used for this scenario can be found at [[ https://opensips.org/pub/images/stress-tests/cfg/1-tm.cfg | here ]] .

Find the test results in this [[ https://www.opensips.org/About/PerformanceTests-3-4#toc6 | table ]], where this particular test is marked as test '''1'''.


!!! 2. Stateful proxy with loose routing

In the second test, OpenSIPS script implements also the "Record-Route" mechanism, recording the path in initial requests, and then making sequential requests follow the determined path. The purpose of this test was to see what is the performance penalty introduced by the mechanism of record and loose routing.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_B.cfg | here ]] .

In this scenario we stopped the test at 12000 CPS with an average load of 20.6 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test B.

!!!! Stateful proxy with loose routing and dialog support

In the 3rd test we additionally made OpenSIPS dialog aware. The purpose of this particular test was to determine the performance penalty introduced by the dialog module.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_C.cfg | here ]] .

In this scenario we stopped the test at 9000 CPS with an average load of 20.5 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test C.

!!!! Default Script

The 4th test had OpenSIPS running with the default script (provided by OpenSIPS distros). In this scenario, OpenSIPS can act as a SIP registrar, can properly handle CANCELs and detect traffic loops. OpenSIPS routed requests based on USRLOC, but only one subscriber was used. The purpose of this test was to see what is the performance penalty of a more advanced routing logic, taking into account the fact that the script used by this scenario is an enhanced version of the script used in the [[http://www.opensips.org/Resources/StressTests#toc5 | 3.2 test ]] .

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_D.cfg | here ]] .

In this scenario we stopped the test at 9000 CPS with an average load of 17.1 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test D.

!!!! Default Script with dialog support

This scenario added dialog support on top of the previous one. The purpose of this scenario was to determine the performance penalty introduced by the the dialog module.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_E.cfg | here ]] .

In this scenario we stopped the test at 9000 CPS with an average load of 22.3 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test E.

!!!! Default Script with dialog support and authentication

Call authentication was added on top of the previous scenario. 1000 subscribers were used, and a local MYSQL was used as the DB back-end. The purpose of this test was to see the performance penalty introduced by having the proxy authenticate users.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_F.cfg | here ]].

In this scenario we stopped the test at 6000 CPS with an average load of 26.7 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test F.

!!!! 10k subscribers

This test used the same script as the previous one, the only difference being that there were 10 000 users in the subscribers table. The purpose of this test was to see how the USRLOC module scales with the number of registered users.

In this scenario we stopped the test at 6000 CPS with an average load of 30.3 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test G.

!!!! Subscriber caching

In the test, OpenSIPS used the localcache module in order to do less database queries. The cache expiry time was set to 20 minutes, so during the test, all 10k registered subscribers have been in the cache. The purpose of this test was to see how much DB queries are affecting OpenSIPS performance, and how much can caching help.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_H.cfg | here ]].

In this scenario we stopped the test at 6000 CPS with an average load of 18 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test H.

!!!! Accounting

This test had OpenSIPS running with 10k subscribers, with authentication ( no caching ), dialog aware and doing old type of accounting ( two entries, one for INVITE and one for BYE ). The purpose of this test was to see the performance penalty introduced by having OpenSIPS do the old type of accounting.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_I.cfg | here ]].

In this scenario we stopped the test at 6000 CPS with an average load of 43.8 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test I.

!!!! CDR accounting

In this test, OpenSIPS was directly generating CDRs for each call, as opposed to the previous scenario. The purpose of this test was to see how the new type of accounting compares to the old one.

The actual script used for this scenario can be found [[http://www.opensips.org/uploads/Resources/PerformanceTests/test_J.cfg | here ]].

In this scenario we managed to achieve 6000 CPS with an average load of 38.7 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]], where this particular test is marked as test J.

!!!! CDR accounting + Auth Caching

In the last test, OpenSIPS was generating CDRs just as in the previous test, but it was also caching the 10k subscribers it had in the MYSQL database.

In this scenario we stopped the test at 6000 CPS with an average load of 28.1 % ( actual load returned by htop )

See [[http://www.opensips.org/Resources/StressTests#toc15 | chart ]]