• the new OpenSIPS TCP connection balancing is based on the load: statistic, so when doing TCP engine stress-testing in single-connection mode (on the clients' side), make sure to start the UACs gradually, one-by-one in order to give the load: statistic a bit of time to update, such that the new high-throughput connections do not all end up in the same TCP worker!

• the CPU-bound tests (1-6) used a maximum of 8 UDP workers (typically 4), in order to minimize context-switching (since the OpenSIPS system was a quad-core -- 1:1 worker/CPU mapping)

This page has been visited 2665 times.

• the B2B modules currently have a lower CPS performance, due to the internal complexity of the code. We are still evaluating whether there is room for optimization in the current shape of the codebase.

• the newly introduced load: statistic is critical for monitoring the behavior and performance of your OpenSIPS instance. It can help you spot which workers are busy or not. Or when you need extra capacity on your instance, due to being either CPU-bound or I/O-bound.
  • recap: this statistic monitors the "idleness" of your OpenSIPS workers. If they are doing anything other than waiting for a new SIP job, then they are "busy". Otherwise, they are "idle". For example, if an OpenSIPS worker is running a sleep(1000) in your opensips.cfg, its load: value will be 100% (fully busy).
  • a low CPU usage from your OpenSIPS instance does not mean it's necessarily not loaded. It could be stuck in I/O operations and asking for more SIP workers.
• when adding DB query caching to your OpenSIPS instance, do not be surprised if it's running a higher CPU usage, because the database will be at 0% CPU usage afterwards, resulting in a overall net gain of CPU resource, as well as dramatically reduced I/O wait time (again, watch the load: statistic).
• the new OpenSIPS TCP connection balancing is based on the load: statistic, so when doing TCP engine stress-testing in single-connection mode, make sure to start the UACs gradually, one-by-one in order to give the load: statistic a bit of time to update, such that the new high-throughput connections do not all end up in the same TCP worker!

Unauthenticated Calls

Authenticated Calls

https://opensips.org/pub/images/stress-tests/opensips-st-graph-2.png

https://opensips.org/pub/images/stress-tests/opensips-st-graph-1.png

https://opensips.org/pub/images/stress-tests/opensips-st-graph-1.png

\\

• latest git revision the tests were run on: b0068befd (May 9th, master branch)

• the upper limit of each test was determined by various metrics: either max CPU usage on the OpenSIPS box, various error logs at capacity limit or UDP/TCP accumulating Recv-Queue
• once the CPS limit was discovered -> perform profiling, analyze the CPU usage map and try to spot bottlenecks

• latest git revision the tests were run on: b0068befd (May 9th, master branch)

• Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz (4 cores, 8 threads, launch date: Q1'17)

• UDP was used as transport protocol for the majority of tests, unless stated otherwise

The following table shows the raw CPS data used in each scenario. Notes:

• the Avg. CPU column represents the average CPU usage of the SIP worker processes, as shown by top
• the Load-1m column represents the average OpenSIPS load over 1 minute of the SIP worker processes only, extracted from the load: statistic

TCP Test Scenarios

TODO

Conclusions

The objective of the stress tests was to re-assess the performance of various OpenSIPS subsystems, ahead of the upcoming 3.4 beta release. Apart from putting updated maximum capacity numbers on these modules, the tests also pinpointed various performance bottlenecks in each scenario, thanks to code profiling.

• average call duration: 30 seconds

The basic calling tests progressively add more signaling until the following call flow is achieved:

https://opensips.org/pub/images/stress-tests/opensips-st-call-flow.png

ACD: 30 seconds

\\

• the CPU-bound tests (1-6) used 4 UDP workers, in order to minimize context-switching (since the OpenSIPS system was a quad-core -- 1:1 worker/CPU mapping)
• starting with test #7, the SIP workers were bumped to 8, to cope with the added I/O operations (8 workers were enough to satisfy the required ~6k CPS)
• in tests 1-6, the proxy was pushed to the maximum possible CPU load, while on tests 7-14 the traffic was kept constant at 6000 CPS and we instead monitored the CPU load penalty as we progressed through the tests

Basic Calling Scenarios (transactions, dialogs)

Setup Description

• all tests used the F_MALLOC memory allocator (the default in all public builds). A performance comparison between F_MALLOC, Q_MALLOC and HP_MALLOC can be found in a separate set of tests below
• all CPU-bound tests used 4 UDP workers, in order to minimize context-switching (since the OpenSIPS system was a quad-core -- 1:1 worker/CPU mapping)
• for the I/O bound tests, 8 workers were typically used (enough to satisfy the required ~6k CPS)
• in tests 1-6, the proxy was pushed to the maximum possible CPU load
• since tests 7-14 are a mix between CPU-bound and I/O bound, we kept the traffic constant at 6000 CPS and instead monitored the CPU load penalty as we progressed through the tests

• since tests 5-14 are a mix between CPU-bound and I/O bound, we kept the traffic constant at 6000 CPS and instead monitored the CPU load penalty as we progressed through the tests

• Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz (4 cores, 8 threads)

The following table shows the raw CPS data used in each scenario.

• all tests used the F_MALLOC memory allocator (the default in all public builds). A performance comparison between F_MALLOC, Q_MALLOC and HP_MALLOC can be found in a separate set of tests below.
• in tests 1-4, the proxy was pushed to the maximum possible CPU load
• since tests 5-14 are a mix between CPU-bound and I/O bound, we kept the traffic constant at 6000 CPS and observed the CPU load penalty

https://opensips.org/pub/images/stress-tests/sipp-setup.png

For all SIP traffic generation purposes, sipp was the main tool which got the job done. Being a single-threaded application, both the sipp UAC and UAS were found to reach their capacity limitation at around 2500 - 3000 CPS. So we simply scale them horizontally, by launching more clients and servers!

• Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
• 16 GB DDR4 (Kingston)
• SSD 850 EVO 250GB (Samsung)

Simple Calling Scenarios

Complex Calling Scenarios (B2B)

• the stress-tests were broken down into three categories: calling tests, B2B tests and TCP engine tests
• within each category, we gradually increased the amount of features (code) ran through by each test
• the upper limit of each test was determined by various metrics: either max CPU usage on the OpenSIPS box or UDP/TCP accumulating Recv-Queue
• once the CPS limit was discovered -> perform profiling and analyze the resulting

Setup

TODO: proper picture

Hardware

About -> Performance Tests

(:toc-float Table of Content:)

A collection of performance tests and measurements performed on OpenSIPS 3.4, on various subsystems: database, transactions, dialogs, etc. These tests should give you a broad idea on what you could achieve on your own OpenSIPS setup using similar hardware!

Purpose

The objective of the stress tests was to re-assess the performance of various OpenSIPS subsystems, ahead of the upcoming 3.4 beta release. Apart from putting updated maximum capacity numbers on these modules, the tests also pinpointed performance bottlenecks in these systems, thanks to code profiling.

Overview

Raw Results