The wsperf project was a set of low latency load testing tools for WebSocket connections based on the WebSocket++ library. Initial versions included a command server that ran tests similar to those from the performance section of the Autobahn Test suite to produce performance reports without latency limitations from Autobahn's CPython runtime. Later iterations included a standalone command line tool for WebSocket connection handshake stress testing.
The original wsperf program was included as an example program for WebSocket++ 0.2.x. It features message latency tests similar to the Autobahn 9.7.x and 9.8.x series and payload processing throughput tests simular to the Autobahn 9.1.x and 9.2.x series. It runs as a persistent WebSocket server. Clients implementing the wsperf subprotocol can connect and feed test commands to the server, which performs the tests and returns raw data back over the admin WebSocket connection. A simple Javascript client was included and the Autobahn test suite included a client that generated fancier reports.
This version also had a worker mode where multiple worker programs could connect to the command server and be assigned tasks. This was useful for stress testing servers from multiple locations or with traffic volumes larger than a single machine could produce.
Test results included min, max, median, mean, stddev for all samples as well as aggregated quantiles with configurable bin counts. It could optionally return individual, un-aggregated results for further statistical analysis by external tools.
Later on, wsperf was rebuilt as a standalone tool, avaliable on Github. The primary mode implemented was a connection/handshake mass stress test. You provided a number of connections and some rough guidance on how many concurrent connections you were shooting for. The probe would then attempt to open connections as quickly as possible until the target band was reached and from then would proceed to periodically close and open connections. The goal was to simulate mass connection load, especially in cases of a server coming back up after a restart and recieving a barage of reconnecting clients.
This tool provided detailed sample data for how much time was spent in each portion of the handshake (TCP handshake, TLS handhsake, WS handshake, and connection close) and output in a format that could be analyzed by external programs. A simple python based stats program is included that calculates handshake rate, failure percents, and handshake latency at various quantiles.
Aggregate results (WebSocket Opening+Closing Handshake)
Duration: 13.4 s
Total: 200000
Success: 200000
Fail: 0
Fail %: 0.00
Handshakes/sec: 14883
Min: 2.0 ms
SD: 10.7 ms
Avg: 67.6 ms
Median: 67.3 ms
q90 : 81.2 ms
q95 : 85.2 ms
q99 : 93.2 ms
q99.9 : 104.9 ms
q99.99: 108.3 ms
Max: 109.2 ms
Analyze done.
Opportunities to work on some other interesting projects came up and I have not done any further work on wsperf beyond the features described above. The original tool was built on version 0.2.x of the WebSocket++ library which is no longer maintained. The more recent standalone tool should still run with the modern version of WebSocket++.