wsperf

The goal of the wsperf project is to provide a comprehensive WebSocket performance and stress testing tool. The project grew from the limitations that the CPython runtime exerted on the Limits/Performance section of the Autobahn test suite. wsperf is written using the WebSocket++ library which provides a low level reference counted C++/Boost ASIO implementation of the WebSocket protocol suitable for providing a performance testing baseline.

Planned Features and Status

Test Types

  • Message Latency/RTT (similar to Autobahn 9.7.x and 9.8.x tests)
  • Payload processing throughput (similar to Autobahn 9.1.x and 9.2.x tests)
  • Broadcast server throughput and message rates (in development)
  • Massive concurrent connection numbers for servers (in development)

Usage Modes

wsperf uses a very simple command language and outputs JSON for easy analysis by other software packages. wsperf will run in three different modes:

  • Master Mode (implemented): wsperf runs as a persistent websocket server. Clients implementing the wsperf subprotocol can connect and feed test commands to the server. The server will perform the tests and return the results to the client over the admin websocket connection.
  • Slave Mode (implemented): wsperf runs as a persistent websocket client that connects to a command server that implements the wsperf subprotocol. A single command server can have multiple wsperf slaves connected concurrently and use them to run distributed tests.
  • Command line utility (in development): Accept commands via a text file or standard input and output to a text file or standard output.

Statistics Produced

  • Min, Max, Median, Mean, Standard Deviation
  • Aggregated quantiles with configurable bin count. Reports bin mean and boundaries
  • Optionally, individual non-aggregated results of each message or connection

Integration with the Autobahn Test Suite

The Autobahn Test Suite is integrating support for wsperf into its wstest utility. This will include a command client and command server capable of talking to wsperf in both Master and Slave modes. These tools will provide convenient ways of controlling groups of wsperf slaves via a browser and aggregating the results of distributed tests.

Learn More (and get code)

wsperf is presently included as an example in the WebSocket++ project. To build wsperf, clone the WebSocket++ git repository and build the examples. Additional detailed wsperf documentation is available on the WebSocket++ github wiki