Performance Testing

Test Results

Test Descriptions and Results (most recent first)
Date	Test	Environment	Version	Results
August 03, 2021	Latency	OCI "Centipede" Virtual Test Environment 2021	3.18.0-dev (6155a3c)	cent210803
August 30, 2017	Latency	OCI Lab 2017	3.12	lab170830
March 17, 2010	Latency loopback: host1 == host2	SGI Lab, 3.3 GHz	2.1.2	sgi100317
January 27, 2010	Latency loopback: host1 == host2	SGI Lab, 3.0 GHz	2.0	sgi100127
January 25, 2010	Latency	OCI Lab	2.0	lab100125

Test Configuration

Testing using the OpenDDS-Bench performance testing framework is based on the use of configuration files to specify the behavior and topology of the test. There are two basic types of configuration file used for the testing. These are transport configuration files, used to configure the instances of transports for the test processes, and test configuration files, used to establish the test conditions and topology. These are briefly described here.

Transport Configuration Files

The transport configuration files specify the specific configuration of transport instances by instance key. Each transport used within a test will have a transport key associated with it, and that key is configured with a section in a transport configuration file. Typically a single transport configuration file is used per test, with each process selecting the appropriately configured transport instances that it uses from the file. The following table shows the transport configuration files provided in the etc directory for use during testing.

Files in $DDS_ROOT/performance-tests/Bench/etc
File	Description
transport.ini	general purpose configuration file defining 10 transport instances of each type
transport-tcp.ini	configures transport instances using the *tcp* transport type
transport-udp.ini	configures transport instances using the *udp* transport type due to the nature of this transport implementation, this file will need to be tailored for each test host on which it is used
transport-multi-be.ini	configures transport instances using the *multicast* transport type to operate in best effort mode
transport-multi-rel.ini	configures transport instances using the *multicast* transport type to operate in reliable mode

Test Configuration Files

The test configuration files used for each test are unique are are defined as part of the OpenDDS-Bench performance test framework. Each test will have a configuration file to be used for each test process for each test case. The configuration files specify both the DDS Quality of Service policy values for each Entity in the test process and the test specific configuration for that process and those Entities. The data which can be specified in these configuration files is defined in the $DDS_ROOT/performance-tests/Bench/doc/datamodel.txt file.

Test Environments

Performance tests of the OpenDDS implementation were executed in various environments. The environments for which we are reporting results are described in the following table.

Test Environments
Name	Short Description
OCI "Centipede" Virtual Test Environment 2021	3 ProxMox PVE Machines, 30 4-core 2GB VMs running RHEL 7.6 with 10GB Network
OCI test lab	SUSE 9.1, Intel P4 2.6 GHz, 1.5 GB RAM, gigabit ethernet
OCI test lab 2017	CentOS 7, Intel Core2 Duo E8400 3.00 GHz, 5.5 GB RAM, gigabit ethernet
SGI 3.0 GHz test lab	SUSE SLES 10, SGI Altix ICE 8200, Intel Harpertown (Xeon 3.0 GHz), 2 GB RAM
SGI 3.3 GHz test lab	SUSE SLES 10, SGI Altix ICE 8200, Intel W5590 (Xeon 3.3 GHz), 2 GB RAM

Test Descriptions

The tests available with configurations in the OpenDDS-Bench performance test framework are summarized in the table below.

Test Descriptions
Name	Short Description
Latency	Measure latency between 2 hosts from start of send to end of read.
Latency (loopback)	Measure latency using loopback interface from start of send to end of read. This is the same test as the 2 host test, but with host1 == host2.