OpenDDS - Performance Testing Results
Latency Details
The results of executing the latency tests are included in the table below. The table is organized with rows of transport types and columns of message size for each test executed.Each transport and message size combination in the table below contains information about the latency and jitter for that test. The 'plot' links are to quad-plots of the individual test run results. These include a timeline plot for both latency and jitter as well as histograms of the latency and jitter measurements.
The plots for each individual test include two timelines and two historgrams - one each for latency data and one each for jitter data. The timeline plots show that actualy measured values from the test execution. The histograms show the data collected and segregated into 25 bins with frequency per bin plotted versus the center value of the bin.
For latency plots the average value indicates the most likely latency to be encountered for the test conditions. The standard deviation gives a measure of the variability or "spread" of the data for those test conditions. Note that the histogram of latency delay values is clearly not Gaussian, so the standard deviation has little value other than for comparative purposes. The actual distribution appears closer to a Rayleigh distribution, which is not unexpected.
For jitter plots the average indicates the most likely jitter value to be encountered for the test conditions. We expect this to be zero for all cases, but since we have a finite number of values, the actual average is sometimes not exactly zero. The actual average values can be seen to be very close to zero increasing confidence that the actual average is 0. The distribution of the jitter values appears to be Gaussian and the standard deviation measure can be used to evaluate performance in the different conditions.
| Transport | Message Size (bytes) | |||||||||||||||||||
| 50 | 100 | 250 | 500 | 1000 | 2500 | 5000 | 8000 | 16000 | 32000 | |||||||||||
| Multicast (best effort) | Latency | Mean 3.34e-05 | Latency | Mean 3.88e-05 | Latency | Mean 3.88e-05 | Latency | Mean 3.78e-05 | Latency | Mean 4.03e-05 | Latency | Mean 4.49e-05 | Latency | Mean 4.84e-05 | Latency | Mean 5.63e-05 | Latency | Mean 7.28e-05 | Latency | Mean 1.25e-04 |
| Dev 6.61e-07 | Dev 4.04e-06 | Dev 3.18e-06 | Dev 4.51e-06 | Dev 3.57e-06 | Dev 3.81e-06 | Dev 3.01e-06 | Dev 3.31e-06 | Dev 1.08e-06 | Dev 1.45e-05 | |||||||||||
| Jitter | Mean -1.00e-10 | Jitter | Mean -4.07e-24 | Jitter | Mean 0.00e+00 | Jitter | Mean -1.00e-09 | Jitter | Mean 0.00e+00 | Jitter | Mean 1.40e-09 | Jitter | Mean 4.00e-10 | Jitter | Mean -2.00e-10 | Jitter | Mean 7.00e-10 | Jitter | Mean -2.40e-09 | |
| Dev 8.51e-07 | Dev 4.78e-06 | Dev 3.65e-06 | Dev 3.97e-06 | Dev 3.59e-06 | Dev 2.26e-06 | Dev 3.67e-06 | Dev 3.81e-06 | Dev 1.49e-06 | Dev 2.04e-05 | |||||||||||
| plot | plot | plot | plot | plot | plot | plot | plot | plot | plot | |||||||||||
| Multicast (reliable) | Latency | Mean 4.21e-05 | Latency | Mean 3.16e-05 | Latency | Mean 4.32e-05 | Latency | Mean 3.50e-05 | Latency | Mean 4.47e-05 | Latency | Mean 5.35e-05 | Latency | Mean 4.95e-05 | Latency | Mean 4.83e-05 | Latency | Mean 7.89e-05 | Latency | Mean 1.27e-04 |
| Dev 1.55e-06 | Dev 2.09e-06 | Dev 4.34e-06 | Dev 2.02e-06 | Dev 1.84e-06 | Dev 3.77e-06 | Dev 6.30e-06 | Dev 1.47e-06 | Dev 1.41e-06 | Dev 1.47e-05 | |||||||||||
| Jitter | Mean -1.00e-10 | Jitter | Mean 2.71e-24 | Jitter | Mean 0.00e+00 | Jitter | Mean -4.07e-24 | Jitter | Mean -4.00e-10 | Jitter | Mean -9.00e-10 | Jitter | Mean 1.60e-09 | Jitter | Mean 5.00e-10 | Jitter | Mean 2.00e-10 | Jitter | Mean 7.90e-09 | |
| Dev 2.03e-06 | Dev 2.81e-06 | Dev 4.02e-06 | Dev 2.81e-06 | Dev 2.49e-06 | Dev 4.85e-06 | Dev 6.36e-06 | Dev 2.01e-06 | Dev 1.95e-06 | Dev 1.19e-05 | |||||||||||
| plot | plot | plot | plot | plot | plot | plot | plot | plot | plot | |||||||||||
| TCP (reliable) | Latency | Mean 3.64e-05 | Latency | Mean 3.71e-05 | Latency | Mean 3.66e-05 | Latency | Mean 3.72e-05 | Latency | Mean 3.69e-05 | Latency | Mean 3.86e-05 | Latency | Mean 4.23e-05 | Latency | Mean 4.41e-05 | Latency | Mean 5.24e-05 | Latency | Mean 7.19e-05 |
| Dev 3.69e-06 | Dev 3.80e-06 | Dev 7.26e-06 | Dev 5.27e-06 | Dev 3.06e-06 | Dev 5.55e-06 | Dev 8.94e-06 | Dev 8.55e-06 | Dev 9.95e-06 | Dev 1.13e-05 | |||||||||||
| Jitter | Mean -8.00e-10 | Jitter | Mean -8.00e-10 | Jitter | Mean 6.00e-10 | Jitter | Mean 6.40e-09 | Jitter | Mean -1.00e-10 | Jitter | Mean 3.00e-10 | Jitter | Mean 5.00e-10 | Jitter | Mean -5.00e-10 | Jitter | Mean 1.00e-09 | Jitter | Mean 1.80e-09 | |
| Dev 5.18e-06 | Dev 5.26e-06 | Dev 7.64e-06 | Dev 6.48e-06 | Dev 4.50e-06 | Dev 6.40e-06 | Dev 9.34e-06 | Dev 7.07e-06 | Dev 9.58e-06 | Dev 1.07e-05 | |||||||||||
| plot | plot | plot | plot | plot | plot | plot | plot | plot | plot | |||||||||||
| UDP (best effort) | Latency | Mean 2.89e-05 | Latency | Mean 2.50e-05 | Latency | Mean 2.85e-05 | Latency | Mean 3.02e-05 | Latency | Mean 3.07e-05 | Latency | Mean 3.25e-05 | Latency | Mean 3.40e-05 | Latency | Mean 3.99e-05 | Latency | Mean 4.54e-05 | Latency | Mean 6.00e-05 |
| Dev 1.01e-06 | Dev 4.31e-06 | Dev 3.95e-06 | Dev 3.91e-06 | Dev 3.04e-06 | Dev 3.78e-06 | Dev 3.01e-06 | Dev 3.61e-06 | Dev 8.32e-07 | Dev 4.90e-06 | |||||||||||
| Jitter | Mean 4.00e-10 | Jitter | Mean 6.00e-10 | Jitter | Mean -3.00e-10 | Jitter | Mean 8.00e-10 | Jitter | Mean 2.00e-10 | Jitter | Mean 2.40e-09 | Jitter | Mean -1.00e-10 | Jitter | Mean 1.36e-24 | Jitter | Mean 0.00e+00 | Jitter | Mean 1.00e-09 | |
| Dev 1.41e-06 | Dev 4.92e-06 | Dev 4.05e-06 | Dev 4.19e-06 | Dev 3.44e-06 | Dev 3.65e-06 | Dev 3.16e-06 | Dev 4.20e-06 | Dev 1.11e-06 | Dev 5.18e-06 | |||||||||||
| plot | plot | plot | plot | plot | plot | plot | plot | plot | plot | |||||||||||
$Id: latency_details.html 50 2010-01-29 23:53:21Z martinezm $