Performance Testing in a Provisioning World

Then comes the time it takes to build the configuration files. The more complex the file, the longer it will take us to build. Other variations can occur on the network itself and the CMTS. By themselves, none of these factors will impact on the time it takes to provision a device. However, when added together, and over tens of thousands of devices, the delays become more significant.

When you consider all these issues, it’s easy to understand that we cannot run every possible variation when we test performance. Otherwise, it would take years to report performance numbers. Instead, we test a standard configuration to gain performance data. But how do we generate 250,000 cable modems? Surely we do not have that many in our lab! While that would be cool, we’ve built a simulator that can generate the full DHCP cycle. Even so, this can’t account for actual DOCSIS delays that may be caused by the HFC network or the registration process. As a result, your mileage will vary.

We use our performance numbers internally to compare between versions to make sure that any new features will not affect performance. So take any vendor promising great riches with a grain of salt. We never claim anything we can’t deliver and try to make our testing as realistic to real world scenarios as possible.