A Business Technology Place

Customer focused eCommerce: Volume testing techniques

“If you build it, they will come…”.  That’s a classic line from Field of Dreams. While I love baseball, this isn’t a post about our great American sport and its glory.  I’ve heard this quote it used on many occasions in casual conversation about things other than baseball. One example, is when building Internet sites. If you are planning to build a site, it’s an essential task to test your site for the type of volume you expect. After all, “If you build it, they will come…” A poor customer experience due to a slow web site, unavailable web site, or system error messages could mean lost business and a bad reputation.

There are three types of volume testing that are commonly used to assist with web site development:

Performance Testing

The primary purpose of performance testing is to find system bottlenecks. This type of testing typically involves software tools that can simulate concurrent users on the system. The response time of the system is compared against the goals for such things as page response, query response, etc.  If there are discrepancies between the time recorded and the goal, then one should examine the performance of individual components in the system to determine the bottlenecks.

Another purpose of performance testing is to record a baseline of system performance for key transactions (login, order history lookup, order submit, etc.). After you establish the performance baseline, you can use it with future releases to determine if the release is improving, decreasing, or holding the existing performance bench mark. Certainly any release that decreases performance would be a concern and may require adjustments before releasing to the production environment.

Load Testing

Load testing is intended to simulate the highest expected load a system is expected to handle. Using automated testing tools, the highest expected load is setup in the test plan to run against the system. This could involve a certain number of concurrent users, a specific number of open database connections, or a set number of transaction calls. The goal of load testing is not to break the system, but to observe how the system operates under a constant maximum load. This type of testing could reveal slow memory leaks that result in gradual system performance degradation.

Stress Testing

Stress testing tries to break a system by giving it more or less inputs than it can process. There are two main goals with this type of testing:

1. Record at what level the system breaks. This gives you an idea of your maximum load and can be used for pro-active system sizing.

2. Make sure that the system has a graceful fail and recovery mechanism. This could be the result of adding too many inputs (say concurrent users) than the system can handle, or it could be the result of taking something away from the system such as its database. If you take away the database for a brief time how does the system notate this response to the users and does it recover automatically when the database is active again?

How does this relate to Customer focused eCommerce?

Have you ever been on a site that experienced one of these problems?

  • Page does not respond
  • HTTP 500 error
  • Java.Lang…. error
  • Error message that indicates the system has reached capacity
  • Timeout waiting for response for server

Those are not good feelings and can often be very frustrating if you’ve invested time in a shopping, registering, or searching for information. In many cases, these types of errors could be found by following the volume testing techniques mentioned above.  Focus on your customers, by making sure your site is tuned and available. They’ll be more likely thank you with their business and come back the next time they need your product or service.