Load Testing

Definition

Load testing is the system testing of an integrated, blackbox application that attempts to cause failures involving how its performance varies under normal conditions of utilization (e.g., as the load increases and becomes heavy).

Goals

The typical goals of load testing are to:

• Cause the application to fail to scale gracefully under normal conditions so that the underlying defects can be identified, analyzed, fixed, and prevented in the future.

Objectives

The typical objectives of load testing are to:

• Partially validate the application (i.e., to determine if it fulfills its scalability requirements):
  • Scalability requirements (e.g., the number of users increases).
  • Distribution and load-balancing mechanisms.
• Cause failures concerning the load requirements that help identify defects that are not efficiently found during unit and integration testing.
• Report these failures to the development teams so that the associated defects can be fixed.
• Determine if the application will support typical production load conditions.
• Identify the point at which the load becomes so great that the application fails to meet performance requirements.
• Locate performance bottlenecks including those in I/O, CPU, network, and database.
• Help determine the extent to which the application is ready for launch.
• Provide input to the defect trend analysis effort.

Examples

Typical examples include load testing of an application that is:

• Software only.
• A system including software, hardware, and data components.
• Huge (e.g., number of users, number of transactions, amount of data).
• Batch with no realtime requirements.
• Soft realtime (i.e., human reaction times).
• Hard realtime (e.g., avionics, radar, automotive engine control).
• Embeded within another system (e.g., flight-control software, cruise-control software).
• Client/server or n-tier distributed.
• A research prototype that will not be placed into service.
• Business-critical or safety-critical.

Preconditions

Load test execution can typically begin when the following preconditions hold:

• The scalability requirements to be tested have been specified.
• A test suite of test cases for every scalability requirement exists.
• The relevant software components have passed unit testing.
• Software integration has occurred (i.e., load testing can begin prior to the distribution of the software components onto the hardware components).
• The relevant system components have passed system integration testing.
• The independent test team is adequately staffed and trained in load testing.
• The test environment is ready.

Completion Criteria

Load testing is typically complete when the following postconditions hold:

• At least one load test suite exists for each scalability requirement.
• The test suites for every scheduled scalability requirement execute successfully.

Tasks

Load testing typically involves the independent test team performing the following testing tasks using the following techniques:

• Test Planning
• Test Reuse
• Test Design:
  • Use Case Based Testing
  • Workload analysis to determine the typical production workloads.
• Test Implementation:
  • Develop test scripts simulating real-life workloads.
• Test Execution:
  • Regression Testing
  • Profiling.
• Test Reporting

Environments

Load testing is typically performed on the following environments using the following tools:

• Test Environment:
  • Test Harness
  • Use case tool
  • Record and playback tool
  • Performance analyzer
  • Profiler

Work Products

Load testing typically results in the production of all or part of the following work products from the test work product set:

• Documents:
  • Project Test Plan
  • Master Test List
  • Test Procedures
  • Test Report
  • Test Summary Report
• Software and Data:
  • Test Harness
  • Test Scripts
  • Test Suites
  • Test Cases
  • Test Data

Phases

Load testing typically consists of the following tasks being performed during the following phases:

Guidelines

• An application can fulfil its operational requirements and still be a failure if it does not scale.
• The iterative and incremental development cycle implies that load testing is regularly performed in an iterative and incremental manner.
• Load testing must be automated if adequate regression testing is to occur.
• Load testing can elicit failures prior to launch.
• Perform workload analysis to determine the typical production workloads.
• Develop test scripts simulating real-life workloads.
• Perform load testing for several minutes to several hours.
• To the extent practical, reuse functional test cases as load test cases.