Performance Profiling
[Performance profiling is a performance test in which response times, transaction rates, and other time-sensitive requirements are measured and evaluated. The goal of Performance Profiling is to verify performance requirements have been achieved. Performance profiling is implemented and executed to profile and tune a target-of-test's performance behaviors as a function of conditions such as workload or hardware configurations.
Note: Transactions below refer to “logical business transactions”. These transactions are defined as specific use cases that an actor of the system is expected to perform using the target-of-test, such as add or modify a given contract.]
| Test Objective: | [Verify performance behaviors for designated transactions or business functions under the following conditions: · normal anticipated workload · anticipated worst case workload] |
| Technique: | · [Use Test Procedures developed for Function or Business Cycle Testing. · Modify data files to increase the number of transactions or the scripts to increase the number of iterations each transaction occurs. · Scripts should be run on one machine (best case to benchmark single user, single transaction) and be repeated with multiple clients (virtual or actual, see Special Considerations below).] |
| Completion Criteria: | · [Single Transaction or single user: Successful completion of the test scripts without any failures and within the expected or required time allocation per transaction.] · [Multiple transactions or multiple users: Successful completion of the test scripts without any failures and within acceptable time allocation.] |
| Special Considerations: | [Comprehensive performance testing includes having a background workload on the server. There are several methods that can be used to perform this, including: · “Drive transactions” directly to the server, usually in the form of Structured Query Language (SQL) calls. · Create “virtual” user load to simulate many clients, usually several hundred. Remote Terminal Emulation tools are used to accomplish this load. This technique can also be used to load the network with “traffic”. · Use multiple physical clients, each running test scripts to place a load on the system. Performance testing should be performed on a dedicated machine or at a dedicated time. This permits full control and accurate measurement. The databases used for Performance Testing should be either actual size or scaled equally.] |
Load Testing
|
|
|
[Load testing is a performance test which subjects the target-of-test to varying workloads to measure and evaluate the performance behaviors and ability of the target-of-test to continue to function properly under these different workloads. The goal of load testing is to determine and ensure that the system functions properly beyond the expected maximum workload. Additionally, load testing evaluates the performance characteristics, such as response times, transaction rates, and other time sensitive issues).]
[Note: Transactions below refer to “logical business transactions”. These transactions are defined as specific functions that an end user of the system is expected to perform using the application, such as add or modify a given contract.]
| Test Objective: | [Verify performance behavior time for designated transactions or business cases under varying workload conditions.] |
| Technique: | · [Use tests developed for Function or Business Cycle Testing. · Modify data files to increase the number of transactions or the tests to increase the number of times each transaction occurs.] |
| Completion Criteria: | [Multiple transactions or multiple users: Successful completion of the tests without any failures and within acceptable time allocation.] |
| Special Considerations: | · [Load testing should be performed on a dedicated machine or at a dedicated time. This permits full control and accurate measurement. · The databases used for load testing should be either actual size or scaled equally.] |
Stress Testing
[Stress testing is a type of performance test implemented and executed to find errors due to low resources or competition for resources. Low memory or disk space may reveal defects in the target-of-test that aren't apparent under normal conditions. Other defects might result from competition for shared resources like database locks or network bandwidth. Stress testing can also be used to identify the peak workload the target-of-test can handle.]
[Note: References to transactions below refer to logical business transactions.]
|
|
|
| Test Objective: | [Verify that the target-of-test functions properly and without error under the following stress conditions: · little or no memory available on the server (RAM and DASD) · maximum actual or physically capable number of clients connected or simulated · multiple users performing the same transactions against the same data or accounts · worst case transaction volume or mix (see Performance Testing above). Notes: The goal of Stress Testing might also be stated as identify and document the conditions under which the system FAILS to continue functioning properly. Stress Testing of the client is described under section 3.1.11, Configuration Testing.] |
| Technique: | · [Use tests developed for Performance Profiling or Load Testing. · To test limited resources, tests should be run on a single machine, and RAM and DASD on server should be reduced or limited. · For remaining stress tests, multiple clients should be used, either running the same tests or complementary tests to produce the worst-case transaction volume or mix. |
| Completion Criteria: | [All planned tests are executed and specified system limits are reached or exceeded without the software failing or conditions under which system failure occurs is outside of the specified conditions.] |
| Special Considerations: | · [Stressing the network may require network tools to load the network with messages or packets. · The DASD used for the system should temporarily be reduced to restrict the available space for the database to grow. · Synchronization of the simultaneous clients accessing of the same records or data accounts.] |
Volume Testing
[Volume Testing subjects the target-of-test to large amounts of data to determine if limits are reached that cause the software to fail. Volume Testing also identifies the continuous maximum load or volume the target-of-test can handle for a given period. For example, if the target-of-test is processing a set of database records to generate a report, a Volume Test would use a large test database and check that the software behaved normally and produced the correct report.]
|
|
|
| Test Objective: | [Verify that the target-of-test successfully functions under the following high volume scenarios: · Maximum (actual or physically- capable) number of clients connected, or simulated, all performing the same, worst case (performance) business function for an extended period. · Maximum database size has been reached (actual or scaled) and multiple queries or report transactions are executed simultaneously.] |
| Technique: | · [Use tests developed for Performance Profiling or Load Testing. · Multiple clients should be used, either running the same tests or complementary tests to produce the worst-case transaction volume or mix (see Stress Testing above) for an extended period. · Maximum database size is created (actual, scaled, or filled with representative data) and multiple clients used to run queries and report transactions simultaneously for extended periods.] |
| Completion Criteria: | · [All planned tests have been executed and specified system limits are reached or exceeded without the software or software failing.] |
| Special Considerations: | [What period of time would be considered an acceptable time for high volume conditions, as noted above?] |
Дата добавления: 2015-12-17; просмотров: 21; Мы поможем в написании вашей работы! |
Мы поможем в написании ваших работ!