Performance testing practitioners always aspire to achieve a production like scenario and once that is achieve a load that is steady enough during the execution of the test. Though steady load is not what is observed in production, however from testing and issue resolution perspective it is important to achieve this target. So how should one get this steady load and that for the entire test duration?
First point to take into account is that in a test the starting point should be ‘number of user sessions per hour’ rather than the number of virtual users. It is important to know how each of the user sessions remained active during its visit to the website.
In case of most performance testing tools, adding think time values and pacing values come in handy. Think time allows the virtual user to pause in the business flow while moving from one business transaction to another. This takes into the account the time a real user might wait (think) while moving to the next transaction of the site. Similarly pacing allows the user to wait after one business flow cycle completion. Most testers ignore the testing functionality and allow the user sessions to be spaced after one another almost instantaneously. This is not the best approach to follow since there is always some time gap between two sessions by the same user in production environment.
But to incorporate the production like situation, one has to reproduce a scenario in which there are variations in the time each user session will take. The rate at which the user sessions start will also be uneven and after the test it would be impossible for the tester to judge the exact load he subjected upon the application.
Thus to apply an even load on the system which is exactly how it is in production, I employ a trick and I calculate the pacing time using this trick.
All load testing tools provide the feature of starting a transaction and ending it. In the script, at the beginning of the cycle, just put a start timer for a transaction and immediately end it. Immediate closure means that the tool will calculate the response time for that transaction to be zero. At the first iteration the transaction added will have a time when it was started (and immediately ended). The transaction will then be called ‘start timer’ again in second iteration. This difference between the times of first start timer and second start timer would give us the time user spent in one session. After knowing the average time between to subsequent sessions for users, one can calculate the pacing time as the time to be added at the end of the script to achieve time between two subsequent sessions as is observed in production. Once the test is over, the entire data associated with that transaction can be exported into an excel-sheet and the intervals between the transactions can help us calculate the pacing time.
So this is one of the multiple approaches that we can follow to identify the right pacing time that we must use during the test. There can be some other approaches also that you follow / employ for the same. Please feel free to share those and help the community learn more.
This post is also available in: French