I have worked with many mobile app developers and I have often come across scenarios wherein the clear difference between testing an app and monitoring an app is not clear to the developers. I am hence making an attempt to clarify the concepts and highlight the differences here –
Here are some key differences:
1.) Definition: Testing Vs Monitoring:
Reproducing the production like scenario and trying to break the system using different business scenarios is testing. This reproduction of scenarios can be accomplished using automated scripts and regression testing is achieved in this fashion. Scripts help the testers simulate the production scenario and test it under production like conditions. In case of performance testing, after reproducing the production like scenario, many performance issues observed at the GUI and database end are identified.
Monitoring involves watching the systems, servers and infrastructure of the application when the testing is under process. Monitoring plays a key role during performance testing wherein the load subjected on the application may degrade the performance of the system. Issues related to actual memory, network bandwidth and CPU cycles are resolved using monitor.
2.) Estimated Vs Actual behavior (performance) of system:
In case of mobile apps there is a lot of difference in the infrastructure (mobile phone) that end users are using and are subjecting the app to. Some key differences include –
- Different variety of mobile handset used by different users
- Network bandwidth available at the different locations
- Volume of data downloaded by one user may be significantly different from that of another
- Usage of app for one user would be significantly different from that of the other user
- For a specific app, there may be significant difference between the number of users across different geographies.
These differences can lead to performance numbers that are significantly different from those of actual numbers. Thus complimenting server monitoring with performance testing can help the team in achieving improved performance to a much greater extent. (Given the fact that actual results would be sufficiently different from those of the estimated ones).
3.) Testing in Dev, QA, Performance test & Production environments:
Performance testing can be done in most of these environments in some or other format. Dev is the first infra where the app is tested for its functionality. QA is where the app is again tested for its functionality by testers. Sometimes performance testing is also conducted in the QA environment. However the best situation is to test the application under load in a separate performance test environment which is scaled up as much as that of the production environment. While testing on performance test environment, monitoring the infrastructure helps in identifying a lot more performance issues. A real time production test scenario helps in catching many performance issues before release. Monitoring helps in identifying issues which were missed out in testing.
Thus mobile app development can gain and benefit a lot from performance testing and server monitoring during the testing. Now-a-days the tests are planned in such a way that both testing and monitoring (during test & also in production) are performed simultaneously. It helps in enhancing the application performance and thus helps the business create a better app.