Testers are facing more and more challenges with data-centric mobile applications as the growth of features, OS’s, devices is growing with lightning speed. Luckily there is always a nice safe place called automation that comes to assistance covered in shining armor and with a sword-of-making-things-less-hard thingy. What may be automated? Well, pretty much all of functional as well as non-functional features of the app’s behavior. Basically test automation is a blend of Environmental, Performance, Behavioral as well as Complete Ecosystem testing. But let’s Deal with one at a time.
- Usage of actual devices. This is the very first thing any tester is to consider with environmental testing. Real deal devices just prove to be more accurate than even the best of emulators. One more thing to remember here is that app’s behavior may be highly affected by target geolocation, latency, carrier infrastructure and, of course, a network type. You are to include various network changes into your testing strategy.
- Automated switching of networks. This may assist with locating changes in the app’s behavior and will assist with identifying performance issues.
- Automate app installation via OTA. A successful app is bond to run on multiple platforms thus you are to be 100% sure it’s installation and update are working as planned. This is also important due many different ways of software distribution.
- Notification services are not to be misses as well. All kinds of notification tests from a low battery charge indicator to an email with updates and error messages are to be automated to make life easier.
- A 3-d party app is also to be installed on the device. An app that will be doing things like log running or multiple server-related operations in order for the original app’s performance is under test with all the data related to things like CPU loads, etc.
- CPU and memory analysis. A fairly nice practice for testing will be collecting and analyzing various CPU, battery and memory statistics. This is a great starting point for locating possible issues. Thus you will require an automation tool that will do just that very task.
- Responsiveness of the app. A nice thing to do here is calculating the total app’s screen collapse time as well as time required for navigation. All has to be fast and smooth for users to love it.
- UI nav performance. Focus on actual UI-response times in your strategy as it is of the most important things in your app to your end-users.
- Change of Screen Orientation. The tricky part is that those changes may be triggered by different things in different OS’s. If we are talking Android you are to define all of the configuration changes for all the changes like keyboard extension, etc. remember automate android testing is different from Windows or iOS automate testing. Plus consider what will happen if the phone is not connected to the network due, let’s say, airplane mode. Also you are to test how the app is handling screen changes with native features of the platform itself.
- System VS User events. Happenings that are called system events are the codetriggering this or that button written by the developer when user events are the actions triggered by the end-user tapping this or clicking that. This means that the final event is to be triggered at the very same location of the user event and your automation tool has to differentiate between those events. Also try to think in advance as users are unpredictable. Test how the app manages inputs of invalid data.
The perfect Automation cycle
- Test-&-result protocol. It has to be defined clear and accurate. Especially at the actual ‘reason of failure’ part.
- Management of the devices. Make sure there is an appropriate system that manages and categorizes your devices.
- Test case management. With the vast amount of test cases that will be emerging a proper system of managing and categorizing those also has to be present. And may also be automated meaning test cases will be assigned to their very own target devices, etc.
- Proper result reporting system. This includes the statistics of crucial defects.
Image via Matt Groening