In this article I am going to talk about achieving the goal of Automation testing for Android apps. For last few years there has been huge emphasis on automating the testing process especially for Mobile Applications the reasons being:
- Requires less number of Quality Assurance Engineers.
- Automated tests are fast.
- Not many resources are required to maintain the test suite.
But the ground reality is that it takes a lot of efforts to maintain the automated tests so that they bring good value to the release process especially in the case of Android Apps. In my experience there is no such state during the automation process when one can say that they do not require any manual intervention in their app testing.
Let’s talk about Espresso testing for Android Application. I admit that Espresso is one of the best tools for Android Automation testing but Espresso tests have their limitations. For example: Testing Network calls from espresso tests is not easy and developers have to put in a lot of efforts to make it possible because espresso is not aware of asynchronous operations. Testing the flow of an android application is a doable task with Espresso but is often confused with testing logics inside the application.
Another article The 3 tiers of the Android test pyramid by Chuck Greb talks about the Android Testing Pyramid and beautifully explains the importance of each block.
The UI tests are at the top because ideally the UI tests should be minimum but effective. In my opinion UI tests should satisfy the following criteria only:
- App flow which cannot be tested using Unit testing. For example: when a button is clicked and the app navigates from screen 1 to screen 2, the UI test should test that the navigation happens to the correct screen.
- The UI tests should never test the logics of a screen, the logic testing should be done in Unit Testing only. In my opinion it is the responsibility of Pull Request reviewer to make sure that the logic is always tested in Unit Tests and not in UI tests.
- Accessibility testing: Automated testing is a good point where accessibility testing can be done.
Espresso tests are heavy and take a lot of resources and money to run. A simple setup includes setting up real devices or emulators on cloud. Just setting up those devices and making sure that they keep running on those particular configuration is not easy. For example: Often we have to turn off animation on Android devices in order for espresso tests to run and we have to make sure make sure that the cloud devices/emulators have animation off. There are many times when the devices become unresponsive in middle of test running and the result of that is failing test.
In my opinion even after making sure that there are adequate UI tests it is very important that there is one Quality Assurance Engineer who will test the application. For example: enlarging the device font and making sure that the all the text in the application fits properly or all the network calls are happening the way they should be. A QA engineer also makes sure that the app has followed the business flow along with making sure that the app is not crashing unexpectedly. Another example is clicking a button 50 times only a QA can do that.
