Introduction
Device Farm is an app testing service that enables you to test and interact as an AWS engineer with your Android, iOS, and Web apps on real, physical phones and tablets that are hosted by Amazon Web Services (AWS). There are two main ways to use Device Farm:
- Automated testing of apps using a variety of available testing frameworks
- Remote access of devices onto which you can load, run, and interact with apps in real time
Device Farm allows you to upload your own tests or use built-in, script-free compatibility tests. Because testing is automatically performed in parallel, tests on multiple devices begin in minutes.
A test report containing high-level results, low-level logs, pixel-to-pixel screenshots, and performance data is updated as tests are completed.
Device Farm supports testing of native and hybrid Android, iOS, and Fire OS apps, including those created with PhoneGap, Titanium, Xamarin, Unity, and other frameworks. It also supports remote access to Android apps for interactive testing.
Device Farm currently provides support for the following test types:
Note: For the development of the automated tests, we used the Appium JUnit framework for the “Calculator” application on Android devices.
Prepare Android Appium Java JUnit Tests for running on AWS Device Farm
Android Appium Java JUnit tests must be contained in a .zip file. The POM file needs to be set up according to the official AWS Device Farm documentation for JUnit.
The following dependencies should be added to the maven POM file:
<p class="has-text-align-left"><br><script></script></p>Go into your Appium Maven project directory from the command prompt and run the following command to package the test content.
mvn clean package -DskipTests=true
After execution of this command, the “zip-with-dependencies.zip” file will be created in the target folder, which will contain .jar files of the test classes and all dependency .jar files.
Note: The “zip-with-dependencies.zip” file needs to be uploaded when creating a run on AWS Device Farm.
Create a test run on AWS Device Farm
- Sign in to the Device Farm console at https://console.aws.amazon.com/devicefarm
2. Click on “Create a new project”, enter a project name, and click on the “Create project” button.
3. Click on the “Create a new run” button on the project page.
4. On the “Choose your application” page, choose either Native application (the Android and Apple button) or Web application (the HTML5 button). Upload your application file and click on the “Next step” button.
5. On Configure a test page, select test type, ex. “Appium Java JUnit”, upload the “zip-with-dependencies.zip” file. It will take a few minutes for the tests to upload, after the test upload is finished select “Appium version” and click on the “Next step” button.
6. Select the device (ex. Huawei P9) on “Select devices” and click on “Next step” button.
7. Click on the “Next step” button on the “Specify device state” page.
8. On “Review and start run” page set the execution timeout per device “ex. 10 minutes” and click on “Confirm and start run”.
9. A report in Device Farm contains information about a run, which is a request for Device Farm to test a single app against one or more devices. Reports in Device Farm contain pass and fail information, crash reports, test and device logs, screenshots, and performance data. Reports include both detailed per-device data as well as high-level results, such as the number of occurrences of a given problem.
9.1 The “Video” section displays a downloadable video recording of the test.
9.2 The “Suites” section displays all test steps that contain pass and fail information and time of execution for each test step.
9.3 The “Logs” section displays any information Device Farm logged during the test.
9.4 The “Performance” section displays information about any performance data Device Farm generated during the test.
Conclusion
AWS Device Farm is easy to use and offers a very detailed view of the results, providing all the information needed to reproduce the bug. The main advantage is the possibility to execute tests on a wide range of devices, allowing us to discover problems, which can’t be seen on the local test devices.
