IssueTracker

Context

• UI testing fails if 0 tests ran on the test device.

Scenario

A project with multiple application modules chooses to enable test sharding using GMD. One application module has 1 UI test while another has 20. If the project has a shard param of 4 (-P android.experimental.androidTest.numManagedDeviceShards=4), then the application module with 1 UI test will cause a failure of the build due to 0 tests running on 3 of the devices, e.g.

Workaround

• A project can have its UI testing tasks run on different application modules as separate ./gradlew executions, but doing so results in a much longer test run time as it necessitates the building of the project for both targets.
• Another workaround is to fluff the low-test targets with assert(true) tests until seeing successful runs (noting that sharding may choose to put 2 tests on device A, 0 on device B, 1 on C, and 1 on D, still failing the test in this scenario, i.e. tests are not distributed evenly)

Proposed solution

• Change numManagedDeviceShards to maxManagedDeviceShards such that any particular GMD run would choose to run the maximum possible shards for the number of tests available, only fail on 0 if there are 0 total tests run