IssueTracker

Component used: baseline gradle plugin / benchmark
Version used: 1.2.2
Devices/Android versions reproduced on: Pixel 8 Pro, Emulator / Android 14

I'm trying to make reproducible project with b/313968931

And it seems I found another issue.

By https://developer.android.com/training/testing/instrumented-tests/androidx-test-libraries/runner?hl=en#enable-android, my gradle project set Androidx Test Orchestrator globally by build-logic

And when I added baselineprofile generator and perform generateBaselineProfile task, it shows following warning, and cannot generate baselineprofile result.

$ git clone https://git.linecorp.com/VersionCodeDemo -b baselineprofile_generator_1

$ cd VersionCodeDemo

$ ./gradlew :app:generateProductionReleaseBaselineProfile

> Task :app:mergeProductionReleaseBaselineProfile
No baseline profile rules were generated for the variant `productionRelease`.
This is most likely because there are no instrumentation test for it. If this
is not intentional check that tests for this variant exist in the `baselineProfile`
dependency module.
No startup profile rules were generated for the variant `productionRelease`.
This is most likely because there are no instrumentation test with baseline profile
rule, which specify `includeInStartupProfile = true`. If this is not intentional
check that tests for this variant exist in the `baselineProfile` dependency module.

> Task :app:copyProductionReleaseBaselineProfileIntoSrc
No baseline profile rules were generated for the variant `productionRelease`.
This is most likely because there are no instrumentation test for it. If this
is not intentional check that tests for this variant exist in the `baselineProfile`
dependency module.
No startup profile rules were generated for the variant `productionRelease`.
This is most likely because there are no instrumentation test with baseline profile
rule, which specify `includeInStartupProfile = true`. If this is not intentional
check that tests for this variant exist in the `baselineProfile` dependency module.

After I commented out execution = "ANDROIDX_TEST_ORCHESTRATOR", plugin can generate baselineprofile result like this

$ ./gradlew :app:generateProductionReleaseBaselineProfile

> Task :app:copyProductionReleaseBaselineProfileIntoSrc
A baseline profile was generated for the variant `productionRelease`:
file:///Users/user/AndroidStudioProjects/VersionCodeDemo/app/src/productionRelease/generated/baselineProfiles/baseline-prof.txt
A startup profile was generated for the variant `productionRelease`:
file:///Users/user/AndroidStudioProjects/VersionCodeDemo/app/src/productionRelease/generated/baselineProfiles/startup-prof.txt

I'm guessing several developers will enable Test Orchestrator in their projects, and it is hard to find why baselineprofile generator is not working.

Baseline Profile Generator should be friendly with Test Orchestrator or give warning when cannot generate result of baseline profile by Test Orchestrator