IssueTracker

Thanks for your detailed post.

However, benchmark build type is under configured: at least isProfileable is not set to true for existing build type, probably there's more.

We should set isProfileable = true by default when overriding an existing benchmark build type.

This issue is not about some specific configuration flag, but the general approach of dealing with external configuration. As a developer adopting baseline profiles, it seems extremely risky to me using a custom configuration due to how it's applied under the hood and the fact it may break default configuration.

I agree that is not great but this is a little tricky to do. For custom baseline profile build types we override all the properties. For benchmark I left it open to configure but it's mostly about these 2 properties:

isMinifyEnabled
isShrinkResources

I don't have a way to see if the user is setting them before overriding, so for this reason, I'd prefer not to. I agree with you that some other properties could be set by default to make this easier, i.e.:

isJniDebuggable = false
isDebuggable = false
isProfileable = true

The reason why I mentioned the release signing config in the beginning is because I want to use debug signing config.

In the specific of your issue, i.e. using a debug certificate can you override the benchmark and baseline profile setting? You should be able to do something like:

android {
    buildTypes {
        release { ... }
        debug { ... }
        benchmarkRelease {
            ...
            signingConfig signingConfigs.debug
        }
        nonMinifiedRelease {
            ...
            signingConfig signingConfigs.debug
        }
    }
}

Project: platform/frameworks/support
Branch: androidx-main
Author: Marcello Albano <maralb@google.com>
Link:      https://android-review.googlesource.com/3303459

Added override for debuggable and profileable for benchmark builds in bpgp

Added override for debuggable and profileable for benchmark builds in bpgp

Test: ./gradlew :benchmark:benchmark-baseline-profile-gradle-plugin:test
Bug: 369213505
Relnote: "isProfileable is always overridden in benchmark builds,
  and isDebuggable is also now always overridden in both benchmark and
  nonMinified (baseline profile capture) builds."
Change-Id: I487fa71083921682173f04fcbb477be5baf165f8

I've landed a change that will set the following properties also when the benchmark build type already exists:

isJniDebuggable = false
isDebuggable = false
isProfileable = true

As well as the following for agp 8.0:

isDebuggable = false

I'm going ahead and closing this - if you've further questions please answer here and will reopen. Thanks.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-baseline-profile-gradle-plugin:1.4.0-alpha04

Since release signing config is used by default instead of debug is not mentioned in release notes - maybe this is a bug?

Cause the last time I found it mentioned in the release notes was in version 1.1

signingConfig.debug is used as the default signing config (b/153583269)

So, if the switch to the release one indeed happened - maybe it's an issue?

The new build types should inherit the certificates from the release build type, unless none is specified in which case debug is used: code link

This should be updated in the documentation, thanks for mentioning it. Will take care of updating it.

I was expecting more along the lines of 5% not 20%-30%. Do other benchmarks improve in a similar way?

I don't know, I only looked at the one to get it up and running - you can cherry pick the linked CL to try it out on other benchmarks.

I always use the command-line and a custom script to interpret the results so not being able to run them from Studio would not affect me

Depending on which benchmarks we want to modify, it may hit others who are accustomed to running from Studio. If we want to commit this, keeping it scoped to runtime only may make sense.

couldn't figure out how to get connectedCheck to recognize

I don't think I did anything special, though I did play around with sourcesets a lot. Are you able to run this CL locally? It may have been only working for me due to some cached gradle/studio state.

Just checking I follow your logic in #3 - the challenge is that you want all the test entry points to be roots for R8 to keep - which currently is not supported for app tests (and is potentially a bit odd for 'normal' app users as having the production shrunk build depending on tests might be suprising) and that moving them to the main sourceset breaks studio being able to run them.

So possibilities look like androidx only

• Create an app with the tests with different build types for shrunk/not shrunk
  • For studio support split the @Test bodies in to a class in src/main and add keep rules for that class, or manually curate keep rules. agp/studio
• Fix studio running tests that are in main in an app
• Support tracing @Tests in androidTest when shrinking the app in AGP
• Add special support to AGP for shrinking library tests
  • Shrinker support in library is
• Add test only module support for shrinking and library targets.

Yes, (not sure what you mean by androidx only, since some of them are general solutions) though I'd add another possibility is the AGP support for macrobenchmarks source directory within existing modules, as we discussed a while back.

Also, the more general bug for R8 + Microbenchmark is here: b/171496730, this bug was filed to track investigating temporary, compose-specific workarounds.

This is now possible with AGP changes, I was able to get simple tests running with minification on in a standalone project.

Once the required version of AGP lands in AndroidX, I'll add an example for local experimentation with R8 in microbenchmarks, and we can consider options for how we want to use this in CI.

These are the rules I used to test the integration in a standalone project:

# basic protection against junit/androidx.test reflection, shouldn't affect library/test code
-keepclasseswithmembers class androidx.test.** { *; }
-keepclasseswithmembers class org.junit.** { *; }
-dontwarn com.google.errorprone.annotations.MustBeClosed

## keep test classes
-keepclasseswithmembers @org.junit.runner.RunWith class * { *; }

## needed for org.junit.Test annotation to be discoverable by reflection
-keepattributes *Annotation*

With the above CL, this can be enabled locally with the following in your build.gradle:

    android {
        buildTypes.release.androidTest.enableMinification = true
    }

No further proguard customization needed in most cases.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-junit4:1.3.0-alpha02
• androidx.compose.foundation:foundation:1.7.0-alpha05
• androidx.compose.foundation:foundation-android:1.7.0-alpha05
• androidx.compose.foundation:foundation-desktop:1.7.0-alpha05