IssueTracker

Thanks for your detailed post.

However, benchmark build type is under configured: at least isProfileable is not set to true for existing build type, probably there's more.

We should set isProfileable = true by default when overriding an existing benchmark build type.

This issue is not about some specific configuration flag, but the general approach of dealing with external configuration. As a developer adopting baseline profiles, it seems extremely risky to me using a custom configuration due to how it's applied under the hood and the fact it may break default configuration.

I agree that is not great but this is a little tricky to do. For custom baseline profile build types we override all the properties. For benchmark I left it open to configure but it's mostly about these 2 properties:

isMinifyEnabled
isShrinkResources

I don't have a way to see if the user is setting them before overriding, so for this reason, I'd prefer not to. I agree with you that some other properties could be set by default to make this easier, i.e.:

isJniDebuggable = false
isDebuggable = false
isProfileable = true

The reason why I mentioned the release signing config in the beginning is because I want to use debug signing config.

In the specific of your issue, i.e. using a debug certificate can you override the benchmark and baseline profile setting? You should be able to do something like:

android {
    buildTypes {
        release { ... }
        debug { ... }
        benchmarkRelease {
            ...
            signingConfig signingConfigs.debug
        }
        nonMinifiedRelease {
            ...
            signingConfig signingConfigs.debug
        }
    }
}

Project: platform/frameworks/support
Branch: androidx-main
Author: Marcello Albano <maralb@google.com>
Link:      https://android-review.googlesource.com/3303459

Added override for debuggable and profileable for benchmark builds in bpgp

Added override for debuggable and profileable for benchmark builds in bpgp

Test: ./gradlew :benchmark:benchmark-baseline-profile-gradle-plugin:test
Bug: 369213505
Relnote: "isProfileable is always overridden in benchmark builds,
  and isDebuggable is also now always overridden in both benchmark and
  nonMinified (baseline profile capture) builds."
Change-Id: I487fa71083921682173f04fcbb477be5baf165f8

I've landed a change that will set the following properties also when the benchmark build type already exists:

isJniDebuggable = false
isDebuggable = false
isProfileable = true

As well as the following for agp 8.0:

isDebuggable = false

I'm going ahead and closing this - if you've further questions please answer here and will reopen. Thanks.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-baseline-profile-gradle-plugin:1.4.0-alpha04

Since release signing config is used by default instead of debug is not mentioned in release notes - maybe this is a bug?

Cause the last time I found it mentioned in the release notes was in version 1.1

signingConfig.debug is used as the default signing config (b/153583269)

So, if the switch to the release one indeed happened - maybe it's an issue?

The new build types should inherit the certificates from the release build type, unless none is specified in which case debug is used: code link

This should be updated in the documentation, thanks for mentioning it. Will take care of updating it.

@chris is this bug fixed?

Marking as fixed since the default is now working correctly, filed b/316174880 to make further improvements and reenable tracing when it's safe, warning when it's not.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-common:1.3.0-alpha01

This isn't fully addressed - we noticed that the error for measurement after method tracing isn't on by default, working on a CL for that.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-common:1.3.0-beta01

Romain saw this on an API 35 device, reopened the ART bug, and marking this blocked on that.

Project: platform/frameworks/support
Branch: androidx-main
Author: Chris Craik <ccraik@google.com>
Link:      https://android-review.googlesource.com/3460462

Fix method tracing affecting measurements when art built from source on API 34

Fix method tracing affecting measurements when art built from source on API 34 
 
Fixes: 303660864 
Test: TrivialKotlinBenchmark (with duplicated benchmark on VP1A.230706.001) 
Test: DeviceInfoTest 
 
Change-Id: Ic7cad122150c515779365d6565f1c62c69213f82

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-common:1.4.0-alpha07
• androidx.benchmark:benchmark-macro:1.4.0-alpha07