IssueTracker

Thanks for your detailed post.

However, benchmark build type is under configured: at least isProfileable is not set to true for existing build type, probably there's more.

We should set isProfileable = true by default when overriding an existing benchmark build type.

This issue is not about some specific configuration flag, but the general approach of dealing with external configuration. As a developer adopting baseline profiles, it seems extremely risky to me using a custom configuration due to how it's applied under the hood and the fact it may break default configuration.

I agree that is not great but this is a little tricky to do. For custom baseline profile build types we override all the properties. For benchmark I left it open to configure but it's mostly about these 2 properties:

isMinifyEnabled
isShrinkResources

I don't have a way to see if the user is setting them before overriding, so for this reason, I'd prefer not to. I agree with you that some other properties could be set by default to make this easier, i.e.:

isJniDebuggable = false
isDebuggable = false
isProfileable = true

The reason why I mentioned the release signing config in the beginning is because I want to use debug signing config.

In the specific of your issue, i.e. using a debug certificate can you override the benchmark and baseline profile setting? You should be able to do something like:

android {
    buildTypes {
        release { ... }
        debug { ... }
        benchmarkRelease {
            ...
            signingConfig signingConfigs.debug
        }
        nonMinifiedRelease {
            ...
            signingConfig signingConfigs.debug
        }
    }
}

Project: platform/frameworks/support
Branch: androidx-main
Author: Marcello Albano <maralb@google.com>
Link:      https://android-review.googlesource.com/3303459

Added override for debuggable and profileable for benchmark builds in bpgp

Added override for debuggable and profileable for benchmark builds in bpgp

Test: ./gradlew :benchmark:benchmark-baseline-profile-gradle-plugin:test
Bug: 369213505
Relnote: "isProfileable is always overridden in benchmark builds,
  and isDebuggable is also now always overridden in both benchmark and
  nonMinified (baseline profile capture) builds."
Change-Id: I487fa71083921682173f04fcbb477be5baf165f8

I've landed a change that will set the following properties also when the benchmark build type already exists:

isJniDebuggable = false
isDebuggable = false
isProfileable = true

As well as the following for agp 8.0:

isDebuggable = false

I'm going ahead and closing this - if you've further questions please answer here and will reopen. Thanks.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-baseline-profile-gradle-plugin:1.4.0-alpha04

Since release signing config is used by default instead of debug is not mentioned in release notes - maybe this is a bug?

Cause the last time I found it mentioned in the release notes was in version 1.1

signingConfig.debug is used as the default signing config (b/153583269)

So, if the switch to the release one indeed happened - maybe it's an issue?

The new build types should inherit the certificates from the release build type, unless none is specified in which case debug is used: code link

This should be updated in the documentation, thanks for mentioning it. Will take care of updating it.

The patch should be in baseline profile gradle plugin 1.2.4 and 1.3.0-alpha03

The following release(s) address this bug.It is possible this bug has only been partially addressed:

• androidx.benchmark:benchmark-baseline-profile-gradle-plugin:1.2.4
• androidx.benchmark:benchmark-baseline-profile-gradle-plugin:1.3.0-alpha03

I can still reproduce this in 1.3.0-alpha03. I've pushed a new branch. To reproduce:

• run ./gradlew assembleDevBenchmarkRelease
• the android/app/build/intermediates/merged_startup_profile/devBenchmarkRelease/mergeDevBenchmarkReleaseStartupProfile/ folder is still empty
• the android/app/build/intermediates/merged_art_profile/devBenchmarkRelease/mergeDevBenchmarkReleaseArtProfile/baseline-prof.txt only has profiles from libraries
• the build completes even when the baseline-prof.txt and startup-prof.txt in src/main/generated/baselineProfiles have unparsable content foo and bar.

Was looking at the project and I think the issue is here:

    beforeVariants {
        it.enable = it.name in listOf(
            "devDebug",
            "demoDebug",
            "mockDebug",
            "prodRelease",
            "devNonMinifiedRelease",
            "devBenchmarkRelease",
            "prodBenchmarkRelease",
        )
    }

In this beforeVariants block, the variant devRelease is disabled, so the onVariants callback for that variant won't run. The set up work for devBenchmarkRelease is done in that callback.

Unfortunately this code is a bit tricky because it's not possible to access a variant from another variant, using the variant api. So the baseline profile plugin processes the base variant and then schedules some work for when the benchmark variant is processed, in order to add the baseline profile src sets.

The (non-optimal) "fix" here would be to re-enable devRelease.

For reference: code.

Thanks, enabling devRelease does work. For now I've worked around this by enabling benchmark related variants only when running benchmarks by checking gradle.startParameter.