Fixed
Status Update
Comments
cc...@google.com <cc...@google.com> #2
Thanks, i was able to reproduce. It's a bit odd because it happens only when flavors are specified and generation is for a single variant.
cc...@google.com <cc...@google.com> #3
Found the issue, fix should be in the next 1.3 release.
Thank you!
ra...@google.com <ra...@google.com>
ra...@google.com <ra...@google.com> #4
Project: platform/frameworks/support
Branch: androidx-main
commit c16b952992febb4660bd9158d7c9813b5c9bbd87
Author: Marcello Albano <maralb@google.com>
Date: Mon Jun 03 13:13:02 2024
Ensure merge art and startup profiles always waits for profile generation
Test: added
Bug: 343086054
Change-Id: I623d65b8fedc7abb1d22fb2ca016582703f51176
M benchmark/baseline-profile-gradle-plugin/src/main/kotlin/androidx/baselineprofile/gradle/consumer/BaselineProfileConsumerPlugin.kt
M benchmark/baseline-profile-gradle-plugin/src/test/kotlin/androidx/baselineprofile/gradle/consumer/BaselineProfileConsumerPluginTest.kt
M benchmark/integration-tests/baselineprofile-producer/src/main/java/androidx/benchmark/integration/baselineprofile/producer/BaselineProfileTest.kt
https://android-review.googlesource.com/3111647
Branch: androidx-main
commit c16b952992febb4660bd9158d7c9813b5c9bbd87
Author: Marcello Albano <maralb@google.com>
Date: Mon Jun 03 13:13:02 2024
Ensure merge art and startup profiles always waits for profile generation
Test: added
Bug: 343086054
Change-Id: I623d65b8fedc7abb1d22fb2ca016582703f51176
M benchmark/baseline-profile-gradle-plugin/src/main/kotlin/androidx/baselineprofile/gradle/consumer/BaselineProfileConsumerPlugin.kt
M benchmark/baseline-profile-gradle-plugin/src/test/kotlin/androidx/baselineprofile/gradle/consumer/BaselineProfileConsumerPluginTest.kt
M benchmark/integration-tests/baselineprofile-producer/src/main/java/androidx/benchmark/integration/baselineprofile/producer/BaselineProfileTest.kt
cc...@google.com <cc...@google.com>
cc...@google.com <cc...@google.com> #5
Thanks. I also kindly ask you to escalate 342316793. For my team it's a very severe issue that causes problems when running baseline generation in CI env.
ap...@google.com <ap...@google.com> #6
The following release(s) address this bug.It is possible this bug has only been partially addressed:
androidx.benchmark:benchmark-baseline-profile-gradle-plugin:1.3.0-beta01
ap...@google.com <ap...@google.com> #7
Project: platform/frameworks/support
Branch: androidx-main
commit 0ba5c3dad9652a85319894f9aa7ad52664be72d2
Author: Chris Craik <ccraik@google.com>
Date: Tue Oct 18 20:56:23 2022
Fix deleteShaderCache by using Profileinstaller or fallback correctly with root
Test: SmallListStartupBenchmark
Test: MacrobenchmarkScopeTest
Fixes: 231455742
Relnote: """
BENCHMARK RELEASE NOTES
Fixed MacrobenchmarkScope.dropShaderCache(). This removes roughly 20ms
of noise from StartupMode.COLD benchmarks, as shaders are now
consistently cleared each iteration. Previously, `Partial` compilation
using warmup iterations would report incorrectly fast numbers, as
shader caching was more likely to happen during warmup. This fix
requires either a rooted device, or using
`profileinstaller:1.3.0-alpha02` in the target app.
PROFILEINSTALLER RELEASE NOTES
Added a hook for benchmarks to drop the shader cache, to ensure
consistent performance for cold startups, especially when compiling
with profiles from warmup iterations. This update is required to
measure cold startups using `benchmark-macro-junit4:1.2.0-alpha05` or
later.
"""
Change-Id: Ia5171b0f40dd8ce6f64f5ccf0a33281a4d8b121e
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkScope.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/ProfileInstallBroadcast.kt
M profileinstaller/profileinstaller/api/current.txt
M profileinstaller/profileinstaller/api/public_plus_experimental_current.txt
M profileinstaller/profileinstaller/api/restricted_current.txt
A profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/BenchmarkOperation.java
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstallReceiver.java
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstaller.java
https://android-review.googlesource.com/2262662
Branch: androidx-main
commit 0ba5c3dad9652a85319894f9aa7ad52664be72d2
Author: Chris Craik <ccraik@google.com>
Date: Tue Oct 18 20:56:23 2022
Fix deleteShaderCache by using Profileinstaller or fallback correctly with root
Test: SmallListStartupBenchmark
Test: MacrobenchmarkScopeTest
Fixes: 231455742
Relnote: """
BENCHMARK RELEASE NOTES
Fixed MacrobenchmarkScope.dropShaderCache(). This removes roughly 20ms
of noise from StartupMode.COLD benchmarks, as shaders are now
consistently cleared each iteration. Previously, `Partial` compilation
using warmup iterations would report incorrectly fast numbers, as
shader caching was more likely to happen during warmup. This fix
requires either a rooted device, or using
`profileinstaller:1.3.0-alpha02` in the target app.
PROFILEINSTALLER RELEASE NOTES
Added a hook for benchmarks to drop the shader cache, to ensure
consistent performance for cold startups, especially when compiling
with profiles from warmup iterations. This update is required to
measure cold startups using `benchmark-macro-junit4:1.2.0-alpha05` or
later.
"""
Change-Id: Ia5171b0f40dd8ce6f64f5ccf0a33281a4d8b121e
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkScope.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/ProfileInstallBroadcast.kt
M profileinstaller/profileinstaller/api/current.txt
M profileinstaller/profileinstaller/api/public_plus_experimental_current.txt
M profileinstaller/profileinstaller/api/restricted_current.txt
A profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/BenchmarkOperation.java
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstallReceiver.java
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstaller.java
na...@google.com <na...@google.com> #8
The following release(s) address this bug.It is possible this bug has only been partially addressed:
androidx.benchmark:benchmark-macro:1.2.0-alpha07
androidx.profileinstaller:profileinstaller:1.3.0-alpha02
Description
Alert on dashboard:https://androidx-perf.skia.org/t/?begin=1651608277&end=1651608278&subset=all
CLs in build:https://android-build.googleplex.com/builds/jump-to-build/8534762
Caused byhttps://android-review.googlesource.com/c/platform/frameworks/support/+/2086023/ - Improve macrobench iter speed by optimizing shell commands
Regression only affects baseline profile and full compilation variants of cold startup benchmarks.
Need to see if this makes the numbers more or less correct, and why. Triaging to 1.1 for this reason.