Fixed
Status Update
No update yet.
Comments
se...@google.com
Rahul, if needed we can make changes to the skip code as well.
cc...@google.com
It's likely the skip code doesn't expect someone to call cmd package compile --reset <packagename>.
Do we need a parameter to the profile install broadcast to force reinstallation? Or can the skip code check if reset has been performed somehow?
ra...@google.com
When the ProfileInstaller is invoked via the broadcast, I think it should ignore skip code checks altogether.
ap...@google.com
Project: platform/frameworks/support
Branch: androidx-main
commit 94e856176e65acf13e87974d0cfc522766bff3c5
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Fri Aug 20 11:59:54 2021
Clear profiles for StartupMode.COLD and CompilationMode.None.
* With this change COLD startup benchmarks will become more reliable.
ART dumps profile state every N seconds to the `cur` profile path.
ART could decide to compile code on the fly, if methods are hot enough
across runs. We want to prevent this from happening for the first
iteration of a StartupMode.COLD & a CompilationMode.None benchmark.
Test: N/A
Bug:
Relnote: `StartupMode.COLD` and `CompilationMode.None` benchmarks are now more stable.
Change-Id: I770cd76def3add73a14bee1b6acb1deea8900e93
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
https://android-review.googlesource.com/1804377
Branch: androidx-main
commit 94e856176e65acf13e87974d0cfc522766bff3c5
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Fri Aug 20 11:59:54 2021
Clear profiles for StartupMode.COLD and CompilationMode.None.
* With this change COLD startup benchmarks will become more reliable.
ART dumps profile state every N seconds to the `cur` profile path.
ART could decide to compile code on the fly, if methods are hot enough
across runs. We want to prevent this from happening for the first
iteration of a StartupMode.COLD & a CompilationMode.None benchmark.
Test: N/A
Bug:
Relnote: `StartupMode.COLD` and `CompilationMode.None` benchmarks are now more stable.
Change-Id: I770cd76def3add73a14bee1b6acb1deea8900e93
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
ap...@google.com
Project: platform/frameworks/support
Branch: androidx-main
commit b255edc640d90610530169c5144356f5ba814f31
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Mon Aug 23 12:19:02 2021
Ensure ProfileInstaller always installs a profile for `CompilationMode.BaselineProfile`.
* Throw an error message, when the target APK is not using the latest `profileinstaller` version.
Test: While the numbers are still not what we expect to see, here are the current benchmarks:
```
SmallListStartupBenchmark_startup[startup=COLD,compilation=None]
startupMs min 256, median 258, max 316
Traces: Iteration 0 1 2
SmallListStartupBenchmark_startup[startup=COLD,compilation=BaselineProfile]
startupMs min 252, median 262, max 265
Traces: Iteration 0 1 2
SmallListStartupBenchmark_startup[startup=COLD,compilation=SpeedProfile(iterations=3)]
startupMs min 186, median 187, max 187
Traces: Iteration 0 1 2
```
Bug:
Change-Id: I426579600594e238b5b46adc20a6d4b33da3bab5
Relnote: Fix `ProfileInstaller` to make it easier for apps using baseline profiles to run MacroBenchmarks using `CompilationMode.BaselineProfile`.
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstallReceiver.java
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstaller.java
https://android-review.googlesource.com/1798393
Branch: androidx-main
commit b255edc640d90610530169c5144356f5ba814f31
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Mon Aug 23 12:19:02 2021
Ensure ProfileInstaller always installs a profile for `CompilationMode.BaselineProfile`.
* Throw an error message, when the target APK is not using the latest `profileinstaller` version.
Test: While the numbers are still not what we expect to see, here are the current benchmarks:
```
SmallListStartupBenchmark_startup[startup=COLD,compilation=None]
startupMs min 256, median 258, max 316
Traces: Iteration 0 1 2
SmallListStartupBenchmark_startup[startup=COLD,compilation=BaselineProfile]
startupMs min 252, median 262, max 265
Traces: Iteration 0 1 2
SmallListStartupBenchmark_startup[startup=COLD,compilation=SpeedProfile(iterations=3)]
startupMs min 186, median 187, max 187
Traces: Iteration 0 1 2
```
Bug:
Change-Id: I426579600594e238b5b46adc20a6d4b33da3bab5
Relnote: Fix `ProfileInstaller` to make it easier for apps using baseline profiles to run MacroBenchmarks using `CompilationMode.BaselineProfile`.
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstallReceiver.java
M profileinstaller/profileinstaller/src/main/java/androidx/profileinstaller/ProfileInstaller.java
Description
For a while, the benchmark was somewhat faster than uncompiled, but it appears to have swung back upward, I'm guessing due to the above CL.
Strangely
Possibly an issue with how profile timestamps are reused multiple times now?
I don't see an affect fromhttps://android-review.googlesource.com/c/platform/frameworks/support/+/1790831 , so not sure what's going on there.
We need some integration tests for macrobench + baseline profile.