Baseline profile macrobenchmarks failing in CI [192084204]

Fixed

Bug

Status Update

No update yet.

Description

cc...@google.com

created issue #1

Jun 25, 2021 06:04PM

See https://blackbox.googleplex.com/dashboard/androidx/target/aosp-androidx-main/androidx_device_tests-sailfish-userdebug-29/6/MetricTests/2

Baseline profile benchmarks in androidx.compose.integration.macrobenchmark package are failing (on Sailfish API 29), with an exception like:

java.lang.RuntimeException: The baseline profile install broadcast was not received. This most likely means that the profileinstaller library is not in the target APK. It must be in order to use CompilationMode.BaselineProfile.
java.lang.RuntimeException: The baseline profile install broadcast was not received. This most likely means that the profileinstaller library is not in the target APK. It must be in order to use CompilationMode.BaselineProfile.
	at androidx.benchmark.macro.CompilationModeKt.compile(CompilationMode.kt:121)
	at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmark(Macrobenchmark.kt:129)
	at androidx.benchmark.macro.MacrobenchmarkKt.macrobenchmarkWithStartupMode(Macrobenchmark.kt:241)
	at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated(MacrobenchmarkRule.kt:68)
	at androidx.benchmark.macro.junit4.MacrobenchmarkRule.measureRepeated$default(MacrobenchmarkRule.kt:58)
	at androidx.testutils.MacrobenchUtilsKt.measureStartup(MacrobenchUtils.kt:34)
	at androidx.testutils.MacrobenchUtilsKt.measureStartup$default(MacrobenchUtils.kt:28)
	at androidx.compose.integration.macrobenchmark.IoSettingsStartupBenchmark.startup(IoSettingsStartupBenchmark.kt:40)

Sample failure: https://android-build.googleplex.com/builds/tests/view?invocationId=I66500009645614737&redirect=http%3A%2F%2Fab%2FI66500009645614737&testResultId=TR79926289235033187

Comments

cc...@google.com <cc...@google.com> #2Jun 29, 2021 11:59PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit c60f33e229e31ab328ef6b59dab63b264954831c
Author: Alexandre Elias <aelias@google.com>
Date: Fri Jul 10 16:23:09 2020

Semantics no-op cleanups

Partly in response to lmr's broad code review, I did a pass of
superficial API/implementation cleanup. The main changes are:

- I changed each Boolean SemanticsProperty where false is equivalent to
not being present to take "Unit" instead. This is conceptually
clearer: it avoids questions like "can I cancel out a semantics from a
merged child by setting it to false?" Because "property = Unit" looks
weird, I also changed the style of these to "property()".

- I moved the Semantics id generator closer to where it's used, in
SemanticsModifierCore. I made it internal and an AtomicInt.
(Note that integer ids are heavily used in the Android
AccessibilityNodeInfo APIs so I can't simply remove them entirely.)

- I deleted dead code. Some examples include SemanticsHintOverrides,
a public API not connected to anything, and SemanticsPropertyKey
merge() open method which is never called. (In both cases I have
a different plan in mind for accessibility.)

Fixes: 145951226
Fixes: 145955412
Test: existing tests
Relnote: "Single-value semantics properties now use a calling style.
For example, 'semantics { hidden = true }' is now written as:
'semantics { hidden() }'."

Change-Id: Ic1afd12ea22c926babc9662f1804d80b33aa0cfc

M ui/integration-tests/benchmark/src/androidTest/java/androidx/ui/benchmark/test/LayoutNodeModifierBenchmark.kt
M ui/ui-core/api/0.1.0-dev15.txt
M ui/ui-core/api/current.txt
M ui/ui-core/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-core/api/public_plus_experimental_current.txt
M ui/ui-core/api/restricted_0.1.0-dev15.txt
M ui/ui-core/api/restricted_current.txt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/graphics/vector/VectorTest.kt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/semantics/SemanticsTests.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidActuals.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeView.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeViewAccessibilityDelegateCompat.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidPopup.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/Expect.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsConfiguration.kt
D ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsHintOverrides.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsModifier.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsNode.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsOwner.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsWrapper.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/semantics/SemanticsProperties.kt
M ui/ui-foundation/api/0.1.0-dev15.txt
M ui/ui-foundation/api/current.txt
M ui/ui-foundation/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-foundation/api/public_plus_experimental_current.txt
M ui/ui-foundation/api/restricted_0.1.0-dev15.txt
M ui/ui-foundation/api/restricted_current.txt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Clickable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Dialog.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Scroller.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Selectable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Toggleable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/semantics/FoundationSemanticsProperties.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ButtonTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CardTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CheckboxScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/RadioButtonScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ScaffoldTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SnackbarTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SurfaceTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ripple/RippleIndicationTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/textfield/TextFieldScreenshotTest.kt
M ui/ui-material/src/main/java/androidx/ui/material/AppBar.kt
M ui/ui-material/src/main/java/androidx/ui/material/TextFieldImpl.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/AssertsTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/CallSemanticsActionTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ErrorMessagesTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/FindersTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/PrintToStringTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ScrollToTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/TextActionsTest.kt
M ui/ui-test/src/main/java/androidx/ui/test/Actions.kt
M ui/ui-test/src/main/java/androidx/ui/test/Filters.kt
M ui/ui-text/api/0.1.0-dev15.txt
M ui/ui-text/api/current.txt
M ui/ui-text/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-text/api/public_plus_experimental_current.txt
M ui/ui-text/api/restricted_0.1.0-dev15.txt
M ui/ui-text/api/restricted_current.txt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/CoreTextField.kt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/TextSemanticsProperties.kt

https://android-review.googlesource.com/1360099

le...@google.com <le...@google.com> Jun 30, 2021 07:45PM

Reassigned to ra...@google.com.

ra...@google.com <ra...@google.com> #3Jun 30, 2021 08:23PM

Accepted by ra...@google.com.

This is happening because we call MacrobenchmarkScope.killProcess() which triggers a force-stop on the package. This causes the broadcasts to not get deliverabled, until a user interacts with the target app.

ap...@google.com <ap...@google.com> #4Jul 2, 2021 10:11PM

Marked as fixed.

Project: platform/frameworks/support
Branch: androidx-main

commit a5e7913de40ac67088647fa80ed20bb027857fc7
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Fri Jul 02 10:13:23 2021

Uses explicit broadcasts to deliver intents to receivers.

* This is because the app is going to be force-stopped when running the
Macrobenchmark.
* Also add Jetifier migration configuration for Profile installer.

Fixes:

b/192084204

Test: Compose macrobenchmarks pass successfully.
Relnote: Use an `EmptyActivity` to bring the target app out of a force-stopped state to better support `CompilationMode.BaselineProfile`.

06/30 13:48:51: Launching 'IoSettingsStartupBen...' on Google Pixel 5.
Install successfully finished in 2 s 119 ms.
Running tests

$ adb shell am instrument -w -m -e androidx.benchmark.output.enable true --no-window-animation -e debug false -e class 'androidx.compose.integration.macrobenchmark.IoSettingsStartupBenchmark' androidx.compose.integration.macrobenchmark.test/androidx.test.runner.AndroidJUnitRunner
Connected to process 32016 on device 'google-pixel_5-localhost:33237'.
IoSettingsStartupBenchmark_startup[startup=HOT,compilation=None]
startupMs min 60, median 63, max 71
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=HOT,compilation=SpeedProfile(iterations=3)]
startupMs min 42, median 60, max 62
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=HOT,compilation=BaselineProfile]
startupMs min 51, median 56, max 66
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=WARM,compilation=None]
startupMs min 60, median 69, max 71
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=WARM,compilation=SpeedProfile(iterations=3)]
startupMs min 56, median 58, max 66
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=WARM,compilation=BaselineProfile]
startupMs min 52, median 55, max 58
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=COLD,compilation=None]
startupMs min 53, median 55, max 58
Traces: Iteration 0 1 2

IoSettingsStartupBenchmark_startup[startup=COLD,compilation=SpeedProfile(iterations=3)]
startupMs min 54, median 62, max 63
Traces: Iteration 0 1 2

Change-Id: Id7cac284a5a42f5aeaf92ba6522061f794afcc96

M benchmark/macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M jetifier/jetifier/migration.config

https://android-review.googlesource.com/1752966

Issue 192084204

Description

Issue summary

Comments

cc...@google.com <cc...@google.com> #2Jun 29, 2021 11:59PM

le...@google.com <le...@google.com> Jun 30, 2021 07:45PM

ra...@google.com <ra...@google.com> #3Jun 30, 2021 08:23PM

ap...@google.com <ap...@google.com> #4Jul 2, 2021 10:11PM

Add comment

Issue metadata