Macrobenchmarks - Regression found between 11748432 and 11748459 [336588271]

Fixed

Bug

Status Update

No update yet.

Description

sk...@skia-public.iam.gserviceaccount.com

created issue on behalf of an...@google.com #1

Apr 23, 2024 07:12PM

Perf Regression (High) found, matching 214 tracked metrics from benchmarks.

To triage this regression, see the guide at go/androidx-bench-triage.

Tests affected:

 androidx.benchmark.integration.macrobenchmark.GridBenchmark_scroll_compilation_WarmupProfile_iterations_3__
 androidx.benchmark.integration.macrobenchmark.SeparateProcessBenchmark_startup
 androidx.benchmark.integration.macrobenchmark.SmallListStartupBenchmark_startup_startup_COLD_compilation_None_
 androidx.benchmark.integration.macrobenchmark.SmallListStartupBenchmark_startup_startup_COLD_compilation_WarmupProfile_iterations_3__
 androidx.benchmark.integration.macrobenchmark.StartupPressHomeBenchmark_startup_pressHomeInMeasure_false_startup_WARM_
 androidx.benchmark.integration.macrobenchmark.StartupPressHomeBenchmark_startup_pressHomeInMeasure_true_startup_COLD_
 androidx.benchmark.integration.macrobenchmark.TrivialListScrollBenchmark_start_compilation_Full_
 androidx.benchmark.integration.macrobenchmark.TrivialListScrollBenchmark_start_compilation_None_
 androidx.compose.integration.hero.macrobenchmark.jetsnack.JetsnackScrollBenchmark_scrollHome_compilation_WarmupProfile_iterations_3__
 androidx.compose.integration.hero.macrobenchmark.jetsnack.JetsnackStartupBenchmark_startup_startup_COLD_compilationMode_None_
 androidx.compose.integration.hero.macrobenchmark.jetsnack.JetsnackStartupBenchmark_startup_startup_HOT_compilationMode_BaselineProfile_
 androidx.compose.integration.macrobenchmark.AndroidViewListScrollBenchmark_scroll_compilation_BaselineProfile_
 androidx.compose.integration.macrobenchmark.AndroidViewListScrollBenchmark_scroll_compilation_None_
 androidx.compose.integration.macrobenchmark.AndroidViewListScrollBenchmark_scroll_compilation_WarmupProfile_iterations_3__
 androidx.constraintlayout.compose.integration.macrobenchmark.MotionLayoutBenchmark_messageDsl
 androidx.constraintlayout.compose.integration.macrobenchmark.MotionLayoutBenchmark_messageJson
 androidx.profileinstaller.integration.macrobenchmark.ProfileinstallerStartupBenchmark_startup_startup_COLD_compilation_WarmupProfile_iterations_3__
 androidx.wear.benchmark.integration.macrobenchmark.ScrollBenchmark_start_compilation_BaselineProfile_
 androidx.wear.benchmark.integration.macrobenchmark.ScrollBenchmark_start_compilation_Full_
 androidx.wear.benchmark.integration.macrobenchmark.ScrollBenchmark_start_compilation_None_
 androidx.wear.benchmark.integration.macrobenchmark.ScrollBenchmark_start_compilation_WarmupProfile_iterations_3__
 androidx.wear.benchmark.integration.macrobenchmark.StartupBenchmark_startup_startup_HOT_compilation_BaselineProfile_
 androidx.wear.benchmark.integration.macrobenchmark.StartupBenchmark_startup_startup_WARM_compilation_BaselineProfile_
 androidx.wear.benchmark.integration.macrobenchmark.SwipeBenchmark_start_compilation_None_
 androidx.wear.benchmark.integration.macrobenchmark.SwipeBenchmark_start_compilation_WarmupProfile_iterations_3__
 androidx.wear.compose.integration.macrobenchmark.PositionIndicatorBenchmark_start_compilation_Full_
 androidx.wear.compose.integration.macrobenchmark.PositionIndicatorBenchmark_start_compilation_None_
 androidx.wear.compose.integration.macrobenchmark.PositionIndicatorBenchmark_start_compilation_WarmupProfile_iterations_3__
 androidx.wear.compose.integration.macrobenchmark.test.ScrollBenchmark_start_compilation_BaselineProfile_
 androidx.wear.compose.integration.macrobenchmark.test.ScrollBenchmark_start_compilation_Full_
 androidx.wear.compose.integration.macrobenchmark.test.ScrollBenchmark_start_compilation_WarmupProfile_iterations_3__
 androidx.wear.compose.integration.macrobenchmark.test.StartupBenchmark_startup_startup_COLD_compilation_Full_
 androidx.wear.compose.integration.macrobenchmark.test.SwipeBenchmark_start_compilation_BaselineProfile_
 androidx.wear.compose.integration.macrobenchmark.test.SwipeBenchmark_start_compilation_Full_
 androidx.wear.compose.integration.macrobenchmark.test.SwipeBenchmark_start_compilation_None_
 androidx.wear.compose.integration.macrobenchmark.test.SwipeBenchmark_start_compilation_WarmupProfile_iterations_3__

Devices affected:

 sailfish

API Level:

 API_29_Q

Comments

ra...@google.com <ra...@google.com> #2Apr 24, 2024 06:50PM

Accepted by ra...@google.com.

Project: platform/frameworks/support
Branch: androidx-master-dev

commit c60f33e229e31ab328ef6b59dab63b264954831c
Author: Alexandre Elias <aelias@google.com>
Date: Fri Jul 10 16:23:09 2020

Semantics no-op cleanups

Partly in response to lmr's broad code review, I did a pass of
superficial API/implementation cleanup. The main changes are:

- I changed each Boolean SemanticsProperty where false is equivalent to
not being present to take "Unit" instead. This is conceptually
clearer: it avoids questions like "can I cancel out a semantics from a
merged child by setting it to false?" Because "property = Unit" looks
weird, I also changed the style of these to "property()".

- I moved the Semantics id generator closer to where it's used, in
SemanticsModifierCore. I made it internal and an AtomicInt.
(Note that integer ids are heavily used in the Android
AccessibilityNodeInfo APIs so I can't simply remove them entirely.)

- I deleted dead code. Some examples include SemanticsHintOverrides,
a public API not connected to anything, and SemanticsPropertyKey
merge() open method which is never called. (In both cases I have
a different plan in mind for accessibility.)

Fixes: 145951226
Fixes: 145955412
Test: existing tests
Relnote: "Single-value semantics properties now use a calling style.
For example, 'semantics { hidden = true }' is now written as:
'semantics { hidden() }'."

Change-Id: Ic1afd12ea22c926babc9662f1804d80b33aa0cfc

M ui/integration-tests/benchmark/src/androidTest/java/androidx/ui/benchmark/test/LayoutNodeModifierBenchmark.kt
M ui/ui-core/api/0.1.0-dev15.txt
M ui/ui-core/api/current.txt
M ui/ui-core/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-core/api/public_plus_experimental_current.txt
M ui/ui-core/api/restricted_0.1.0-dev15.txt
M ui/ui-core/api/restricted_current.txt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/graphics/vector/VectorTest.kt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/semantics/SemanticsTests.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidActuals.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeView.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeViewAccessibilityDelegateCompat.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidPopup.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/Expect.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsConfiguration.kt
D ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsHintOverrides.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsModifier.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsNode.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsOwner.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsWrapper.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/semantics/SemanticsProperties.kt
M ui/ui-foundation/api/0.1.0-dev15.txt
M ui/ui-foundation/api/current.txt
M ui/ui-foundation/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-foundation/api/public_plus_experimental_current.txt
M ui/ui-foundation/api/restricted_0.1.0-dev15.txt
M ui/ui-foundation/api/restricted_current.txt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Clickable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Dialog.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Scroller.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Selectable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Toggleable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/semantics/FoundationSemanticsProperties.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ButtonTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CardTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CheckboxScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/RadioButtonScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ScaffoldTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SnackbarTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SurfaceTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ripple/RippleIndicationTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/textfield/TextFieldScreenshotTest.kt
M ui/ui-material/src/main/java/androidx/ui/material/AppBar.kt
M ui/ui-material/src/main/java/androidx/ui/material/TextFieldImpl.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/AssertsTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/CallSemanticsActionTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ErrorMessagesTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/FindersTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/PrintToStringTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ScrollToTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/TextActionsTest.kt
M ui/ui-test/src/main/java/androidx/ui/test/Actions.kt
M ui/ui-test/src/main/java/androidx/ui/test/Filters.kt
M ui/ui-text/api/0.1.0-dev15.txt
M ui/ui-text/api/current.txt
M ui/ui-text/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-text/api/public_plus_experimental_current.txt
M ui/ui-text/api/restricted_0.1.0-dev15.txt
M ui/ui-text/api/restricted_current.txt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/CoreTextField.kt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/TextSemanticsProperties.kt

https://android-review.googlesource.com/1360099

cc...@google.com <cc...@google.com> #3Apr 25, 2024 09:50PM

This is expected for now. We are now turning on method traces where we did not do so previously. Given method tracing is turned-on in CI automatically, this will start affecting all Macrobenchmarks.

It should not be, even without your follow up CL - we turn on method tracing by default in micro, but macro attempts to avoid that - https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:benchmark/benchmark-common/src/main/java/androidx/benchmark/Arguments.kt;l=288;drc=e8c21d81df2242168e21b1310b62609887e70406

ap...@google.com <ap...@google.com> #4Apr 26, 2024 07:00PM

Marked as fixed.

Project: platform/frameworks/support
Branch: androidx-main

commit 95c0ded0f9d7b0853d72a732e57d86032ae5f568
Author: Chris Craik <ccraik@google.com>
Date: Fri Apr 26 10:49:10 2024

Fix method tracing to be off by default in macrobench

Fixes: 336588271
Test: SmallListStartupBenchmark (no tracing)

Change-Id: I9896030622be844e631c8aff69ba55c01f0661d7

M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt

https://android-review.googlesource.com/3062971

ap...@google.com <ap...@google.com> #5Apr 30, 2024 07:48PM

Project: platform/frameworks/support
Branch: androidx-main

commit 2d1785e9837b97c6d64878413efb96d873895cd0
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Wed Apr 24 16:57:02 2024

Move method tracing to a separate phase.

* Method tracing now runs as a separate phase, without interfering with metrics.

Fixes: 285912360
Fixes: 336588271
Test: Existing unit tests.
Change-Id: If9a504a7bce7228840339b34294ba3fdf98aceeb
Relnote: Method tracing runs as a seperate phase during a Macrobenchmark, and it no longer affects measurements.

M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkPhase.kt

https://android-review.googlesource.com/3059411

pr...@google.com <pr...@google.com> #6May 14, 2024 05:22PM

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.benchmark:benchmark-macro:1.3.0-alpha05

Issue 336588271