Status Update
Comments
cc...@google.com <cc...@google.com>
ra...@google.com <ra...@google.com> #2
Branch: androidx-master-dev
commit c60f33e229e31ab328ef6b59dab63b264954831c
Author: Alexandre Elias <aelias@google.com>
Date: Fri Jul 10 16:23:09 2020
Semantics no-op cleanups
Partly in response to lmr's broad code review, I did a pass of
superficial API/implementation cleanup. The main changes are:
- I changed each Boolean SemanticsProperty where false is equivalent to
not being present to take "Unit" instead. This is conceptually
clearer: it avoids questions like "can I cancel out a semantics from a
merged child by setting it to false?" Because "property = Unit" looks
weird, I also changed the style of these to "property()".
- I moved the Semantics id generator closer to where it's used, in
SemanticsModifierCore. I made it internal and an AtomicInt.
(Note that integer ids are heavily used in the Android
AccessibilityNodeInfo APIs so I can't simply remove them entirely.)
- I deleted dead code. Some examples include SemanticsHintOverrides,
a public API not connected to anything, and SemanticsPropertyKey
merge() open method which is never called. (In both cases I have
a different plan in mind for accessibility.)
Fixes: 145951226
Fixes: 145955412
Test: existing tests
Relnote: "Single-value semantics properties now use a calling style.
For example, 'semantics { hidden = true }' is now written as:
'semantics { hidden() }'."
Change-Id: Ic1afd12ea22c926babc9662f1804d80b33aa0cfc
M ui/integration-tests/benchmark/src/androidTest/java/androidx/ui/benchmark/test/LayoutNodeModifierBenchmark.kt
M ui/ui-core/api/0.1.0-dev15.txt
M ui/ui-core/api/current.txt
M ui/ui-core/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-core/api/public_plus_experimental_current.txt
M ui/ui-core/api/restricted_0.1.0-dev15.txt
M ui/ui-core/api/restricted_current.txt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/graphics/vector/VectorTest.kt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/semantics/SemanticsTests.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidActuals.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeView.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeViewAccessibilityDelegateCompat.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidPopup.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/Expect.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsConfiguration.kt
D ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsHintOverrides.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsModifier.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsNode.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsOwner.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsWrapper.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/semantics/SemanticsProperties.kt
M ui/ui-foundation/api/0.1.0-dev15.txt
M ui/ui-foundation/api/current.txt
M ui/ui-foundation/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-foundation/api/public_plus_experimental_current.txt
M ui/ui-foundation/api/restricted_0.1.0-dev15.txt
M ui/ui-foundation/api/restricted_current.txt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Clickable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Dialog.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Scroller.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Selectable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Toggleable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/semantics/FoundationSemanticsProperties.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ButtonTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CardTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CheckboxScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/RadioButtonScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ScaffoldTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SnackbarTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SurfaceTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ripple/RippleIndicationTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/textfield/TextFieldScreenshotTest.kt
M ui/ui-material/src/main/java/androidx/ui/material/AppBar.kt
M ui/ui-material/src/main/java/androidx/ui/material/TextFieldImpl.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/AssertsTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/CallSemanticsActionTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ErrorMessagesTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/FindersTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/PrintToStringTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ScrollToTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/TextActionsTest.kt
M ui/ui-test/src/main/java/androidx/ui/test/Actions.kt
M ui/ui-test/src/main/java/androidx/ui/test/Filters.kt
M ui/ui-text/api/0.1.0-dev15.txt
M ui/ui-text/api/current.txt
M ui/ui-text/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-text/api/public_plus_experimental_current.txt
M ui/ui-text/api/restricted_0.1.0-dev15.txt
M ui/ui-text/api/restricted_current.txt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/CoreTextField.kt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/TextSemanticsProperties.kt
ap...@google.com <ap...@google.com> #3
Branch: androidx-main
commit 22fe26bc907af50d557254d406eb38469a7bb81b
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Fri Apr 16 11:15:58 2021
Add unbundled perfetto support.
With this change, we are much closer in bringing Macrobenchmarks to API 21.
The things we need to tackle after this CL lands are:
* Follow up on this CL (
The binaries checked in, include this change but currently its failing presubmits because of a compiler warning.
Something primiano@ is currently looking into.
* Expand trace_processor_shells ABI support to include arm, and x86_64.
Remove the explicit checks for API Q.
* Remove the use of @RequiresApi(...) in MacrobenchmarkRule, and explicit checks for API Q.
* verifyTraceEnabled(...) does seem to cause issues on lower versions of the platform. Something we need to investigate further.
This does not happen consistently though.
* Fix our perfetto trace configs. There are options that are unsupported on older versions of the platform.
* M, and N dont support several ART compilation modes. No op on them.
Test: PerfettoCaptureTest is now parameterized
Bug:
Change-Id: I888fa15e02acc4f99fbf548722a11a9953988357
M benchmark/macro/build.gradle
M benchmark/macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
M benchmark/macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoCapture.kt
M benchmark/macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoCaptureWrapper.kt
M benchmark/macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoHelper.kt
ap...@google.com <ap...@google.com> #4
Branch: androidx-main
commit 9f143a6b876ef5b2be841ed2ce1be138e325bf51
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Wed Mar 31 14:49:19 2021
Add stripped perfetto and trace_processor_shell binaries.
* These binaries are built from source (
* Build instructions are
* The config looks something like:
```
target_os = "android"
target_cpu = "arm|arm64|arm_x86_x86"
is_debug = false
monolithic_binaries = true
```
* After building, we stripped the binaries using ndk `22.1.7171670` using `/path/to/sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/<variant>-linux-android/bin/strip`
Test: ./gradlew :benchmark:integration-tests:macrobenchmark:connectedDebugAndroidTest
Bug:
Change-Id: I052b6caf214a3d0dfd101302040f6ae43bc42b22
M README.md
A perfetto/perfetto_aarch64
A perfetto/perfetto_arm
A perfetto/perfetto_x86_64
M trace_processor_shell/trace_processor_shell_aarch64
A trace_processor_shell/trace_processor_shell_arm
A trace_processor_shell/trace_processor_shell_x86_64
A traced/traced_aarch64
A traced/traced_arm
A traced/traced_x86_64
A traced_probes/traced_probes_aarch64
A traced_probes/traced_probes_arm
A traced_probes/traced_probes_x86_64
ap...@google.com <ap...@google.com> #5
Branch: androidx-main
commit ce3df5c24d36ac56873d13bb27a084e54e420dd7
Author: Rahul Ravikumar <rahulrav@google.com>
Date: Tue Apr 20 12:16:02 2021
Expand ABI support for `perfetto_trace_processor`.
Test: ./gradlew :benchmark:benchmark-macro:connectedCheck
Bug: Bug:
Change-Id: Ic7c5579b497890b2eab8f202b41d2a2c2560c1fb
M benchmark/macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
M benchmark/macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessor.kt
cc...@google.com <cc...@google.com>
cc...@google.com <cc...@google.com> #6
Punting from initial alpha - unfortunately, the trace signals that perfetto uses for startup metrics MetricsLogger:launchObserverNotifyIntentStarted
and launching: $packageName
were added in Q, so we'll need to figure out an alternative timing mechanism on older devices. Quite likely that looks like looking for the end of drawFrame
on RenderThread, but coming up with a clear definition with sample traces will take some time.
ca...@google.com <ca...@google.com> #7
ap...@google.com <ap...@google.com> #8
Branch: androidx-main
commit 7fd221c0c8f7948351293a7fbf75eef0bf698a42
Author: Chris Craik <ccraik@google.com>
Date: Fri Apr 30 17:19:22 2021
Fix output issue on API 21, suppress still-broken tests
Bug: 183129298
Test: ./gradlew benchmark:b-c:cC benchmark:b-m:cC
Shell can't access externalCacheDir (but somehow can access cacheDir)
on API 21.
Also on API 21, UiDevice.executeShellCommand doesn't capture the
output of `echo foo`. executeShellScript still partially works
(e.g. `ls` works), but we'll need more investigation to better
understand shell command limitations on that API level.
Change-Id: I2bafe8218ed50ac54cd990190d2f9a2e2364028d
M benchmark/common/src/main/java/androidx/benchmark/Outputs.kt
A benchmark/macro/src/androidTest/java/androidx/benchmark/macro/OutputsTest.kt
M benchmark/macro/src/androidTest/java/androidx/benchmark/macro/perfetto/ShellUtilsTest.kt
M benchmark/macro/src/main/java/androidx/benchmark/macro/perfetto/ShellUtils.kt
cc...@google.com <cc...@google.com> #9
(Coming back to this after I/O) Carmen - filed
ap...@google.com <ap...@google.com> #10
Branch: androidx-main
commit 3dff7103380e8f0561029da051bd444d8bbcd26b
Author: Chris Craik <ccraik@google.com>
Date: Wed Jul 28 16:58:05 2021
Only use atrace tags supported on the local platform version
Bug: 183129298
Test: PerfettoConfigTest
Test: AtraceTagTest
Codifies list of in-use atrace tags with which platform versions
they're supported on. Validated on API 21 -> 29 emulators.
Currently, the way we invoke perfetto doesn't tell us if starting the
process failed, so this eliminates one source of this failure.
Planned follow up work
- investigate shell wrapper issues on API 21/22.
- improve perfetto launch failures to throw clearly
- add additional config validation
- (low prior) stash shell session rooted status
Change-Id: I99f5db15d1d0ee84d340b6879ef363f41c4327b5
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
A benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/AtraceTagTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoConfigTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/ShellUtilsTest.kt
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/AtraceTag.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoCapture.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoConfig.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/ShellUtils.kt
ap...@google.com <ap...@google.com> #11
Branch: androidx-main
commit 48ae41ec2aada7a167afbd9e03ce67e9114aed23
Author: Chris Craik <ccraik@google.com>
Date: Thu Jul 29 09:43:38 2021
Fix ShellUtils on API 21/22 - chmod +x not supported
Bug: 183129298
Test: ShellUtilsTest # verified on API 21 through 29
Change-Id: I8f235b8044fb54f9f23129a62e8a732536900add
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/ShellUtilsTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/ShellUtils.kt
ap...@google.com <ap...@google.com> #12
Branch: androidx-main
commit e137997eab441062cc59ddaec31cb6609cd47039
Author: Chris Craik <ccraik@google.com>
Date: Thu Jul 29 10:21:32 2021
Validate PerfettoTraceProcessor back to API 21
Bug: 183129298
Test: PerfettoTraceProcessor # validated on API 21 through 26
Also adds fixed-trace query test.
Change-Id: I1a47f77c218d1918bd2273e3f6b1f43451f6ac76
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessor.kt
ap...@google.com <ap...@google.com> #13
Branch: androidx-main
commit c75fd5f903d15795ab43417debd86822c21bcca4
Author: Chris Craik <ccraik@google.com>
Date: Thu Jul 29 17:55:05 2021
Test trace capture back to API 28, macrobench scope to 27
Test: PerfettoCaptureTest
Test: MacrobenchmarkScopeTest
Bug: 183129298
Clarify with comment that perfetto support for O is blocking lowering,
as config doesn't appear to be the issue currently.
Throw if trying to use bundled version before 29, since its behavior
is unpredictable (passes on local Pixel2 API 28, but not Pixel1 API 28
in CI)
Additionally, switched bench:integ-tests:macrobenchmark to minSdk=29,
since 28 won't capture metrics yet.
Change-Id: I087c4b812df3a223ed239062d35a9b731cab39e7
M benchmark/benchmark-macro/build.gradle
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoHelper.kt
M benchmark/integration-tests/macrobenchmark/build.gradle
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialListScrollBenchmark.kt
ap...@google.com <ap...@google.com> #14
Branch: androidx-main
commit eeb028cd24aee126f6f89a59df531304e57ad937
Author: Chris Craik <ccraik@google.com>
Date: Fri Jul 30 15:30:20 2021
Lower MacrobenchmarkScope min API to 21
Test: MacrobenchmarkScopeTest
Test: ShellUtilsTest
Test: ShellBehaviorTest
Bug: 183129298
Fixes usage of ps -A and pidof.
Started documenting shell version inconsistencies with
ShellBehaviorTest, to be sure we understand which behavior is where,
as testing (local especially) will always be limited, and it's good to
understand these behaviors as best as possible.
Note that low-API level testing has been limited rooted devices so
far, as that's what's available as emulators. Additional fixes likely
needed for user builds of these platform versions.
Change-Id: I7a5243a0570a81b676c4adff2d8267a3b68b61ca
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
A benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/ShellBehaviorTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/ShellUtilsTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoHelper.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/ShellUtils.kt
M benchmark/integration-tests/macrobenchmark-target/src/main/java/androidx/benchmark/integration/macrobenchmark/target/NotExportedActivity.kt
cc...@google.com <cc...@google.com> #15
Removed
cc...@google.com <cc...@google.com>
ap...@google.com <ap...@google.com> #16
Branch: androidx-main
commit 872057a24acf988c6d43e8ff066ae3ce4c881768
Author: Chris Craik <ccraik@google.com>
Date: Tue Aug 03 12:17:49 2021
Fix app cmdline wildcard to only be used API 28+
Bug: 183129298
Test: ./gradlew benchmark:benchmark-macro:cC
Related improvements:
* Fixed trace processor query to use "OR" for multi-slice queries
* Expanded PerfettoCaptureTest to also test non-app tag (to enable
easily isolating app-tag tracing issues)
* Fixed crash in FileLinkingRule when run from gradle cmdline
Change-Id: Ia194d886834b7a1c1c24800cb102c6f8bab39684
M benchmark/benchmark-macro-junit4/src/main/java/androidx/benchmark/macro/junit4/PerfettoRule.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/FileLinkingRule.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoConfigTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoCapture.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoCaptureWrapper.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoConfig.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessor.kt
ap...@google.com <ap...@google.com> #17
Branch: androidx-main
commit 6a161d9d5b72a8921a87f728e69267ed476aa180
Author: Chris Craik <ccraik@google.com>
Date: Thu Aug 12 15:20:56 2021
Raise min API of benchmark-macro(-junit4) libraries to 21
Bug: 183129298
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC benchmark:benchmark-macro-junit4:cC benchmark:benchmark-junit4:cC
Relnote: "Raised min API to 21 to reflect intended lowest API level to
be supported in the future. Current min API supported continues to
be conveyed via RequiredApi(), and is currently 29"
Removes all SdkSuppress(21) and RequireApi(21) in macrobench, as
they're no longer needed. Also unifies OutputTest into
benchmark-common, as it should continue to using SdkSuppress(21).
Change-Id: I440d67683d41150fc9dd84d4f0a7100f2c9f32bd
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/OutputsTest.kt
M benchmark/benchmark-macro-junit4/build.gradle
M benchmark/benchmark-macro/api/current.txt
M benchmark/benchmark-macro/api/public_plus_experimental_current.txt
M benchmark/benchmark-macro/api/restricted_current.txt
M benchmark/benchmark-macro/build.gradle
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/CompilationModeTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkScopeTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkTest.kt
D benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/OutputsTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkScope.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessor.kt
M testutils/testutils-macrobenchmark/build.gradle
ap...@google.com <ap...@google.com> #18
Branch: androidx-main
commit 4d383e006bca3302eac0729de693ddc1c94d17ac
Author: Chris Craik <ccraik@google.com>
Date: Mon Aug 23 17:45:50 2021
Enable perfetto capture testing back to API 21/23
Bug: 183129298
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC
While API 21 and 22 mostly work, UiState will fail to append to the
trace, due to a file access permission issue. We can investigate this
in a follow-up.
The major fix was generalizing how we detect package/executable names
in ps output to handle old platform behavior.
This change also:
- removes the no-longer-necessary socket naming args
- speculatively fixes a flake discovered in PerfettoCaptureTest
(reproing on older devices), where legit tracepoint usage would
cause the test to flake
Change-Id: Ib637f9ef30bf557f5b03198e0803f5e4fa32278c
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/PerfettoHelperTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Shell.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoHelper.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
ap...@google.com <ap...@google.com> #19
Branch: androidx-main
commit addb956b048ef75eb639935ce3c5de878456162e
Author: Chris Craik <ccraik@google.com>
Date: Wed Sep 08 15:33:28 2021
Trace-based FrameTimingMetric, Step 1
Bug: 195987329
Bug: 183129298
Test: TrivialListScrollBenchmark, FrameTimingQueryTest
Test: ./gradlew benchmark:benchmark-macro:cC
Initial step of moving FrameTimingMetric to be based on Perfetto trace
data. This has several advantages:
- Improved precision - floating point ms measurements
- Metric customization - e.g. deadline-aware metrics
- Metric clarification - enables labeling different metrics per API
level, as opposed to dumpsys which changes behavior across api
levels.
Currently, new metrics are private, opt-in per benchmark, and tagged
separately from other metrics for comparison in Studio/CI.
Planned next steps:
- Merge (new) frame timing metrics to be merged across
iterations. This will significantly simply output, as we can avoid
min/med/max and rely entirely on percentiles to show distribution.
- Move FrameTimingMetric itself to be trace based, with metric
emphasis depending on current platform version.
- Make macrobench no longer throw on API < 29 (initially, for
FrameTimingMetric benchmarks only)
- Go through this for startup as well
Change-Id: Ib07e51ea2e5822ed75187731530cf1f5a40b8e66
M benchmark/benchmark-common/src/main/java/androidx/benchmark/MetricResult.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoConfig.kt
M benchmark/benchmark-macro/api/restricted_current.txt
A benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/FrameTimingQueryTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/FrameTimingQuery.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessor.kt
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/Slice.kt
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialListScrollBenchmark.kt
ap...@google.com <ap...@google.com> #20
Branch: androidx-main
commit 47fd60aaf2b811b6d7f1b4d40d2380d6c0f73fc9
Author: Chris Craik <ccraik@google.com>
Date: Wed Sep 15 16:10:00 2021
Trace-based FrameTimingMetric, Step 2
Bug: 195987329
Bug: 183129298
Fixes: 200082324
Test: TrivialListScrollBenchmark # Validated on 29 & 31
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC
Completes migration of FrameTimingMetric to be trace-based, and
surface FrameNegativeSlack when available.
This change also makes several related changes to frame metrics coming
from FrameTimingQuery:
* Deleted FrameTime, as it's always preferable to use something
deadline based
* Renamed BasicFrameTime -> FrameCpuTime for clarity, and to match metric plan
* Made FrameSlack negative, so the *worst* frames show up in 99P,
instead of best. This avoids having to differentate between low=bad
and high=bad metrics.
Adds FrameTimingGfxInfoMetric to existing scroll benchmarks for
temporary continuity in CI tracking.
Change-Id: Ied56278b1f76f37df44181dc736590753aa1cb30
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/FrameTimingQuery.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/FrameTimingQueryTest.kt
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialListScrollBenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/IdeSummaryString.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/TrivialListScrollBenchmark.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/NestedListsScrollBenchmark.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/IdeSummaryStringTest.kt
M benchmark/benchmark-macro/api/restricted_current.txt
ap...@google.com <ap...@google.com> #21
Branch: androidx-main
commit 358637d914f7c6b6cfc3d442aaa4eee1c6bd611a
Author: Chris Craik <ccraik@google.com>
Date: Wed Sep 15 17:25:13 2021
Lower min macrobenchmark to API23 (scrolling benchmarks only)
Bug: 183129298
Test: TrivialListScrollBenchmark # validated on API25
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC # API 25
Relnote: Enable scrolling macrobenchmarks to run back to API 23
Lowers Macrobenchmark @RequiresApi -> 23. StartupTimingMetric still
requires 29, which is under investigation.
Change-Id: If39c203832b9aed765aa05d7573eb9c88bf1dc70
M compose/integration-tests/macrobenchmark/build.gradle
M benchmark/benchmark-macro/api/current.txt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/SmallListStartupBenchmark.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/NestedListsScrollBenchmark.kt
M benchmark/benchmark-macro/api/public_plus_experimental_current.txt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoCaptureWrapper.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialStartupBenchmark.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/IoSettingsStartupBenchmark.kt
M benchmark/integration-tests/macrobenchmark/build.gradle
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/SmallListStartupBenchmark.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/MacrobenchmarkTest.kt
M benchmark/benchmark-macro-junit4/src/main/java/androidx/benchmark/macro/junit4/MacrobenchmarkRule.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/TrivialStartupBenchmark.kt
M benchmark/benchmark-macro-junit4/api/current.txt
M benchmark/benchmark-macro-junit4/api/restricted_current.txt
M benchmark/benchmark-macro-junit4/api/public_plus_experimental_current.txt
M benchmark/benchmark-macro/api/restricted_current.txt
ap...@google.com <ap...@google.com> #22
Branch: androidx-main
commit c155f6c5c18095c77e02f25e2c35179d225d2cf1
Author: Chris Craik <ccraik@google.com>
Date: Mon Oct 04 09:53:10 2021
Initial impl of StartupTimingQuery, part of new startup metric
Bug: 183129298
Bug: 190219056
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC
Tested only on API 31 so far, as there are issues with capturing
traces (with app trace tags enabled) on older platform versions.
Adds check for atrace args to be too long on older platform versions,
which was causing problems when attempting to test on API 23.
Known compatibility issues for lowering min API (e.g. 'launching'
missing for hot launches on certain old versions) will be handled in
follow up work.
Change-Id: I2eb4700f2c7cbc822ef5a23da7b97192da0ee177
M benchmark/integration-tests/macrobenchmark-target/build.gradle
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
A benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialStartupFullyDrawnBenchmark.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoConfig.kt
A benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/StartupTimingQueryTest.kt
M benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/PerfettoRule.kt
A benchmark/integration-tests/macrobenchmark-target/src/main/java/androidx/benchmark/integration/macrobenchmark/target/TrivialStartupFullyDrawnActivity.kt
M benchmark/integration-tests/macrobenchmark-target/src/main/res/values/donottranslate-strings.xml
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/integration-tests/macrobenchmark-target/src/main/AndroidManifest.xml
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupTimingQuery.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/perfetto/PerfettoConfigTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Shell.kt
ap...@google.com <ap...@google.com> #23
Branch: androidx-main
commit b5b2d530418de72c6b8f7420c86e78e6efb19c4f
Author: Chris Craik <ccraik@google.com>
Date: Mon Oct 11 23:02:44 2021
Lower min api tested for StartupTimingQuery to API 24
Bug: 183129298
Bug: 190219056
Test: StartupTimingQueryTest
Relnote: N/A
Adds support for hot launches pre-api 29, which weren't traced. We do
this with a very simple approximation for now, measuring from
activityresume to end of the next DrawFrame.
Additionally, adds tracing wrapper with fallback reflection for the
single startup-critical trace section, as it factors into
StartupTimingQuery.
This accounts for the only known platform trace section content
differences going back to intended min support level of 21.
This does not solve other compatibility issues on these old devices,
such as missing trace content, which needs to be investigated
separately. For creating the sample traces, `Thread.sleep()` and
retrying when missing content were both used.
Change-Id: Ief6dd7eff6ab90607df1088adf456c89921a55bb
M benchmark/benchmark-macro/api/current.txt
M benchmark/benchmark-macro/api/restricted_1.1.0-beta01.txt
A benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/ForceTracing.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoConfig.kt
M benchmark/benchmark-macro/api/public_plus_experimental_1.1.0-beta01.txt
M benchmark/benchmark-macro/api/1.1.0-beta01.txt
M benchmark/benchmark-macro/api/public_plus_experimental_current.txt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/StartupTimingQueryTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/MacrobenchmarkScope.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupTimingQuery.kt
M benchmark/benchmark-macro/api/restricted_current.txt
ap...@google.com <ap...@google.com> #24
Branch: androidx-main
commit 14225e715dfe78937377f70d2dc786d69a70f35c
Author: Chris Craik <ccraik@google.com>
Date: Wed Oct 13 17:34:52 2021
Switch StartupTimingMetric to custom trace-based implementation
Bug: 183129298
Bug: 190219056
Test: TrivialStartupBenchmark # macrobench
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC
Relnote: """Backport StartupTimingMetric to work back to API 23. This
new implementation also better handles reportFullyDrawn() to wait
until corresponding content has been rendered."""
Switch StartupTimingMetric to custom trace metric. This new version:
- Is validated against the old in a few specific cases on newer devices
- Supports old API levels (where trace section names were different, and sometimes not present)
- Fixes reportFullyDrawn to not define the fully drawn point
immediately, and instead wait until rendering completes.
Change-Id: If3ac9a339bedbb61785c80ea78c0e7ae8f9018e9
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/benchmark-macro/api/current.txt
M benchmark/benchmark-macro/api/restricted_1.1.0-beta01.txt
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialStartupFullyDrawnBenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/ForceTracing.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/SmallListStartupBenchmark.kt
M testutils/testutils-macrobenchmark/src/main/java/androidx/testutils/MacrobenchUtils.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/perfetto/UiStateTest.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/NestedListsScrollBenchmark.kt
M benchmark/benchmark-macro/api/public_plus_experimental_1.1.0-beta01.txt
M benchmark/benchmark-macro/api/1.1.0-beta01.txt
M benchmark/benchmark-macro/api/public_plus_experimental_current.txt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/StartupTimingQueryTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Metric.kt
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/TrivialStartupBenchmark.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/IoSettingsStartupBenchmark.kt
M benchmark/integration-tests/macrobenchmark/src/androidTest/java/androidx/benchmark/integration/macrobenchmark/SmallListStartupBenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
M compose/integration-tests/macrobenchmark/src/androidTest/java/androidx/compose/integration/macrobenchmark/TrivialStartupBenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/IterationResult.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/StartupTimingQuery.kt
M benchmark/benchmark-macro/api/restricted_current.txt
cc...@google.com <cc...@google.com> #25
Fixed with
As API 21 doesn't have tracing of Choreographer#doFrame, we've decided to only support API 23 as a minimum for now. If there's need for it, we can file a new bug and investigate our options down the line.
Description
Currently min API 30 due to using perfetto trace_processor_shell binary from Q platform build.
Switch to unbundled perfetto (at least on old devices) and trace_processor_shell built from github repo. Currently this is blocked on documentation for binary stripping ( b/182513943 ). Min API for these is API 21, and planned to be maintained (see discussion in perfetto-dev group: https://groups.google.com/a/google.com/g/perfetto-dev/c/vRHKg4Cjpok )