Status Update
Comments
ap...@google.com <ap...@google.com> #2
Branch: androidx-master-dev
commit c60f33e229e31ab328ef6b59dab63b264954831c
Author: Alexandre Elias <aelias@google.com>
Date: Fri Jul 10 16:23:09 2020
Semantics no-op cleanups
Partly in response to lmr's broad code review, I did a pass of
superficial API/implementation cleanup. The main changes are:
- I changed each Boolean SemanticsProperty where false is equivalent to
not being present to take "Unit" instead. This is conceptually
clearer: it avoids questions like "can I cancel out a semantics from a
merged child by setting it to false?" Because "property = Unit" looks
weird, I also changed the style of these to "property()".
- I moved the Semantics id generator closer to where it's used, in
SemanticsModifierCore. I made it internal and an AtomicInt.
(Note that integer ids are heavily used in the Android
AccessibilityNodeInfo APIs so I can't simply remove them entirely.)
- I deleted dead code. Some examples include SemanticsHintOverrides,
a public API not connected to anything, and SemanticsPropertyKey
merge() open method which is never called. (In both cases I have
a different plan in mind for accessibility.)
Fixes: 145951226
Fixes: 145955412
Test: existing tests
Relnote: "Single-value semantics properties now use a calling style.
For example, 'semantics { hidden = true }' is now written as:
'semantics { hidden() }'."
Change-Id: Ic1afd12ea22c926babc9662f1804d80b33aa0cfc
M ui/integration-tests/benchmark/src/androidTest/java/androidx/ui/benchmark/test/LayoutNodeModifierBenchmark.kt
M ui/ui-core/api/0.1.0-dev15.txt
M ui/ui-core/api/current.txt
M ui/ui-core/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-core/api/public_plus_experimental_current.txt
M ui/ui-core/api/restricted_0.1.0-dev15.txt
M ui/ui-core/api/restricted_current.txt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/graphics/vector/VectorTest.kt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/semantics/SemanticsTests.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidActuals.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeView.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeViewAccessibilityDelegateCompat.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidPopup.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/Expect.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsConfiguration.kt
D ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsHintOverrides.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsModifier.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsNode.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsOwner.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsWrapper.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/semantics/SemanticsProperties.kt
M ui/ui-foundation/api/0.1.0-dev15.txt
M ui/ui-foundation/api/current.txt
M ui/ui-foundation/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-foundation/api/public_plus_experimental_current.txt
M ui/ui-foundation/api/restricted_0.1.0-dev15.txt
M ui/ui-foundation/api/restricted_current.txt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Clickable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Dialog.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Scroller.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Selectable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Toggleable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/semantics/FoundationSemanticsProperties.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ButtonTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CardTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CheckboxScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/RadioButtonScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ScaffoldTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SnackbarTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SurfaceTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ripple/RippleIndicationTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/textfield/TextFieldScreenshotTest.kt
M ui/ui-material/src/main/java/androidx/ui/material/AppBar.kt
M ui/ui-material/src/main/java/androidx/ui/material/TextFieldImpl.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/AssertsTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/CallSemanticsActionTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ErrorMessagesTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/FindersTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/PrintToStringTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ScrollToTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/TextActionsTest.kt
M ui/ui-test/src/main/java/androidx/ui/test/Actions.kt
M ui/ui-test/src/main/java/androidx/ui/test/Filters.kt
M ui/ui-text/api/0.1.0-dev15.txt
M ui/ui-text/api/current.txt
M ui/ui-text/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-text/api/public_plus_experimental_current.txt
M ui/ui-text/api/restricted_0.1.0-dev15.txt
M ui/ui-text/api/restricted_current.txt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/CoreTextField.kt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/TextSemanticsProperties.kt
sj...@google.com <sj...@google.com>
cc...@google.com <cc...@google.com> #3
Other tests in our infra write to external storage, but that's difficult to configure for benchmark. We expect to be able to write to app-scoped external storage, since that's what the platform wants tests to do, and it's how AGP will typically do this (via additionalTestOutputDir
arg, as built for benchmark json file-style copying).
We can ask benchmarks to write to external storage, by passing -e additionalTestOutputDir /sdcard/androidx_test_output
...
However, on recent versions of the platform, this requires opting out of isolation with the application tag: android:requestLegacyExternalStorage="true"
.
... and on R, this appears to stop working. Even with that opt out, and no-isolated-storage 1
, trying to create a file in /sdcard/androidx_test_output
doesn't work from the app process. My guess is we need to copy it with shell permissions, since shell does have access there.
We should revisit this after, to see if file pulling can support the download path, but for now we can just try and hack benchmark file writing to make this work. Error that Sam reported from his test run below, when trying to use default write path:
10-21 10:05:49 E/ContentProviderHandler: Failed to pull a file at '/storage/emulated/0/Download/androidx.benchmark.benchmark.test-benchmarkData.json' to /tmp/androidx.benchmark.benchmark.test-benchmarkData_11917953806432576087.json using content provider. Error: 'Error while accessing provider:android.tradefed.contentprovider
java.io.FileNotFoundException: open failed: ENOENT (No such file or directory)
at android.database.DatabaseUtils.readExceptionWithFileNotFoundExceptionFromParcel(DatabaseUtils.java:144)
at android.content.ContentProviderProxy.openFile(ContentProviderNative.java:598)
at com.android.commands.content.Content$ReadCommand.onExecute(Content.java:587)
at com.android.commands.content.Content$Command.execute(Content.java:470)
at com.android.commands.content.Content.main(Content.java:690)
at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:340)
ap...@google.com <ap...@google.com> #4
Branch: androidx-master-dev
commit e58f06e6ffd098e73137fd1ddda56e6c86def2c8
Author: Chris Craik <ccraik@google.com>
Date: Wed Oct 28 11:22:45 2020
Report json file as run metric, instead of test metric
Test: adb shell am instrument -w -r -e debug false -e newRunListenerMode true -e class 'androidx.benchmark.benchmark.TrivialJavaBenchmark' -e androidx.benchmark.output.enable true -e listener androidx.benchmark.junit4.InstrumentationResultsRunListener androidx.benchmark.benchmark.test/androidx.benchmark.junit4.AndroidBenchmarkRunner
Relnote: N/A
Bug: 145598917
As the same json file is written/appended by each test, it's incorrect
to request it to be copied multiple times. Now that file specifically
is reported as a run metric, at end of run only. Note that this
requires passing a run listener (see Test: line above), when using
this in internal CI.
Sample output below (note "RESULT" vs what it used to be: "STATUS")
INSTRUMENTATION_RESULT: additionalTestOutputFile_results_json=/storage/emulated/0/Download/androidx.benchmark.benchmark.test-benchmarkData.json
Change-Id: Ife62cc7dae3ec298a76f1a1c465f2abbaec85ea7
M benchmark/common/api/public_plus_experimental_current.txt
M benchmark/common/src/main/java/androidx/benchmark/InstrumentationResults.kt
M benchmark/common/src/main/java/androidx/benchmark/ResultWriter.kt
A benchmark/junit4/src/main/java/androidx/benchmark/junit4/InstrumentationResultsRunListener.kt
ap...@google.com <ap...@google.com> #5
Branch: androidx-master-dev
commit 5d2b205ca996b1cf37a28daa342ffbffb481b887
Author: Chris Craik <ccraik@google.com>
Date: Mon Nov 02 13:27:41 2020
Add benchmark for measuring overhead of Perfetto trace capture
Bug: 171749803
Bug: 145598917
Test: PerfettoOverheadBenchmark
Will also help us validate PerfettoRule capture in CI.
Sample benchmark measurements (Pixel2 locked clocks):
11 ns 0 allocs PerfettoOverheadBenchmark.empty
287 ns 0 allocs PerfettoOverheadBenchmark.runWithTimingDisabled
4,958 ns 0 allocs PerfettoOverheadBenchmark.traceBlock
4,832 ns 0 allocs PerfettoOverheadBenchmark.traceBeginEnd
Change-Id: I1c74fb14f3a13ba10d7ea6f9ff37f450e69aa54e
M benchmark/benchmark/build.gradle
A benchmark/benchmark/src/androidTest/java/androidx/benchmark/benchmark/PerfettoOverheadBenchmark.kt
cc...@google.com <cc...@google.com> #6
While the CL in #5 was reverted, and is in the process of being
Looks like API 29 is having issues writing to output directory though, which started before that change landed.
cc...@google.com <cc...@google.com> #7
Looks like it was caused by enabling benchmark output:
IIRC, we only tested that with forrest runs on API 28. Fix
ap...@google.com <ap...@google.com> #8
Branch: androidx-master-dev
commit 4724b1f55919968f2a486da5f8f8b064927911cf
Author: Chris Craik <ccraik@google.com>
Date: Tue Nov 03 14:45:09 2020
Add requestLegacyExternalStorage to benchmarks
Bug: 145598917
Test: ./gradlew benchmark:b-b:installRelease && adb shell am instrument -w -r -e debug false -e newRunListenerMode true -e class 'androidx.benchmark.benchmark.TrivialJavaBenchmark' -e androidx.benchmark.output.enable true -e listener androidx.benchmark.junit4.InstrumentationResultsRunListener androidx.benchmark.benchmark.test/androidx.benchmark.junit4.AndroidBenchmarkRunner
Turning on benchmark output has caused benchmarks to start crashing on
API 29, due to inability to access output dir.
Note: unable to reproduce this crash on my local Q device, even with
manual am instrument command above
Change-Id: Ib19ec37ec10046b493c8b44207aab0ce93ad24b6
M ads/ads-identifier-benchmark/src/androidTest/AndroidManifest.xml
M appcompat/appcompat-benchmark/src/androidTest/AndroidManifest.xml
M benchmark/benchmark/src/androidTest/AndroidManifest.xml
M collection/collection-benchmark/src/androidTest/AndroidManifest.xml
M navigation/benchmark/src/androidTest/AndroidManifest.xml
M recyclerview/recyclerview-benchmark/src/androidTest/AndroidManifest.xml
M room/benchmark/src/androidTest/AndroidManifest.xml
M slices/benchmark/src/androidTest/AndroidManifest.xml
M work/workmanager-benchmark/src/androidTest/AndroidManifest.xml
cc...@google.com <cc...@google.com> #9
We've got initial traces in CI! For example,
Current Constraints:
- only captured on API 29: Pixel 2016 (can support 28, with some
)additional work - only small subset of benchmarks capture a trace, due to overhead (
)b/171749803
Next step is to capture traces for all benchmarks with app tag disabled. This will mean no e.g. compose trace sections will be captured, but it will allow us to see CPU usage, thread scheduling, GCs, etc. We'll keep the app tag enabled on the *WithTracing
variants.
ap...@google.com <ap...@google.com> #10
Branch: androidx-main
commit f5d35eb9ca852dfac791c2dc1d6ce953b2843880
Author: Chris Craik <ccraik@google.com>
Date: Wed Aug 11 17:35:21 2021
Move perfetto capture and protos to benchmark-common
Test: ./gradlew benchmark:benchmark-common:cC benchmark:benchmark-macro:cC benchmark:benchmark-macro-junit4:cC benchmark:benchmark-junit4:cC
Bug: 196115387
Bug: 145598917
Relnote: N/A
PerfettoTraceProcessor and its binary will stay in benchmark-macro.
Moves PerfettoRule to benchmark-junit, so that microbenchmarks don't
need to depend on benchmark-macro-junit4. Eventually, this
functionality can be merged into BenchmarkRule, as long as
in-process/app tag tracing is configurable, and off by
default. UserspaceTracing will enable us to still capture
BenchmarkState lifecycle events, like different modes (warmup /
measure / allocation count).
This unblocks raising the minAPI of Macrobench to the long-term goal
of API 21, which is the lowest that can support shell commands, as
well as perfetto capture.
This change also moves a few supporting classes, such as DeviceInfo
and ConfigurationError, which were always intended to gradually
replace benchmark-common's error / device state querying. This starts
that process.
As testutils-ktx is used by benchmark-common now, add to all project
config flavors.
Change-Id: I6559b6735d0106e38ec5d8eaeddcac9297d0c431
M benchmark/benchmark-common/api/current.txt
M benchmark/benchmark-common/api/public_plus_experimental_current.txt
M benchmark/benchmark-common/api/restricted_current.txt
M benchmark/benchmark-common/build.gradle
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/ConfigurationErrorTest.kt
A benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/Packages.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/ShellBehaviorTest.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/ShellTest.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/UserspaceTracingTest.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/perfetto/AtraceTagTest.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/perfetto/PerfettoConfigTest.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/perfetto/UiStateTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/ConfigurationError.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/DeviceInfo.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/UserspaceTracing.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/AtraceTag.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoCapture.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoCaptureWrapper.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoConfig.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoHelper.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/UiState.kt
M benchmark/benchmark-common/src/main/java/perfetto/protos/package-info.java
M benchmark/benchmark-common/src/main/proto/perfetto_config.proto
M benchmark/benchmark-common/src/main/proto/perfetto_trace.proto
M benchmark/benchmark-junit4/api/current.txt
M benchmark/benchmark-junit4/api/public_plus_experimental_current.txt
M benchmark/benchmark-junit4/api/restricted_current.txt
M benchmark/benchmark-junit4/build.gradle
M benchmark/benchmark-junit4/src/androidTest/java/androidx/benchmark/junit4/PerfettoRuleTest.kt
M benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/PerfettoRule.kt
M benchmark/benchmark-macro-junit4/api/current.txt
M benchmark/benchmark-macro-junit4/api/public_plus_experimental_current.txt
M benchmark/benchmark-macro-junit4/api/restricted_current.txt
M benchmark/benchmark-macro/api/current.txt
M benchmark/benchmark-macro/api/public_plus_experimental_current.txt
M benchmark/benchmark-macro/api/restricted_current.txt
M benchmark/benchmark-macro/build.gradle
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/FileLinkingRule.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoCaptureTest.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessorTest.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/CompilationMode.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/Macrobenchmark.kt
M benchmark/benchmark-macro/src/main/java/androidx/benchmark/macro/perfetto/PerfettoTraceProcessor.kt
M benchmark/benchmark/build.gradle
M benchmark/benchmark/src/androidTest/java/androidx/benchmark/benchmark/PerfettoOverheadBenchmark.kt
M buildSrc/private/src/main/kotlin/androidx/build/uptodatedness/TaskUpToDateValidator.kt
M compose/ui/ui-graphics/benchmark/build.gradle
M compose/ui/ui-graphics/benchmark/src/androidTest/java/androidx/compose/ui/graphics/benchmark/VectorBenchmarkWithTracing.kt
M development/build_log_simplifier/messages.ignore
M settings.gradle
cc...@google.com <cc...@google.com> #11
I see the next step here being to always enable tracing, but to disable the app tag. Just need to use something (like ForceTracing, which just landed in macrobench) to ensure high level lifecycle events are captured.
ap...@google.com <ap...@google.com> #12
Branch: androidx-main
commit ec8b0e22b5e2415f1c165d5ff1b9e73fff4b956b
Author: Chris Craik <ccraik@google.com>
Date: Fri Nov 19 14:52:10 2021
Capture Perfetto traces during Microbenchmarks
Bug: 205636583
Fixes: 145598917
Test: ./gradlew benchmark-common
Relnote: "Add low-overhead System Tracing to microbench output on
Android Q (API 29+). Note that this does not currently capture custom
tracing (via android.os.Trace or androidx.tracing Jetpack APIs) to
avoid affecting results. This tracing should be useful in diagnosing
instability, especially from sources outside the benchmark."
Uses same timing information as microbench itself to generate
userspace tracing information, which is appended into a captured
perfetto trace.
Eventually this approach could enable other metrics available in
traces (such as time a thread is running vs not).
This approach has multiple drawbacks/constraints currently:
* Ignores calls to Trace.beginSection/endSection
* These have been found to be too high overhead in common
scenarios to be on by default. Turning it on drastically affected
many compose benchmarks.
* Eventually we can either make app tracing tag configurable,
offer microbenchmark-specific userspace tracing (easier than
general case, since we can append manually), or offer a more
general userspace tracing solution (which would also be much
lower overhead).
* Puts microbench library tracing in separate track, separate from
the app process
* This can be fixed with improvements to the proto generation to
pick the correct track id, given the test TID .
* Only traces with bundled perfetto (API 29+)
* If deemed useful enough, we can use unbundled perfetto as well
(move perfetto tracebox from benchmark-macro to benchmark-common)
Change-Id: I298be3c9f9f7ef9ba1d2c63d79c864e8c8b16e04
M benchmark/benchmark-junit4/api/current.txt
M benchmark/benchmark-junit4/api/restricted_current.txt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/UserspaceTracing.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/MetricCaptureTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/InstrumentationResults.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/MetricsContainer.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoCaptureWrapper.kt
M benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/PerfettoRule.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/BenchmarkState.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/InstrumentationResultsTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/MetricCapture.kt
M benchmark/benchmark-junit4/src/main/java/androidx/benchmark/junit4/BenchmarkRule.kt
M benchmark/benchmark-macro/src/androidTest/java/androidx/benchmark/macro/StartupTimingMetricTest.kt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/perfetto/PerfettoCapture.kt
M benchmark/benchmark-junit4/api/public_plus_experimental_current.txt
M benchmark/benchmark-common/src/main/java/androidx/benchmark/Profiler.kt
M benchmark/benchmark-common/src/androidTest/java/androidx/benchmark/BenchmarkStateTest.kt
Description
one-pager: