Status Update
No update yet.
Comments
as...@google.com
Branch: androidx-master-dev
commit ec33fd30f60c310dc47ffeb481c79a50c16232f9
Author: Chris Craik <ccraik@google.com>
Date: Fri Apr 10 12:33:03 2020
Fix startupBenchmark in presubmit
Fixes: 153742334
Test: ./gradlew benchmark:integ-tests:startup-benchmark:cC # fails on device without locked clocks
Test: ./gradlew benchmark:integ-tests:startup-benchmark:cC -i -Pandroid.testInstrumentationRunnerArguments.notAnnotation=android.support.test.filters.FlakyTest,androidx.test.filters.FlakyTest # passes when we pass arg that presubmit passes
Skip StartupBenchmark when running in presubmit configuration by using
@FlakyTest.
Unfortunately, this seems to be the only easy way to skip the test,
without library-side complications. We can't early return from within
the test body, since then the Benchmark library complains that we
didn't include a benchmark block in our test.
The empty test may not be necessary for running in CI, but it does
allow the notAnnotation=FlakyTest gradle command above complete
successfully.
Change-Id: I143454e06f0458adbe71560d78e2ac11cfdc0bba
M benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/ArgumentInjectingApplication.kt
A benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/EmptyTest.kt
M benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/StartupBenchmark.kt
nj...@google.com
Thanks for flagging Andrei! The referenced
Is it possible that this is related to Chuck's CL here?
Chuck could you take a look?
ch...@google.com
This is unlikely to be the pausable composition change as the feature is dormant if unused. There is one place that affects all composition (not just pausable) composition but I am unable to look at this week (as I don't have my performance testing phones with me). I will see if someone else can look at it this week.
ch...@google.com
This looks like the regression is from pausable composition. I am reverting the change while I investigate why this had any affect at all.
ch...@google.com
It turns out that both Andrew and I cannot reproduce this locally. What we thought was a repro of the regression was a misconfigured test. As this is not 3211838, I am handing this back to Nadar.
nj...@google.com
Chris can you take a look to see if this is potential noise/cross talk in the skia benchmarks? as per
cc...@google.com
NOTE: This bug is affected by bad build IDs that we've seen recently, see
You can get the actual flagged CL list by:
- click the
Graph on Dashboardlink above - click on the first regression datapoint
- clicking
Commits At Step (1028821 - 1029380)
The link that this produces does include Chuck's CL:
It turns out that both Andrew and I cannot reproduce this locally.
Chuck, just to confirm, you tried to revert 3211838 locally, and that had no effect?
cc...@google.com
Loading
It's still possible this is layout/crosstalk however, just wanted to double check I understand the unable to reproduce locally part above.
ch...@google.com
No. Andrew (added as a CC) could not reproduce the regression locally (I am remote from my test devices at the moment.) We reviewed the CL for changes that might cause this as well as tried to reproduce the regeression. Andrew was unable to reproduce the regression with a correctly configured device (he initially thought he reproduced it but it turns out the device was not configured correctly).
Description
Perf Regression (High) found, matching 27 tracked metrics from benchmarks.
To triage this regression, see the guide at
Tests affected:
Devices affected:
API Level: