Assigned
Status Update
No update yet.
Comments
lp...@google.com
Project: platform/frameworks/support
Branch: androidx-master-dev
commit ec33fd30f60c310dc47ffeb481c79a50c16232f9
Author: Chris Craik <ccraik@google.com>
Date: Fri Apr 10 12:33:03 2020
Fix startupBenchmark in presubmit
Fixes: 153742334
Test: ./gradlew benchmark:integ-tests:startup-benchmark:cC # fails on device without locked clocks
Test: ./gradlew benchmark:integ-tests:startup-benchmark:cC -i -Pandroid.testInstrumentationRunnerArguments.notAnnotation=android.support.test.filters.FlakyTest,androidx.test.filters.FlakyTest # passes when we pass arg that presubmit passes
Skip StartupBenchmark when running in presubmit configuration by using
@FlakyTest.
Unfortunately, this seems to be the only easy way to skip the test,
without library-side complications. We can't early return from within
the test body, since then the Benchmark library complains that we
didn't include a benchmark block in our test.
The empty test may not be necessary for running in CI, but it does
allow the notAnnotation=FlakyTest gradle command above complete
successfully.
Change-Id: I143454e06f0458adbe71560d78e2ac11cfdc0bba
M benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/ArgumentInjectingApplication.kt
A benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/EmptyTest.kt
M benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/StartupBenchmark.kt
https://android-review.googlesource.com/1284164
Branch: androidx-master-dev
commit ec33fd30f60c310dc47ffeb481c79a50c16232f9
Author: Chris Craik <ccraik@google.com>
Date: Fri Apr 10 12:33:03 2020
Fix startupBenchmark in presubmit
Fixes: 153742334
Test: ./gradlew benchmark:integ-tests:startup-benchmark:cC # fails on device without locked clocks
Test: ./gradlew benchmark:integ-tests:startup-benchmark:cC -i -Pandroid.testInstrumentationRunnerArguments.notAnnotation=android.support.test.filters.FlakyTest,androidx.test.filters.FlakyTest # passes when we pass arg that presubmit passes
Skip StartupBenchmark when running in presubmit configuration by using
@FlakyTest.
Unfortunately, this seems to be the only easy way to skip the test,
without library-side complications. We can't early return from within
the test body, since then the Benchmark library complains that we
didn't include a benchmark block in our test.
The empty test may not be necessary for running in CI, but it does
allow the notAnnotation=FlakyTest gradle command above complete
successfully.
Change-Id: I143454e06f0458adbe71560d78e2ac11cfdc0bba
M benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/ArgumentInjectingApplication.kt
A benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/EmptyTest.kt
M benchmark/integration-tests/startup-benchmark/src/androidTest/java/androidx/benchmark/integration/startup/benchmark/StartupBenchmark.kt
cc...@google.com
Something about this test isn't deterministic, want to investigate deeper.
Before the regression, we see roughly 120 frames, with roughly 470 calls of TextLayout:initLayout in those frames.
After the regression, we still see 120 frames, but roughly 1300 calls of TextLayout:initLayout.
Seems like this test isn't deterministic in the amount of work in each frame.
os...@google.com
Hmm, running the test locally, there seems to be some runs where it doesn't execute properly. It launches, but no buttons are activated.
Description
Perf Regression (High) found, matching 17 tracked metrics from benchmarks.
To triage this regression, see the guide at
Test classes affected:
Test methods affected:
Devices affected:
API Level: