Verified
Status Update
Comments
cc...@google.com <cc...@google.com> #2
Project: platform/frameworks/support
Branch: androidx-master-dev
commit b90079595f33f58fece04026a97faa0d243acdb1
Author: Yuichi Araki <yaraki@google.com>
Date: Wed Sep 18 16:55:49 2019
Change the way to detect mismatch between POJO and query
This fixes cursor mismatch warnings with expandProjection.
Bug: 140759491
Test: QueryMethodProcessorTest
Change-Id: I7659002e5e0d1ef60fc1af2a625c0c36da0664d8
M room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
M room/compiler/src/test/kotlin/androidx/room/processor/QueryMethodProcessorTest.kt
M room/compiler/src/test/kotlin/androidx/room/testing/TestProcessor.kt
https://android-review.googlesource.com/1123258
https://goto.google.com/android-sha1/b90079595f33f58fece04026a97faa0d243acdb1
Branch: androidx-master-dev
commit b90079595f33f58fece04026a97faa0d243acdb1
Author: Yuichi Araki <yaraki@google.com>
Date: Wed Sep 18 16:55:49 2019
Change the way to detect mismatch between POJO and query
This fixes cursor mismatch warnings with expandProjection.
Bug: 140759491
Test: QueryMethodProcessorTest
Change-Id: I7659002e5e0d1ef60fc1af2a625c0c36da0664d8
M room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
M room/compiler/src/test/kotlin/androidx/room/processor/QueryMethodProcessorTest.kt
M room/compiler/src/test/kotlin/androidx/room/testing/TestProcessor.kt
ap...@google.com <ap...@google.com> #3
cc...@google.com <cc...@google.com> #4
Project: platform/frameworks/support
Branch: androidx-master-dev
commit bdde5a1a970ddc9007b28de4aa29d60ffa588f08
Author: Yigit Boyar <yboyar@google.com>
Date: Thu Apr 16 16:47:05 2020
Re-factor how errors are dismissed when query is re-written
This CL changes how we handle errors/warnings if query is
re-written.
There was a bug in expandProjection where we would report warnings
for things that Room already fixes automatically ( b/140759491 ).
The solution to that problem (I7659002e5e0d1ef60fc1af2a625c0c36da0664d8)
solved it by deferring validating of columns until after re-write
decision is made. Unfortunately, this required changing PojoRowAdapter
to have a dummy mapping until it is validating, make it hard to use
as it does have a non-null mapping which is not useful.
This CL partially reverts that change and instead rely on the log
deferring logic we have in Context. This way, we don't need to break
the stability of PojoRowAdapter while still having the ability to
drop warnings that room fixes. This will also play nicer when we
have different query re-writing options that can use more information
about the query results.
Bug: 153387066
Bug: 140759491
Test: existing tests pass
Change-Id: I2ec967c763d33d7a3ff02c1a13c6953b460d1e5f
M room/compiler/src/main/kotlin/androidx/room/log/RLog.kt
M room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
https://android-review.googlesource.com/1288456
Branch: androidx-master-dev
commit bdde5a1a970ddc9007b28de4aa29d60ffa588f08
Author: Yigit Boyar <yboyar@google.com>
Date: Thu Apr 16 16:47:05 2020
Re-factor how errors are dismissed when query is re-written
This CL changes how we handle errors/warnings if query is
re-written.
There was a bug in expandProjection where we would report warnings
for things that Room already fixes automatically (
The solution to that problem (I7659002e5e0d1ef60fc1af2a625c0c36da0664d8)
solved it by deferring validating of columns until after re-write
decision is made. Unfortunately, this required changing PojoRowAdapter
to have a dummy mapping until it is validating, make it hard to use
as it does have a non-null mapping which is not useful.
This CL partially reverts that change and instead rely on the log
deferring logic we have in Context. This way, we don't need to break
the stability of PojoRowAdapter while still having the ability to
drop warnings that room fixes. This will also play nicer when we
have different query re-writing options that can use more information
about the query results.
Bug: 153387066
Bug: 140759491
Test: existing tests pass
Change-Id: I2ec967c763d33d7a3ff02c1a13c6953b460d1e5f
M room/compiler/src/main/kotlin/androidx/room/log/RLog.kt
M room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
cc...@google.com <cc...@google.com> #5
Done a lot more digging on this, retitled to clarify the pattern here. For modules where the first few benchmarks run quickly (each finish in < 500ms), we have a lot of noise - this is true both for recyclerview:recyclerview-benchmark, but also benchmark:benchmark-benchmark (the trivial and proof-of-concept benchmarks owned by the bench library itself).
The core problem is that we're not actually letting JIT finish during warmup. In recent versions of ART, the jit thread's priority is 129 (numbers are 'niceness', so lower is higher priority), default thread priority is 120.
Our single threaded benchmarks are at default priority, and fully saturate one of the two big, enabled cores (since other cores are disabled). If there's any other background work at the same time that's default priority, it's possible for JIT to get starved and not make significant forward progress on jitting the benchmark itself. This is especially bad if JIT happens to be working on benchmark-irrelevant work (e.g. JUnit or other code outside the loop). Warmup finishes quickly, because it looks like performance is stable.
Working to land a fix that will bump thread priority of both the benchmark, and the jit thread (though not as high as the benchmark). Will keep an eye on CI stability as this lands, and will share the broader stability impact of this fix.
The core problem is that we're not actually letting JIT finish during warmup. In recent versions of ART, the jit thread's priority is 129 (numbers are 'niceness', so lower is higher priority), default thread priority is 120.
Our single threaded benchmarks are at default priority, and fully saturate one of the two big, enabled cores (since other cores are disabled). If there's any other background work at the same time that's default priority, it's possible for JIT to get starved and not make significant forward progress on jitting the benchmark itself. This is especially bad if JIT happens to be working on benchmark-irrelevant work (e.g. JUnit or other code outside the loop). Warmup finishes quickly, because it looks like performance is stable.
Working to land a fix that will bump thread priority of both the benchmark, and the jit thread (though not as high as the benchmark). Will keep an eye on CI stability as this lands, and will share the broader stability impact of this fix.
cc...@google.com <cc...@google.com> #6
Screenshot above shows synchronizedIncrementBenchmark finishing warmup *before* JIT of the relevant benchmark code is done. Result numbers started at e.g. ~1000ns, dropping to ~200ns part way through the resultset.
ca...@google.com <ca...@google.com> #7
+ngeoffray as an FYI
ap...@google.com <ap...@google.com> #8
Project: platform/frameworks/support
Branch: androidx-master-dev
commit 345f86d34f7e373f464c0d8185e392b067a2de4a
Author: Chris Craik <ccraik@google.com>
Date: Wed Oct 09 17:37:17 2019
Bump thread priority of benchmarks and JIT during benchmarks
The JIT thread is so low priority that other parallel tasks can starve
it, especially for the first few benchmarks when a process runs.
The system can spin up significant background work right after install
and/or instrumentation start, and on locked devices with only two big
cores, there aren't enough CPUs to go around - warmup and benchmark
both complete before relevant JIT is complete.
Now, we bump the priority of both the benchmark and JIT thread.
Tracing benchmarks show that the JIT thread goes much faster, which
should significantly reduce the chance we capture results on unjitted
code.
This may also motivate us to use CPU affinity + locked small cores in
the future, we can keep monitoring.
Test: ./gradlew benchmark:b-c:cC
Test: ./gradlew benchmark:b-b:cC
Test: ./gradlew recyclerview:r-b:cC
This CL also adds more logging, and unifies all logging under
"benchmark" tag. This logging was very useful in discovering and
diagnosing the priority problem, since it showed the edge cases where
jit finished *during* the measure pass.
Bug: 140773023
Bug: 142058671
Change-Id: If542e3cb8867165cf7b4688090ee534e68a23562
M benchmark/common/src/androidTest/java/androidx/benchmark/BenchmarkStateTest.kt
M benchmark/common/src/main/java/androidx/benchmark/BenchmarkState.kt
A benchmark/common/src/main/java/androidx/benchmark/ThreadPriority.kt
M benchmark/common/src/main/java/androidx/benchmark/WarmupManager.kt
M benchmark/junit4/src/main/java/androidx/benchmark/junit4/BenchmarkRule.kt
https://android-review.googlesource.com/1138018
https://goto.google.com/android-sha1/345f86d34f7e373f464c0d8185e392b067a2de4a
Branch: androidx-master-dev
commit 345f86d34f7e373f464c0d8185e392b067a2de4a
Author: Chris Craik <ccraik@google.com>
Date: Wed Oct 09 17:37:17 2019
Bump thread priority of benchmarks and JIT during benchmarks
The JIT thread is so low priority that other parallel tasks can starve
it, especially for the first few benchmarks when a process runs.
The system can spin up significant background work right after install
and/or instrumentation start, and on locked devices with only two big
cores, there aren't enough CPUs to go around - warmup and benchmark
both complete before relevant JIT is complete.
Now, we bump the priority of both the benchmark and JIT thread.
Tracing benchmarks show that the JIT thread goes much faster, which
should significantly reduce the chance we capture results on unjitted
code.
This may also motivate us to use CPU affinity + locked small cores in
the future, we can keep monitoring.
Test: ./gradlew benchmark:b-c:cC
Test: ./gradlew benchmark:b-b:cC
Test: ./gradlew recyclerview:r-b:cC
This CL also adds more logging, and unifies all logging under
"benchmark" tag. This logging was very useful in discovering and
diagnosing the priority problem, since it showed the edge cases where
jit finished *during* the measure pass.
Bug: 140773023
Bug: 142058671
Change-Id: If542e3cb8867165cf7b4688090ee534e68a23562
M benchmark/common/src/androidTest/java/androidx/benchmark/BenchmarkStateTest.kt
M benchmark/common/src/main/java/androidx/benchmark/BenchmarkState.kt
A benchmark/common/src/main/java/androidx/benchmark/ThreadPriority.kt
M benchmark/common/src/main/java/androidx/benchmark/WarmupManager.kt
M benchmark/junit4/src/main/java/androidx/benchmark/junit4/BenchmarkRule.kt
cc...@google.com <cc...@google.com> #9
After 15+ runs, recyclerview-benchmark and benchmark-benchmark are *significantly* more stable, see attached screenshots.
There may still be more opportunity here on big/little devices - enabling small cores, and using affinity to keep JIT there, with the main benchmark thread on a big core.
There may still be more opportunity here on big/little devices - enabling small cores, and using affinity to keep JIT there, with the main benchmark thread on a big core.
Description
CLs in build: