Room compiler takes 15 mins to do code generation [117401230]

Fixed

Bug

Status Update

No update yet.

Description

an...@gmail.com

created issue #1

Oct 8, 2018 04:10PM

Component used: android.arch.persistence.room:compiler
Version used: 1.1.1
Devices/Android versions reproduced on: Latest android emulator on Mac OS running Oreo image, Lenovo K6 Power
Also tried with Android studio 3.2 stable and Android 3.3 Preview version as well.

Building my project takes 15 min. If I comment out below line

annotationProcessor "android.arch.persistence.room:compiler:$ROOM_VERSION"

from my app's build.gradle, it takes only a minute to build the project. It's not a very big project (entity count wise) and I did not try reproducing the issue in a sample project. However, I can reproduce the

issue 100

% of the time with my project.

I have tried:

1. Cleaning. Rebuilding...
2. Invalidate caches/Restart.
3. Even tried migration to using all the androidx artifacts.

It seems like a bug in Room's annotationProcessor but I don't have a sample project to share.

Comments

yb...@google.com <yb...@google.com> #2Oct 8, 2018 04:44PM

Reassigned to da...@google.com.

There are a few more changes to DataStoreImpl's logic which makes a reevaluation necessary.

Now with Yigit's changes, datastore's cold flow always tries to do a dirty read by readState which calls readAndInitOrPropagateAndThrowFailure that invokes readDataOrHandleCorruption only in InitDataStore's first run. Any following read is triggered by IPC's listener which calls readDataAndUpdateCache which updates the cache to ReadException returned from attemptRead.

A ReadException is not automatically handled and stays in cache until a updateData call which revisits readAndInitOrPropagateAndThrowFailure, but it doesn't handle the corruption exception because it's already not the first run. In such case, the handleUpdate call proceeds to transformAndWrite which does not catch the exceptions thrown from readDataFromFileOrDefault.

an...@gmail.com <an...@gmail.com> #3Oct 8, 2018 04:59PM

can we create a test failure for this? I'm not fully sure what is going wrong here, it is a bit hard to track w/o a repro case.

an...@gmail.com <an...@gmail.com> #4Oct 9, 2018 08:21PM

can we create a test failure for this?

Sure I'm working on the code changes.

In the meantime I think this bug might makes the errors e.g. in b/309382591 harder to recover because it doesn't try to handle the corruption until the app restarts so the DS can readAndInit again (with corruption handler logic).

room-compiler-performance-issue.zip

6.7 MB

Download

an...@gmail.com <an...@gmail.com> #5Oct 10, 2018 05:16AM

Project: platform/frameworks/support
Branch: androidx-main

commit 383a67d0e5a9b35e394b212db35bdee892b167fb
Author: Zhiyuan Wang <zhiyuanwang@google.com>
Date: Wed Feb 21 17:04:40 2024

Add tests to verify that CorruptionException is not properly handled after DataStore's first disk read. If the internal state is [ReadException], it will not recover for the rest of the lifecycle.

Fix in the follow up change.

Bug: 289582516
Test: ./gradlew :datastore:datastore-core:jvmTest
Change-Id: Ie77900b5c2cb8fb8e55d4c40364491dea9250784

M datastore/datastore-core/src/commonTest/kotlin/androidx/datastore/core/SingleProcessDataStoreTest.kt

https://android-review.googlesource.com/2972402

an...@gmail.com <an...@gmail.com> #6Oct 10, 2018 11:39AM

Project: platform/frameworks/support
Branch: androidx-main

commit 70085fee2af278bb8c3e45e1acfcf3f4cbb05c5a
Author: Zhiyuan Wang <zhiyuanwang@google.com>
Date: Thu Feb 22 15:08:34 2024

DataStore handles CorruptionException for all reads and writes.

Before this change, DataStore didn't handle the CorruptionException for both the reads and writes after initialization.

Fixes: 289582516
Test: ./gradlew :datastore:datastore-core:jvmTest
Change-Id: Icb07b140017b62a905d3626f82aeb993f636c053

M datastore/datastore-core/src/commonMain/kotlin/androidx/datastore/core/DataStoreImpl.kt
M datastore/datastore-core/src/commonTest/kotlin/androidx/datastore/core/SingleProcessDataStoreTest.kt

https://android-review.googlesource.com/2974198

an...@gmail.com <an...@gmail.com> #7Oct 10, 2018 11:57AM

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.datastore:datastore-core:1.1.0-beta02
androidx.datastore:datastore-core-android:1.1.0-beta02
androidx.datastore:datastore-core-iosarm64:1.1.0-beta02
androidx.datastore:datastore-core-iossimulatorarm64:1.1.0-beta02
androidx.datastore:datastore-core-iosx64:1.1.0-beta02
androidx.datastore:datastore-core-jvm:1.1.0-beta02
androidx.datastore:datastore-core-linuxx64:1.1.0-beta02
androidx.datastore:datastore-core-macosarm64:1.1.0-beta02
androidx.datastore:datastore-core-macosx64:1.1.0-beta02

da...@google.com <da...@google.com> #8Oct 10, 2018 11:35PM

Thanks for the sample app. I haven't gotten to the bottom of the issue yet but I wanted to give you an update and it does seem indeed that Room is taking a very long time to parse and understand your queries via the ANTLR generated parser that Room uses. I'm not sure what is so special about your queries that is causing this, but I'll comment back with an update once I find out. Sadly for now I don't have a recommendation on a workaround.

da...@google.com <da...@google.com> #9Oct 11, 2018 03:22AM

Alright, I've drill down the problem to being the various "OR <col_name> LIKE '%' : text '%'" in your queries. Specifically they cause the SQLiteParser in Room to go down a path with poor performance as described in

https://github.com/antlr/antlr4/issues/1398. A fix for this involves updating Room's ANTRL runtime library and generated parser code with the latest version. This update will likely be available in Room 2.1.0-alpha02.

an...@gmail.com <an...@gmail.com> #10Oct 11, 2018 02:09PM

Thanks for the update. Is 2.1.0-alpha02 around the corner?

ap...@google.com <ap...@google.com> #11Oct 11, 2018 09:57PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit 8e46b9ca1f55b040a18dd86833880b5106d103b2
Author: Daniel Santiago Rivera <danysantiago@google.com>
Date: Wed Oct 10 19:17:50 2018

Update Room's ANTLR Parser to 4.7.1

Updating to the latest ANTLR version fixes the performance issue
described in

https://github.com/antlr/antlr4/issues/1398 which affects
Room's SQL Parser since it contains left-recursive rules. Specifically
SQLite.g4 has multiple recursions to describe an expression as defined
in

https://www.sqlite.org/lang_expr.html, meaning if a user has certain
queries that causes the parser to use the adaptivePredict algorithm
then the build time increases significantly due to Room's query
parsing.

Also bumped Room's version to 2.1.0-alpha02

Bug: 117401230
Test: ./gradlew room:integration-tests:testapp:cC
Change-Id: I4d0799a11ccee816ab209cbf74e7dcf2b2ce1a18

M buildSrc/src/main/kotlin/androidx/build/LibraryVersions.kt
M buildSrc/src/main/kotlin/androidx/build/dependencies/Dependencies.kt
M room/integration-tests/testapp/src/androidTest/java/androidx/room/integration/testapp/dao/UserDao.java

https://android-review.googlesource.com/786077

https://goto.google.com/android-sha1/8e46b9ca1f55b040a18dd86833880b5106d103b2

da...@google.com <da...@google.com> #12Oct 15, 2018 11:22PM

Marked as fixed.

No concrete date on 2.1.0-alpha02, I want to get a few more bugs in before making the next alpha release. So maybe by the end of the month.

an...@gmail.com <an...@gmail.com> #13Oct 16, 2018 08:22PM

Thanks.

On Tue, Oct 16, 2018, 4:52 AM <buganizer-system@google.com> wrote:

- Show quoted text -

--

- Ankit