Status Update
No update yet.
Comments
ub...@gmail.com
For flowables / observables; we always use the Room IO thread fo rthe actual query since we may repeat the work.
For LiveData, ti si always the Room IO Thread.
Until we can extend Room Query methods to receive non-query parameter (e.g. a Scheduler) we cannot do this.
At least room catches this instantly since it will crash w/ db access on main thread exception.
ub...@gmail.com
Probably I should have described the problem more clearly.
I'm using Flowable, and hopefully it executes queries in background thread (at least, app does not crash with IllegalStateException, as with Single).
But nevertheless subscribing and disposing(!) does cause StrictMode violations. Stack trace examples:
D/StrictMode: StrictMode policy violation; ~duration=17 ms: android.os.StrictMode$StrictModeDiskWriteViolation: policy=31 violation=1
at android.os.StrictMode$AndroidBlockGuardPolicy.onWriteToDisk(StrictMode.java:1111)
at android.database.sqlite.SQLiteConnection.applyBlockGuardPolicy(SQLiteConnection.java:1043)
at android.database.sqlite.SQLiteConnection.execute(SQLiteConnection.java:552)
at android.database.sqlite.SQLiteSession.beginTransactionUnchecked(SQLiteSession.java:323)
at android.database.sqlite.SQLiteSession.beginTransaction(SQLiteSession.java:298)
at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:505)
at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:416)
at androidx.sqlite.db.framework.FrameworkSQLiteDatabase.beginTransaction(FrameworkSQLiteDatabase.java:69)
at androidx.room.InvalidationTracker.syncTriggers(InvalidationTracker.java:426)
at androidx.room.InvalidationTracker.syncTriggers(InvalidationTracker.java:465)
at androidx.room.InvalidationTracker.addObserver(InvalidationTracker.java:249)
at androidx.room.RxRoom$1.subscribe(RxRoom.java:79)
at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:71)
at io.reactivex.Flowable.subscribe(Flowable.java:14479)
at io.reactivex.internal.operators.flowable.FlowableObserveOn.subscribeActual(FlowableObserveOn.java:56)
at io.reactivex.Flowable.subscribe(Flowable.java:14479)
at io.reactivex.internal.operators.flowable.FlowableFlatMapMaybe.subscribeActual(FlowableFlatMapMaybe.java:54)
at io.reactivex.Flowable.subscribe(Flowable.java:14479)
at io.reactivex.Flowable.subscribe(Flowable.java:14416)
at io.reactivex.Flowable.subscribe(Flowable.java:14306)
at gmk57.strictroom.MainActivity.onCreate(MainActivity.java:30)
StrictMode policy violation; ~duration=108 ms: android.os.StrictMode$StrictModeDiskWriteViolation: policy=31 violation=1
at android.os.StrictMode$AndroidBlockGuardPolicy.onWriteToDisk(StrictMode.java:1111)
at android.database.sqlite.SQLiteConnection.applyBlockGuardPolicy(SQLiteConnection.java:1043)
at android.database.sqlite.SQLiteConnection.execute(SQLiteConnection.java:552)
at android.database.sqlite.SQLiteSession.beginTransactionUnchecked(SQLiteSession.java:323)
at android.database.sqlite.SQLiteSession.beginTransaction(SQLiteSession.java:298)
at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:505)
at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:416)
at androidx.sqlite.db.framework.FrameworkSQLiteDatabase.beginTransaction(FrameworkSQLiteDatabase.java:69)
at androidx.room.InvalidationTracker.syncTriggers(InvalidationTracker.java:426)
at androidx.room.InvalidationTracker.syncTriggers(InvalidationTracker.java:465)
at androidx.room.InvalidationTracker.removeObserver(InvalidationTracker.java:281)
at androidx.room.RxRoom$1$2.run(RxRoom.java:83)
at io.reactivex.disposables.ActionDisposable.onDisposed(ActionDisposable.java:30)
at io.reactivex.disposables.ActionDisposable.onDisposed(ActionDisposable.java:19)
at io.reactivex.disposables.ReferenceDisposable.dispose(ReferenceDisposable.java:43)
at io.reactivex.internal.disposables.DisposableHelper.dispose(DisposableHelper.java:124)
at io.reactivex.internal.disposables.SequentialDisposable.dispose(SequentialDisposable.java:72)
at io.reactivex.internal.operators.flowable.FlowableCreate$BaseEmitter.cancel(FlowableCreate.java:301)
at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.cancel(FlowableObserveOn.java:154)
at io.reactivex.internal.operators.flowable.FlowableFlatMapMaybe$FlatMapMaybeSubscriber.cancel(FlowableFlatMapMaybe.java:158)
at io.reactivex.internal.subscriptions.SubscriptionHelper.cancel(SubscriptionHelper.java:189)
at io.reactivex.internal.subscribers.LambdaSubscriber.cancel(LambdaSubscriber.java:119)
at io.reactivex.internal.subscribers.LambdaSubscriber.dispose(LambdaSubscriber.java:104)
at gmk57.strictroom.MainActivity.onDestroy(MainActivity.java:37)
ub...@gmail.com
da...@google.com
We need to investigate this better because these methods were not @WorkerThread before and we had to change them to fix some invalidation problems (see internal bug
Luckily this is not an issue for LiveData since it uses ComputableLiveData and always calls observe on a background thread.
ub...@gmail.com
Branch: androidx-master-dev
commit 79deed62f2188a6ce2d21244dd28c3dc8ab6d0de
Author: Daniel Santiago Rivera <danysantiago@google.com>
Date: Wed Oct 17 11:07:39 2018
Specify subscribeOn and unsubscribeOn scheduler in RxRoom.
Setting subscriber and unsubscriber schedulers prevents triggers being
synced in the main thread when subscribing or disposing a RxRoom
flowable / observable.
Moved beginTransaction() calls in InvalidationTracker outside the try,
finally with endTransaction() so that exceptions thrown by
beginTransaction() aren't aborted nor forgotten.
Bug: 117201279
Test: Manual test with sample app.
Change-Id: Iefb33dc63cf78e125a8805bd66a26ada365d8453
A room/integration-tests/testapp/src/androidTest/java/androidx/room/integration/testapp/test/InvalidationTrackerTest.java
M room/runtime/src/main/java/androidx/room/InvalidationTracker.java
M room/rxjava2/src/main/java/androidx/room/RxRoom.java
M room/rxjava2/src/test/java/androidx/room/RxRoomTest.java
da...@google.com
ub...@gmail.com
yb...@google.com
ub...@gmail.com
As it's an intermittent problem (triggered by specific transaction timing), it's difficult to surface in the first place, so this type of instrumented test cannot really reliably find the problem, it was hard enough trying to surface it with manually run code in a controlled environment. Also, the longer of a time interval we choose, the harder it gets to reproduce within a finite amount of time.
I think the main concern with fixing the issue is not actually the PR's ability to address the issue, but the potential to introduce regressions. I have no idea why there was no transactionality for the TRUNCATE path, was it really just because the thinking was that it was superfluous? This can only be addressed by existing test coverage and maybe insight from whoever wrote the original code and other knowledegable team members. I traced its history and found no helpful pointers, I remember finding that it always looked like this since it was introduced in the early days of Room.
Do you guys really want the sketchy black box test, or maybe just stick with some extra code documentation, existing regression tests, plus the repro case? It almost seems not worth the trouble.
yb...@google.com
Seems like it was added here:
For testing, I agree it is not great as it only shows as a flake but we have a couple tests stress similar to that and we usually use 10secs (to account for cloud devices). Under normal circumstances (device is not slowed down and db is not locked), it shouldn't take more than ~100ms to get the update after the right (usually much faster). But with virtual device testing, we cannot rely on timing hence we pick large enough numbers (if a virtual device idles for 10 seconds, than that is an infra problem).
It is at least better than nothing and if it flakes, that'll keep bugging us until finding a better solution.
Btw, if you can detect the exact ordering of events that will cause the problem, then we can add package private restricted APIs from the InvalidationTracker to have fine tuned control over them in tests.
ub...@gmail.com
da...@google.com
oh-oh, I'm sorry to hear that, we definitely don't require you to enter any SSH key password. If anything, only the built-in Git integration in Android Studio will ask you for Github credentials if you use it to push changes to Github. Maybe you have some additional plugin that is asking for it? The Github-based setup downloads the correct Android Studio version needed for the project, but some plugins are installed separately and can persist across IDE installations.
yb...@google.com
#13 i'm curious what went wrong w/ the Github setup. I don't want to hijack this bug but if you can either file an issue at
el...@google.com
ub...@gmail.com
I don't think I will add the test; ultimately I am unfamiliar with AOSP tooling and in particular the test setup for this, so it's an open-ended time commitment, and I already spent too much time on this tiny code problem.
I can still make a PR for the code change that fixes the bug (based on my testing via repro & actual app), but it's so trivial that you might just as well change it yourselves. As I mentioned in #10, the primary concern with the code change would be regressions, which are not captured by the envisioned test anyway, so I'd consider it defensible to go without test. The alternative, no fix, leaves TRUNCATE in a poor state.
ub...@gmail.com
I meant
yb...@google.com
are you still blocked on the SSH issue? FYI we do have other people sending PRs from Github without that issue so I would like to figure out what is broken in your case (as it might be affecting other people as well).
I want to respect your time though so if you don't want to deal with it anymore, you can send a PR without tests and we'll take it from there to add tests (though unfortunately, we cannot merge a PR without tests so we'll need to add them).
ub...@gmail.com
Thanks for your and Aurimas' help on the SSH issue last year, doing a checkout with filter and SSH caused the problems. I suggested updating the online instructions to point out this gotcha, not sure what happened after that on your end.
I am sending you a PR with the fix & added test. I gave it one more push to get if off my mind.
el...@google.com
ap...@google.com
Branch: androidx-main
commit a5c2900ca26e8f24d4492e5a0e90f0669702c680
Author: Uli Bubenheimer <bubenheimer@users.noreply.github.com>
Date: Mon Apr 19 09:22:25 2021
[GH] Fix intermittent InvalidationTracker issues in JournalMode.TRUNCATE
## Proposed Changes
- Fixes issue in Room JournalMode.TRUNCATE where the InvalidationTracker callback is sometimes invoked invalidly, too late, or not at all.
- Adds InvalidationTrackerBehavioralTest
## Testing
Test: ./gradlew test connectedCheck -x :room:room-benchmark:cC -x :room:integration-tests:room-incremental-annotation-processing:test
## Issues Fixed
Fixes:
This is an imported pull request from
Resolves #159
Github-Pr-Head-Sha: b32fc85dd1368db5fe046b1edfecdb82ac3e5478
GitOrigin-RevId: 169da50d9ce5d0be6aa686d603d51a380f46ab63
Change-Id: I0c04f25303043f45c8efe687df93f7122acbc7d4
A room/integration-tests/testapp/src/androidTest/java/androidx/room/integration/testapp/test/InvalidationTrackerBehavioralTest.java
M room/runtime/src/main/java/androidx/room/InvalidationTracker.java
Description
Component used: Room InvalidationTracker
Version used: 2.2.5
Devices/Android versions reproduced on: Android Studio emulator with Android R DP 2.1
Sample project:https://github.com/bubenheimer/invalidationtrackerbug
I am seeing consistency problems with the updates from InvalidationTracker when using JournalMode.TRUNCATE.
In my production app some updates are skipped - the observer is never called for a percentage of updates to a rarely updated table, causing the associated app screen to not get updated to the current state. I debugged into it and believe that this is associated with a lack of transactionality in InvalidationTracker.mRefreshRunnable. JournalMode.WRITE_AHEAD_LOGGING uses a transaction here, while JournalMode.TRUNCATE does not. Reads and writes of the table update log are disassociated, leaving the door open to overwriting recent changes to the log.
In the provided sample project I see the opposite effect, however: a lot of extra updates are triggered by InvalidationTracker with JournalMode.TRUNCATE. I am not sure what the underlying mechanism is for this behavior.
Exact effects are timing-related. The bottom line is that InvalidationTracker looks essentially broken for JournalMode.TRUNCATE. On the other hand I do not see significant issues with JournalMode.WRITE_AHEAD_LOGGING.