Status Update
No update yet.
Comments
ra...@google.com
Initial loads in LimitOffsetPagingSource run in transaction but subsequent loads do not because of cost of transaction. Loads rely on count query to return correct LoadResult.
If count query is outdated, i.e. large amount of data deleted, LimitOffsetPagingSource can return i.e. empty/wrong data and incorrect itemsBefore/itemsAfter/prevKey/nextKey. This can happen if outdated PagingSource doesn't get invalidated in time.
ar...@gmail.com
Branch: androidx-main
commit 6bd2347ff259c28b34789c8569b7a298d2a8fd8d
Author: Clara Fok <clarafok@google.com>
Date: Fri Jul 09 12:53:22 2021
Implement Invalid return type in PagingSource
Added third return type for LoadResult sealed class in PagingSource. It
was added as a support feature to handle race scenarios between data loads
and database invalidations, such as if Room's InvalidationTracker does
not propagate invalidation signal to the PagingSource in time and the
PagingSource continues querying an updated database. This can result in
data being loaded onto a stale PageFetcherSnapshot causing duplicated or
missing data. Related bug regarding race scenario in Room
For example, if data was deleted and items shifted positions,
positionally-keyed paging sources can end up loading duplicated items.
Paging handles this return type by discarding the loaded data and
invalidating the paging source. This will trigger a new paging source to
be generated. This return type is also supported for Paging2 APIs
leveraging PagingSource such as LivePagedList and RxPagedList.
Aside from handling invalid data due to races, this return type can be
leveraged in general where the database or network returns potentially
invalid or stale data that needs to be discarded.
Bug: 192013267
Test: ./gradlew :paging:paging-common:test
Relnote: "A third LoadResult return type LoadResult.Invalid is added to
PagingSource. When a PagingSource.load returns
LoadResult.Invalid, paging will discard the loaded data and
invalidate the PagingSource. This return type is designed to
handle potentially invalid or stale data that can be returned
from the database or network.
For example, if the underlying database gets written into but
the PagingSource does not invalidate in time, it may return
inconsistent results if its implementation depends on the
immutability of the backing dataset it loads from (e.g., LIMIT
OFFSET style db implementations). In this scenario, it is
recommended to check for invalidation after loading and to
return LoadResult.Invalid, which causes Paging to discard any
pending or future load requests to this PagingSource and
invalidate it.
This return type is also supported by Paging2 API that leverages
LivePagedList or RxPagedList. When using a PagingSource with
Paging2's PagedList APIs, the PagedList is immediately detached,
stopping further attempts to load data on this PagedList and
triggers invalidation on the PagingSource.
LoadResult is a sealed class, which means this is a
source-incomptabile change such that use cases directly using
PagingSource.load results will have to handle LoadResult.Invalid
at compile time. For example, Kotlin users leveraging
exhaustive-when to check return type will have to add a check
for Invalid type."
Change-Id: Id6bd3f2544f1ba97a95a0d590353438a95fedf2a
M paging/common/api/current.txt
M paging/common/api/public_plus_experimental_current.txt
M paging/common/api/restricted_current.txt
M paging/common/src/main/kotlin/androidx/paging/PageFetcherSnapshot.kt
M paging/common/src/main/kotlin/androidx/paging/PagedList.kt
M paging/common/src/main/kotlin/androidx/paging/PagingSource.kt
M paging/common/src/test/kotlin/androidx/paging/PageFetcherSnapshotTest.kt
M paging/common/src/test/kotlin/androidx/paging/PageFetcherTest.kt
M paging/common/src/test/kotlin/androidx/paging/PagedListTest.kt
M paging/common/src/test/kotlin/androidx/paging/PagingDataDifferTest.kt
M paging/common/src/test/kotlin/androidx/paging/PagingSourceTest.kt
M paging/integration-tests/testapp/src/main/java/androidx/paging/integration/testapp/v3room/V3RemoteMediator.kt
M paging/integration-tests/testapp/src/main/java/androidx/paging/integration/testapp/v3room/V3RoomViewModel.kt
ar...@gmail.com
Branch: androidx-main
commit 3882ac8364fd905c9935d38c93f84599009226d6
Author: Clara Fok <clarafok@google.com>
Date: Fri Jul 09 14:17:48 2021
Implement LoadResult.Invalid to handle race
For non-initial loads in LimitOffsetPagingSource, loads will run Room
InvalidationTracker's refreshVersionSync to synchronously check if
tables have been invalidated and notify observers. If invalidated,
load will return LoadResult.Invalid.
Paging3 will handle Invalid return type by invalidating the paging
source and closing the page event flow, such that no more loads will be
attempted on this paging source.
Note that refreshVersionSync is a temporary solution for
LimitOffsetPagingSource to check table invalidation. Another solution
that does not require a transaction is in development. Related
Bug: 191806126
Test: ./gradlew :room:integration-tests:room-testapp-kotlin:cC
Test: ./gradlew room:room-paging:cC
Change-Id: I48634ab95949bae608374f5156a4edd0500a74a3
M room/integration-tests/kotlintestapp/src/androidTest/java/androidx/room/integration/kotlintestapp/test/PagingSourceTest.kt
M room/room-paging/build.gradle
M room/room-paging/src/androidTest/kotlin/androidx/room/paging/LimitOffsetPagingSourceTest.kt
M room/room-paging/src/main/kotlin/androidx/room/paging/LimitOffsetPagingSource.kt
ra...@google.com
We marked it duplicate because if your Worker ran long enough, it would trigger the other bug which had a similar sticky Notification. However, you have a combination of problems one of which we fixed in 2.3.4.
Your RxWorker implementation has a bug. You are calling setForegroundAsync(...), but you are not waiting for completion.
setForegroundAsync(...) returns a ListenableFuture that you should be waiting for.
What happens is that your Worker running on a dedicated Executor is racing against setForegroundAsync() and in some cases it ends up being called after your Worker returns a Result.success().
ar...@gmail.com
Yep, the call does return the Future but get is called on it. I was under an impression that it blocks the execution until the Future delivers a result. Isn’t that the case here?
ra...@google.com
Yes, that is how you should be blocking on the ListenableFuture. When I last looked at your code, you were not doing it correctly.
ar...@gmail.com
Sorry, I’m confused. The sample in the opening comment has this:
Single
.fromCallable {
setForegroundAsync(ForegroundInfo(42, createNotification())).get().toOptional()
}
.map { Result.success() }
What it does (from my POV):
- changes the worker to be foregrounded using
setForegroundAsync; - blocks until the
setForegroundAsyncListenableFutureprovides it; - emits a successful worker result.
Is there a bug in this implementation? I’m confused because #5 states that there is a bug in the sample but #7 confirms that blocking via get() should work.
ra...@google.com
Are you calling setForegroundAsync() in your constructor ?
ra...@google.com
WorkManager has multiple schedulers (in-process, JobScheduler, the AlarmManager based scheduler etc.).
What's happening in your case, is that the in-process Scheduler is picking up the RxWorker implementation and your Worker completes successfully by returning an instance of Result.
However, around the same time JobScheduler also wants to execute the same WorkRequest. For us to be able to de-dupe the execution, we create an instance of the Worker and check to see if it needs to be scheduled. As your logs point out, we correctly identify that the Worker is done.
D/WM-WorkerWrapper: Status for 2d44c9de-0e3c-4518-b960-c282ba924638 is SUCCEEDED; not doing any work
D/WM-WorkerWrapper: REDACTED.Application$ForegroundWorker is not in ENQUEUED state. Nothing more to do.
However in your RxWorker implementation the side-effect of calling setForegroundAsync() in createWork() is that a call to notify() happens despite your Worker not having to run at all. The reason this happens is because there is no easy way for you to be able to tell that the Worker actually started running (because that happens only when an underlying subscription is created).
We made this a lot more ergonomic in 2.4.0-alpha02 (soon to be released). Attempts to notify are ignored when a Worker is done.
--
In your RxWorker for a workaround around the notifications, you could do something like:
Single
.fromCallable {
val workInfo = WorkManager.getInstance(context).getWorkInfoById(getId()).get()
if (workInfo != null && !workInfo.getState().isFinished()) {
setForegroundAsync(ForegroundInfo(42, createNotification())).get().toOptional()
}
}
.map { Result.success() }
Going forward, we automatically handle this for you.
ar...@gmail.com
Awesome, thanks for the detailed explanation! I’ll look forward to the 2.4.0.
Sorry for bothering you with this. It was a bit weird to follow the documentation and still having an issue.
ra...@google.com
No problem. In 2.4.0-alpha02 you can just call setForegroundAsync() and not have to worry about calling get().
We do the rest :)
Description
* Android version: 10
* Device: emulator API level 29, x86
Put the following code in the `Application` subclass and call the `scheduleForegroundWork` method in the `onCreate` one. Run the application.
```kotlin
private fun scheduleForegroundWork() {
val workRequest = OneTimeWorkRequest.Builder(ForegroundWorker::class.java).build()
WorkManager
.getInstance(this)
.enqueueUniqueWork("foreground-work", ExistingWorkPolicy.REPLACE, workRequest)
}
class ForegroundWorker(private val context: Context, parameters: WorkerParameters) : RxWorker(context, parameters) {
companion object {
private const val NOTIFICATION_CHANNEL_ID = "foreground-work"
}
override fun createWork() = Single
.fromCallable {
createNotificationChannel()
setForegroundAsync(ForegroundInfo(42, createNotification())).get().toOptional()
}
.map { Result.success() }
private fun createNotificationChannel() {
val channel = NotificationChannel(
NOTIFICATION_CHANNEL_ID,
"Foreground work",
NotificationManager.IMPORTANCE_LOW
)
context.getSystemService<NotificationManager>()?.createNotificationChannel(channel)
}
private fun createNotification() = Notification.Builder(context, NOTIFICATION_CHANNEL_ID)
.setSmallIcon(R.drawable.ic_notification_upload)
.setContentText("Hello there from the foreground work")
.build()
}
```
* Actual behavior: the notification is shown but is never dismissed — not that the `Result` is provided immediately.
* Expected behavior: the notification is hidden.
The issue can be mitigated via adding a delay before producing a result:
```diff
}
+ .delay(1, TimeUnit.SECONDS)
.map { Result.success() }
```
I suspect that the foreground service is cancelled all the time when a result is provided, maybe there is a race somewhere.