Fixed
Status Update
No update yet.
Comments
ro...@gmail.com
Yigit, do you have time to fix it?
reemission of the same liveData is racy
reemission of the same liveData is racy
ro...@gmail.com
yea i'll take it.
du...@google.com
du...@google.com
Thanks for the detailed analysis. This may not be an issue anymore since we've started using Main.immediate there but I' not sure; I'll try to create a test case.
ro...@gmail.com
just emitting same live data reproduces the issue.
@Test
fun raceTest() {
val subLiveData = MutableLiveData(1)
val subject = liveData(testScope.coroutineContext) {
emitSource(subLiveData)
emitSource(subLiveData) //crashes
}
subject.addObserver().apply {
testScope.advanceUntilIdle()
}
}
@Test
fun raceTest() {
val subLiveData = MutableLiveData(1)
val subject = liveData(testScope.coroutineContext) {
emitSource(subLiveData)
emitSource(subLiveData) //crashes
}
subject.addObserver().apply {
testScope.advanceUntilIdle()
}
}
du...@google.com
With 2.2.0-alpha04 (that use Main.immediate), the issue seems to be still there (I tested it by calling emitSource() twice, like your test case)
ro...@gmail.com
yea sorry immediate does not fix it.
I actually have a WIP fix for it:
https://android-review.googlesource.com/c/platform/frameworks/support/+/1112186
if your case is the one i found (emitting same LiveData multiple times, as shown in #5) you can work around it by adding a dummy transformation.
val subLiveData = MutableLiveData(1)
val subject = liveData(testScope.coroutineContext) {
emitSource(subLiveData.map {it })
emitSource(subLiveData.map {it} )
}
I actually have a WIP fix for it:
if your case is the one i found (emitting same LiveData multiple times, as shown in #5) you can work around it by adding a dummy transformation.
val subLiveData = MutableLiveData(1)
val subject = liveData(testScope.coroutineContext) {
emitSource(subLiveData.map {it })
emitSource(subLiveData.map {it} )
}
ro...@gmail.com
Project: platform/frameworks/support
Branch: androidx-master-dev
commit af12e75e6b4110f48e44ca121466943909de8f06
Author: Yigit Boyar <yboyar@google.com>
Date: Tue Sep 03 12:58:11 2019
Fix coroutine livedata race condition
This CL fixes a bug in liveData builder where emitting same
LiveData source twice would make it crash because the second
emission registry could possibly happen before first one is
removed as source.
We fix it by using a suspending dispose function. It does feel
a bit hacky but we cannot make DisposableHandle.dispose async
and we do not want to block there. This does not mean that there
is a problem if developer disposes it manually since our emit
functions take care of making sure it disposes (and there is
no other way to add source to the underlying MediatorLiveData)
Bug: 140249349
Test: BuildLiveDataTest#raceTest_*
Change-Id: I0b464c242a583da4669af195cf2504e2adc4de40
M lifecycle/lifecycle-livedata-ktx/api/2.2.0-alpha05.txt
M lifecycle/lifecycle-livedata-ktx/api/current.txt
M lifecycle/lifecycle-livedata-ktx/api/public_plus_experimental_2.2.0-alpha05.txt
M lifecycle/lifecycle-livedata-ktx/api/public_plus_experimental_current.txt
M lifecycle/lifecycle-livedata-ktx/api/restricted_2.2.0-alpha05.txt
M lifecycle/lifecycle-livedata-ktx/api/restricted_current.txt
M lifecycle/lifecycle-livedata-ktx/src/main/java/androidx/lifecycle/CoroutineLiveData.kt
M lifecycle/lifecycle-livedata-ktx/src/test/java/androidx/lifecycle/BuildLiveDataTest.kt
https://android-review.googlesource.com/1112186
https://goto.google.com/android-sha1/af12e75e6b4110f48e44ca121466943909de8f06
Branch: androidx-master-dev
commit af12e75e6b4110f48e44ca121466943909de8f06
Author: Yigit Boyar <yboyar@google.com>
Date: Tue Sep 03 12:58:11 2019
Fix coroutine livedata race condition
This CL fixes a bug in liveData builder where emitting same
LiveData source twice would make it crash because the second
emission registry could possibly happen before first one is
removed as source.
We fix it by using a suspending dispose function. It does feel
a bit hacky but we cannot make DisposableHandle.dispose async
and we do not want to block there. This does not mean that there
is a problem if developer disposes it manually since our emit
functions take care of making sure it disposes (and there is
no other way to add source to the underlying MediatorLiveData)
Bug: 140249349
Test: BuildLiveDataTest#raceTest_*
Change-Id: I0b464c242a583da4669af195cf2504e2adc4de40
M lifecycle/lifecycle-livedata-ktx/api/2.2.0-alpha05.txt
M lifecycle/lifecycle-livedata-ktx/api/current.txt
M lifecycle/lifecycle-livedata-ktx/api/public_plus_experimental_2.2.0-alpha05.txt
M lifecycle/lifecycle-livedata-ktx/api/public_plus_experimental_current.txt
M lifecycle/lifecycle-livedata-ktx/api/restricted_2.2.0-alpha05.txt
M lifecycle/lifecycle-livedata-ktx/api/restricted_current.txt
M lifecycle/lifecycle-livedata-ktx/src/main/java/androidx/lifecycle/CoroutineLiveData.kt
M lifecycle/lifecycle-livedata-ktx/src/test/java/androidx/lifecycle/BuildLiveDataTest.kt
du...@google.com
Noting for myself, but looks like this is an issue with using flow.collect vs flow.collectLatest, which stops the uireceiver from getting updated in time.
ro...@gmail.com
Hi again. Managed to rework the logic on using Paging 3, with some of your advices.
But I am now plagued by an SIGABRT error (not often, appears sporadicly, didn't do that before), do this have anything with Paging 3 to do ?
2020-07-03 19:53:14.361 1400-1737/no.rogo.emptyfuel E/StudioTransport: Agent::SubmitAgentTasks error_code=14 'Bad file descriptor' ''
2020-07-03 19:53:16.288 1400-1748/no.rogo.emptyfuel A/libc: fdsan: attempted to close file descriptor 162, expected to be unowned, actually owned by unique_fd 0x79723b86c414
2020-07-03 19:53:16.315 1400-1737/no.rogo.emptyfuel A/libc: fdsan: attempted to close file descriptor 162, expected to be unowned, actually owned by unique_fd 0x79723b86c414
2020-07-03 19:53:16.432 1400-1744/no.rogo.emptyfuel A/libc: fdsan: attempted to close file descriptor 162, expected to be unowned, actually owned by unique_fd 0x79723b86c414
2020-07-03 19:53:17.346 1400-1744/no.rogo.emptyfuel A/libc: Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 1744 (Studio:Heartbea), pid 1400 (.rogo.emptyfuel)
2020-07-03 19:53:17.598 1400-1744/no.rogo.emptyfuel A/libc: crash_dump helper failed to exec
RG
ro...@gmail.com
In another hand...
I see that RemoteMediator's "override suspend load" trigger LoadType.PREPEND each time the db is updated from another process, and LoadType.APPEND might trigger sporadicly with state.lastItemOrNull with null, even there are data in the room table.
(I may file another issue for this one, when I have found the supected reason).
RG
ro...@gmail.com
In another hand 2...
It seems that the LoadType.PREPEND requests are legitimate, but there are subsequent LoadType.APPEND reoccurring (with same PID) when load suspends (IE room database query with "withTransaction") on the first LoadType.APPEND. Shouldn't load wait for the returning MediatorResult before calling load again (Loog for my arrows to the right in table below)?
2020-07-03 20:38:32.883 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = REFRESH )
2020-07-03 20:38:32.885 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.REFRESH returning null (initial load)
2020-07-03 20:38:36.177 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: getNullableDeviceLocation = DeviceLocation(userId=user, location_valid=true, timestamp=2020-07-03 20:35:18, latitude=61.8907267, longitude=6.6761067, locationAccuracy=5.0, hasLocationAccuracy=true, altitude=0.0, altitudeAccuracy=0.5, hasAltitude=true, hasAltitudeAccuracy=true, bearing=90.0, bearingAccuracy=30.0, hasBearing=true, hasBearingAccuracy=true, speed=0.0, speedAccuracy=0.5, hasSpeed=true, hasSpeedAccuracy=true, isFromMockProvider=false, provider=fused, countrycode=NO, countryname=Norway, hasCountry=true, countrysource=GPS)
2020-07-03 20:38:36.190 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: resets range to 1.0
2020-07-03 20:38:36.205 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: range = 1.0
2020-07-03 20:38:36.205 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: countryCode = NO
2020-07-03 20:38:36.205 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: userSelection.selectionArea = Global
2020-07-03 20:38:36.205 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: userSelection.selectionSorting = Nearest
2020-07-03 20:38:36.205 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: global, nearest, range = 1.0
2020-07-03 20:38:40.113 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: req.size = 20
2020-07-03 20:38:40.455 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 123431: Tenden kai: 61.898161, 6.706100 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.602 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 567: Gas Tech Stryn: 61.903396, 6.703216 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.708 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 67335: Shell: 61.90265549999999, 6.7093816 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.757 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 528: Grønn Kontakt Stryn: 61.902648, 6.715088 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.772 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 4: YX Stryn: 61.900956, 6.717184 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.817 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 1: Esso Stryn: 61.902877, 6.717453 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.865 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 243: Rema 1000 Stryn: 61.9036521911621, 6.71779918670654 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.879 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 209: Berge Co: 61.9050559997559, 6.72660493850708 -> 61.8907267, 6.6761067
2020-07-03 20:38:40.943 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 2: Circle K Langesethøgda: 61.899546, 6.613435 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.001 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 44: YX Aut. Innvik : 61.851811, 6.614512 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.073 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 127695: YX Innvik: 61.8517929, 6.614505599999998 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.135 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 3: Circle K Automat Olden: 61.841645, 6.811233 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.280 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 127806: Park Mount Skala: 61.86730249999999, 6.879369699999999 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.403 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 6: YX Hornindal: 61.967029, 6.525517 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.436 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 26876: Grønn Kontakt Grodås: 61.969173, 6.525817 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.475 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 127696: Utvik brygge: 61.8066701, 6.528109599999999 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.482 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 73186: mh24 utvik: 61.8065922, 6.518116999999999 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.494 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 506: MH24 Byrkjelo: 61.734587, 6.5068259 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.539 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 832: Viking Byrkjelo: 61.7332361, 6.507499099999999 -> 61.8907267, 6.6761067
2020-07-03 20:38:41.550 4736-4901/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 96161: Grønn Kontakt Byrkjelo: 61.733450, 6.506469 -> 61.8907267, 6.6761067
2020-07-03 20:38:50.603 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = PREPEND )
2020-07-03 20:38:50.604 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND state.lastItemOrNull()?.stationId = 96161
2020-07-03 20:38:50.604 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND returning endOfPaginationReached = true
2020-07-03 20:38:50.625 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = APPEND )
2020-07-03 20:38:50.625 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.APPEND state.lastItem.stationId = 96161
2020-07-03 20:38:50.632 4736-4904/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.APPEND querying remoteKeyFromStationId lastItem.stationID = 96161 <--- Suspends !!
2020-07-03 20:38:51.226 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = APPEND ) <--- Another arrives !!
2020-07-03 20:38:51.226 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.APPEND state.lastItem.stationId = 96161
2020-07-03 20:38:51.234 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.APPEND querying remoteKeyFromStationId lastItem.stationID = 96161 <--- Suspends again !!
2020-07-03 20:38:51.316 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.APPEND remoteKeyFromStationId found = StationListRemoteKey(stationId=96161, prevKey=null, nextKey=2) <---Data arrives
2020-07-03 20:38:51.316 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.APPEND returning remoteKey.nextKey = 2
2020-07-03 20:38:51.328 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: getNullableDeviceLocation = DeviceLocation(userId=user, location_valid=true, timestamp=2020-07-03 20:38:40, latitude=61.8907267, longitude=6.6761067, locationAccuracy=5.0, hasLocationAccuracy=true, altitude=0.0, altitudeAccuracy=0.5, hasAltitude=true, hasAltitudeAccuracy=true, bearing=90.0, bearingAccuracy=30.0, hasBearing=true, hasBearingAccuracy=true, speed=0.0, speedAccuracy=0.5, hasSpeed=true, hasSpeedAccuracy=true, isFromMockProvider=false, provider=fused, countrycode=NO, countryname=Norway, hasCountry=true, countrysource=GPS)
2020-07-03 20:38:51.355 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: range = 1.0
2020-07-03 20:38:51.356 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: countryCode = NO
2020-07-03 20:38:51.356 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: userSelection.selectionArea = Global
2020-07-03 20:38:51.356 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: userSelection.selectionSorting = Nearest
2020-07-03 20:38:51.356 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: global, nearest, range = 1.0
2020-07-03 20:38:51.785 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: req.size = 20
2020-07-03 20:38:51.788 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 7: CK Bjørnereim : 61.733405, 6.502145 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.789 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 127692: ???? ???? Esso: 61.732023, 6.5071461 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.789 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 45: Best Byrkjelo: 61.7319488525391, 6.50718832015991 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.790 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 130808: Best Byrkjelo: 61.7318791, 6.5069594 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.791 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 82745: YX Hellesylt: 62.08562499999999, 6.870086000000001 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.792 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 835: Circle K Automat Hjelledalen: 61.91847730000001, 7.169982 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.798 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 8: Best Sandane: 61.769847, 6.232689 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.805 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 836: Best: 61.76977239999999, 6.232649899999999 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.811 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 527: Tesla Supercharge Sandane : 61.777968, 6.213336 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.811 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 629: Circle K Automat Sandane: 61.782930, 6.202792 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.813 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 122547: Uno X Nordfjordeid: 61.910468, 6.003159 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.813 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 76385: SHELL CRT NORDFJORDEID: 61.90404, 5.99887 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.817 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 837: Circle K Eid: 61.90622059999999, 5.993726400000001 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.818 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 43: Circle K Nordfjordeid: 61.906135559082, 5.99368333816528 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.819 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 25790: Esso Eid: 61.9063877, 5.991443400000001 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.820 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 760: Grøn Kontakt Amfi Nordfjordeid: 61.908942, 5.985504 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.821 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 838: Yx Geiranger: 62.1026487, 7.2053619 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.824 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 25791: Eid Servicesenter A/S: 61.9121035, 5.980904 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.834 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 10: Esso Nordfjordeid: 61.912197, 5.980782 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.841 4736-4903/no.rogo.emptyfuel I/StationListRemoteMediator2: load: station map 126658: EID: 61.91230869999999, 5.980252399999999 -> 61.8907267, 6.6761067
2020-07-03 20:38:51.943 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = PREPEND )
2020-07-03 20:38:51.944 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND state.lastItemOrNull()?.stationId = 96161
2020-07-03 20:38:51.944 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND returning endOfPaginationReached = true
2020-07-03 20:38:52.441 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = PREPEND )
2020-07-03 20:38:52.441 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND state.lastItemOrNull()?.stationId = 126658
2020-07-03 20:38:52.441 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND returning endOfPaginationReached = true
2020-07-03 20:39:26.220 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = PREPEND )
2020-07-03 20:39:26.221 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND state.lastItemOrNull()?.stationId = 126658
2020-07-03 20:39:26.221 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND returning endOfPaginationReached = true
2020-07-03 20:49:24.820 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: ( loadType = PREPEND )
2020-07-03 20:49:24.820 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND state.lastItemOrNull()?.stationId = 126658
2020-07-03 20:49:24.821 4736-4736/no.rogo.emptyfuel I/StationListRemoteMediator2: load: LoadType.PREPEND returning endOfPaginationReached = true
Here's my new implementation for the RemoteMediator:
private const val STATION_LIST_STARTING_PAGE_INDEX = 1
@OptIn(ExperimentalPagingApi::class)
class StationListRemoteMediator2(
private val appDatabase: AppDatabase
): RemoteMediator<Int, StationWithKindPrice>(){
private val TAG by lazy { this::class.java.simpleName }
override suspend fun load(
loadType: LoadType,
state: PagingState<Int, StationWithKindPrice>
): MediatorResult {
Log.i(TAG, "load: ( loadType = $loadType )")
var page = when(loadType){
LoadType.REFRESH -> {
Log.i(TAG, "load: LoadType.REFRESH returning null (initial load)")
null
}
LoadType.PREPEND -> {
Log.i(TAG,
"load: LoadType.PREPEND state.lastItemOrNull()?.stationId = " +
"${state.lastItemOrNull()?.stationId}")
Log.i(TAG, "load: LoadType.PREPEND returning endOfPaginationReached = true")
return MediatorResult.Success(endOfPaginationReached = true)
}
LoadType.APPEND -> {
val lastItem = state.lastItemOrNull()?.let {
Log.i(TAG, "load: LoadType.APPEND state.lastItem.stationId = ${it.stationId}")
return@let it
} ?: let {
Log.i(TAG, "load: LoadType.APPEND state.lastItem = null, no data to append")
return MediatorResult.Success(endOfPaginationReached = true)
}
val remoteKey = appDatabase.withTransaction { //Suspecting this suspend to open up for another 'load' call
Log.i(
TAG, "load: LoadType.APPEND querying remoteKeyFromStationId " +
"lastItem.stationID = ${lastItem.stationId}"
)
appDatabase.stationListRemoteKeyDao().remoteKeyFromStationId(lastItem.stationId)
}?.let {
Log.i(TAG, "load: LoadType.APPEND remoteKeyFromStationId found = $it")
return@let it
} ?: let {
Log.w(TAG, "load: LoadType.APPEND remoteKeyFromStationID found null")
return MediatorResult.Success(endOfPaginationReached = true)
}
Log.i(
TAG,
"load: LoadType.APPEND returning remoteKey.nextKey = ${remoteKey.nextKey}"
)
remoteKey.nextKey ?: return MediatorResult.Success(endOfPaginationReached = true)
}
}
try {
if(page==null) page=1
val apiService = APIFamappClientFactory.makeAPIFamappInterfaceService()
var endOfPaginationReached = false
val deviceLocation =
appDatabase.deviceLocationDao().getNullableDeviceLocation()?.let {
Log.i(TAG, "load: getNullableDeviceLocation = $it")
return@let it
} ?:let{
Log.w(TAG, "load: getNullableDeviceLocation == null", )
return MediatorResult.Error(Throwable("No location data!!"))
}
val user = appDatabase.userDao().getUser()
?:return MediatorResult.Error(Throwable("No user registered yet !!"))
val userSelection = appDatabase.userSelectionDao().getNullableUserSelection()
?:return MediatorResult.Error(Throwable("No user selections !!"))
var range:String? = if(page <= 1){
Log.i(TAG, "load: resets range to 1.0")
CoroutineScope(Dispatchers.IO).launch {
appDatabase.userSelectionDao().updateCurrentRange("1.0")
}
"1.0"
} else userSelection.currentRange;
Log.i(TAG, "load: range = $range")
val countryCode: String
countryCode = if(userSelection.selectionAutoCountry)
{
deviceLocation.countrycode
?:user.tmsNWCountryISO?.toUpperCase(Locale.ROOT)
?:user.tmsSIMCountryISO?.toUpperCase(Locale.ROOT)
?:userSelection.selectionCountry?:"--"
}
else
{
userSelection.selectionCountry
?:user.tmsNWCountryISO?.toUpperCase(Locale.ROOT)
?:user.tmsSIMCountryISO?.toUpperCase(Locale.ROOT)
?:deviceLocation.countrycode?:"--"
}
Log.i(TAG,"load: countryCode = $countryCode")
Log.i(TAG,"load: userSelection.selectionArea = ${userSelection.selectionArea}")
Log.i(TAG,"load: userSelection.selectionSorting = ${userSelection.selectionSorting}")
var req: List<APISimpleStation>?=null
if((userSelection.selectionArea == "Country" )
&&(userSelection.selectionSorting == "Nearest"))
{
do {
if (userSelection.selectionAutoCountry) {
Log.i(TAG, "load: country, nearest, selectionAutoCountry = true, range = $range")
req = apiService.getSimpleStationsCountryNearest(
userid = user.userId,
passfrase = user.passfrase,
country = countryCode,
latitude = deviceLocation.latitude.toString().format(Locale.ROOT, "%s"),
longitude = deviceLocation.longitude.toString()
.format(Locale.ROOT, "%s"),
limit = state.config.pageSize.toString(),
offset = ((page - 1) * state.config.pageSize).toString(),
lastversion = user.currentVersion,
killed = if (userSelection.showHidden) "1" else "0",
range = range ?: "1.0" //may be changed
)
} else {
Log.i(TAG, "load: country, nearest, selectionAutoCountry = false, range = $range")
val countryData = appDatabase.countryDao().getCountryByCode(countryCode)
Log.i(TAG, "load: autoCountry not auto set = ${countryData?.isoCC2}")
req = apiService.getSimpleStationsCountryNearest(
userid = user.userId,
passfrase = user.passfrase,
country = countryCode,
latitude = (countryData?.latitude_center ?: 0.0F).toString()
.format(Locale.ROOT, "%s"),
longitude = (countryData?.longitude_center ?: 0.0F).toString()
.format(Locale.ROOT, "%s"),
limit = state.config.pageSize.toString(),
offset = ((page - 1) * state.config.pageSize).toString(),
lastversion = user.currentVersion,
killed = if (userSelection.showHidden) "1" else "0",
range = range ?: "1.0" //must be changed
)
}
Log.i(TAG, "load: req.size = ${req.size}")
if(req.isEmpty()) range = increaseRange(range)
else break
}while (range != null)
}else if((userSelection.selectionArea == "Global")
&&(userSelection.selectionSorting == "Nearest"))
{
do{
Log.i(TAG, "load: global, nearest, range = $range")
req = apiService.getSimpleStationsGlobalNearest(
userid = user.userId,
passfrase = user.passfrase,
latitude = deviceLocation.latitude.toString().format(Locale.ROOT,"%s"),
longitude = deviceLocation.longitude.toString().format(Locale.ROOT,"%s"),
limit = state.config.pageSize.toString(),
offset = ((page-1)*state.config.pageSize).toString(),
lastversion = user.currentVersion,
killed = if(userSelection.showHidden) "1" else "0",
range = range?:"1.0"
)
Log.i(TAG, "load: req.size = ${req.size}")
if(req.isEmpty()) range = increaseRange(range)
else break
}while (range != null)
}else if((userSelection.selectionArea == "Country")
&&(userSelection.selectionSorting == "Latest"))
{
Log.i(TAG, "load: country, latest")
req = apiService.getSimpleStationsCountryLatest(
userid = user.userId,
passfrase = user.passfrase,
country = countryCode,
limit = state.config.pageSize.toString(),
offset = ((page-1)*state.config.pageSize).toString(),
lastversion = user.currentVersion,
killed = if(userSelection.showHidden) "1" else "0"
)
}else if((userSelection.selectionArea == "Global")
&&(userSelection.selectionSorting == "Latest"))
{
Log.i(TAG, "load: global, latest")
req = apiService.getSimpleStationsGlobalLatest(
userid = user.userId,
passfrase = user.passfrase,
limit = state.config.pageSize.toString(),
offset = ((page-1)*state.config.pageSize).toString(),
lastversion = user.currentVersion,
killed = if(userSelection.showHidden) "1" else "0"
)
}
//TODO: Add rest
endOfPaginationReached = req?.isEmpty()
?:return MediatorResult.Error(Throwable("No data could be read from backend..."))
appDatabase.withTransaction {
if(loadType == LoadType.REFRESH)
{
appDatabase.stationListRemoteKeyDao().clearStationListRemoteKeys()
appDatabase.stationDao().nukeTable() //Nuke station table
}
val prevKey = if(page == STATION_LIST_STARTING_PAGE_INDEX) null else page -1
val nextKey = if(endOfPaginationReached) null else page + 1
val keys = req.map {
StationListRemoteKey(
stationId = it.idsite,
prevKey = prevKey,
nextKey = nextKey)
}
val stations = req.map{
Log.i(
TAG,
"load: station map ${it.idsite}: ${it.stationname}: ${it.latitude}, ${it.longitude} -> ${deviceLocation.latitude}, ${deviceLocation.longitude}"
)
Station(
stationId = it.idsite,
stationName = it.stationname,
longitude = it.longitude.toDoubleOrNull(),
latitude = it.latitude.toDoubleOrNull(),
airDistance = (locationDistance(
it.latitude.toDoubleOrNull(),
it.longitude.toDoubleOrNull(),
deviceLocation.latitude,
deviceLocation.longitude
)?:0.0)/1000.0,
enterpriseId = it.enterpriseid,
googlePlaceId = it.googleplaceid,
stationAddress = it.stationaddress,
stationPlace = it.stationplace,
stationZip = it.stationzip,
stationCountryCode = it.countrycode,
killed = it.killed?.equals("1")?:false,
dateCreated =it.datecreated,
stationCountry = it.stationcountry
)
}
appDatabase.stationListRemoteKeyDao().insertAll(keys)
appDatabase.stationDao().insertAll(stations)
}
return MediatorResult.Success(endOfPaginationReached = endOfPaginationReached)
}catch (exception: IOException)
{
return MediatorResult.Error(exception)
}
catch (exception: HttpException)
{
return MediatorResult.Error(exception)
}
}
fun increaseRange(inRange: String?): String? {
var fRange:Float = inRange?.toFloatOrNull()?:1.0F
fRange += 1.0F
Log.i(TAG, "increaseRange: fRange = $fRange")
val rRange = if(fRange>180.0F) null
else fRange.toString().format(Locale.ROOT,"%0.1f")
CoroutineScope(Dispatchers.IO).launch {
appDatabase.userSelectionDao().updateCurrentRange(rRange)
}
return rRange
}
fun locationDistance(
fromLatitude: Double?,
fromLongitude: Double?,
toLatitude: Double?,
toLongitude : Double?
):Double?{
fromLatitude?:return null
fromLongitude?:return null
toLatitude?:return null
toLongitude?:return null
val fromLocation = Location("FromLocation")
val toLocation = Location("ToLocation")
fromLocation.latitude = fromLatitude
fromLocation.longitude = fromLongitude
toLocation.latitude = toLatitude
toLocation.longitude = toLongitude
return fromLocation.distanceTo(toLocation).toDouble()
}
}
If I have time, I will give you a working proof-of-concept of the issue.
RG
ro...@gmail.com
du...@google.com
I'm not sure I understand the issue you're hitting, but APPEND and PREPEND can happen at the same time. It's possible for a PagingData to PREPEND and APPEND concurrently given sufficient PagingConfig.maxSize.
ro...@gmail.com
That is correct, but when there is a function suspending inside load (i.e.
doing a query in room) it seems that there will be another load (APPEND)
attempt, which again will do the same processing and suspend call). There
may be many such suspending calls upon each other querying the same values.
Until one of the suspended calls returns with results.
RG
fre. 3. jul. 2020, 21:54 skrev <buganizer-system@google.com>:
doing a query in room) it seems that there will be another load (APPEND)
attempt, which again will do the same processing and suspend call). There
may be many such suspending calls upon each other querying the same values.
Until one of the suspended calls returns with results.
RG
fre. 3. jul. 2020, 21:54 skrev <buganizer-system@google.com>:
du...@google.com
Ah I see, so you expect cancellation to propagate there - interesting. Thanks for the report!
du...@google.com
We actually do have something for this, but looks like it's not working correctly. We're on holiday so I'll need to take a look a bit later :)
ro...@gmail.com
Okei, same here. Happy holiday !! RG
du...@google.com
For others hitting this issue - the workaround is to use .collectLatest instead of .collect on your Flow<PagingData<T>>.
to...@gtempaccount.com
Hi there, I am also getting this issue, but the problem for me occured using the RxRemoteMediator.
It seems to be happening when saving remote data to the database. The only solution Could find for now to prevent this, was to put an arbitrary delay between the fetching of the data and the saving of the data to the db like so:
getOffset()
.flatMap { offset -> remote.getItems(offset) }
.delay(200, TimeUnit.MILLISECONDS)
.map { result -> if(result is Success) local.save(result.items)}
If you want to see an example, my pet project is herehttps://github.com/ThomasWilliamMarsh/Gif-Finder-Android . The file in question is "GifsRemoteMediator".
Thanks
It seems to be happening when saving remote data to the database. The only solution Could find for now to prevent this, was to put an arbitrary delay between the fetching of the data and the saving of the data to the db like so:
getOffset()
.flatMap { offset -> remote.getItems(offset) }
.delay(200, TimeUnit.MILLISECONDS)
.map { result -> if(result is Success) local.save(result.items)}
If you want to see an example, my pet project is here
Thanks
du...@google.com
ap...@google.com
Project: platform/frameworks/support
Branch: androidx-master-dev
commit 93e69a2a4078d574f0d27502e7fb2d7882edcd97
Author: Dustin Lam <dustinlam@google.com>
Date: Mon Jul 20 21:20:44 2020
Prevent ClosedSendChannelException when using synchronous submitData
RelNote: "Fixed a bug where using the synchronous variant of submitData
would sometimes lead to a race causing ClosedSendChannelException"
Fixes: 160192222
Test: ./gradlew paging:paging-common:test
Change-Id: I4d70208a6def82099644ec88b087426a1ae0cffd
M paging/common/src/main/kotlin/androidx/paging/PageFetcherSnapshot.kt
M paging/common/src/test/kotlin/androidx/paging/PageFetcherSnapshotTest.kt
M paging/common/src/test/kotlin/androidx/paging/PageFetcherTest.kt
M testutils/testutils-paging/src/main/java/androidx/paging/TestPagingSource.kt
https://android-review.googlesource.com/1367729
Branch: androidx-master-dev
commit 93e69a2a4078d574f0d27502e7fb2d7882edcd97
Author: Dustin Lam <dustinlam@google.com>
Date: Mon Jul 20 21:20:44 2020
Prevent ClosedSendChannelException when using synchronous submitData
RelNote: "Fixed a bug where using the synchronous variant of submitData
would sometimes lead to a race causing ClosedSendChannelException"
Fixes: 160192222
Test: ./gradlew paging:paging-common:test
Change-Id: I4d70208a6def82099644ec88b087426a1ae0cffd
M paging/common/src/main/kotlin/androidx/paging/PageFetcherSnapshot.kt
M paging/common/src/test/kotlin/androidx/paging/PageFetcherSnapshotTest.kt
M paging/common/src/test/kotlin/androidx/paging/PageFetcherTest.kt
M testutils/testutils-paging/src/main/java/androidx/paging/TestPagingSource.kt
de...@gmail.com
I'm also getting the same issue.
ro...@gmail.com
Theres a new alpha04 release where some of the mentioned issues above are mentioned.
RG
du...@google.com
The "Channel was closed" exception should be fixed by the latest release, thanks for being patient!
Description
Component used:
Version used: Devices/Android versions reproduced on:
If this is a bug in the library, we would appreciate if you could attach:
Have migrated from paging 2.1.0 to 3.0.0, but are plagued by the "Channel was closed" exception when other (not paging 3.0 related code) is updating the Room db where the paging data is resided. This update may change the order in the resulting list also.
Is this due to Flow being used at the moment when the background process is updating data ? Room shoud be able to do this update and suspend/postpone/queue it until the Flow has done its updates to the UI.
Here is the errormessage:
RG