DataSource.Factory with map/mapByPage does not invalidate source in Paging 3 [165313046]

Fixed

Bug

Status Update

No update yet.

Description

ti...@chainels.com

created issue #1

Aug 18, 2020 07:13PM

Component used: Paging
Version used: 3.0 alpha-04
Devices/Android versions reproduced on: n/a

Today (after many hours of debugging) I seem to have found a bug in latest alpha version of the Paging library. In my app I still have various lists that are not yet migrated to the new v3 API's, but since the documentation states it's backward compatible I figured that was okay.

Those lists use the database + remote pattern. One such list displayed strange behavior, it seemed to call all the boundary callbacks correctly, do the API call, and insert the new items in the database, however the items in the RecyclerView did not change.

The only difference that I could find between that list and other lists that were functioning correctly, is that it used a map function on the Datasource.Factory returned by Room

I think I found the underlying issue. The data source does not seem to be invalidated when using a map function.

DataSource has two variants of addOnValidationCallback. An open function DataSource.kt:366 and a final function that accepts Kotlin functions DataSource.kt:385

The Kotlin version is actually called in LegacyPagingSource

However when using a map/mapByPage function on the factory returned by room, the source is wrapped and in that class (WrappedDataSource, but also WrapperPositionalDataSource for example) only the non Kotlin version of addOnValidationCallback is overridden, and thus if called with the Kotlin version, addOnValidationCallback is never called on the source of the wrapper.

I could see when debugging that indeed onInvalidatedCallbacks is empty.

Hopefully this is actually the cause of the behavior I'm seeing. The Paging code is quite complex so I could be missing something else.

Comments

du...@google.com <du...@google.com> #2Aug 19, 2020 05:12PM

A couple of questions:

1. Have you saw crash in real device or only in simulators?
2. Do you use dynamic feature for language ID?

du...@google.com <du...@google.com> #3Sep 4, 2020 03:28AM

Accepted by du...@google.com.

Tested on Android 12 Emulator with custom executor, but cannot repro this issue.

ti...@chainels.com <ti...@chainels.com> #4Sep 4, 2020 12:25PM

Second crash in the description is from a real device. Experienced it myself on two different Xiaomi phones, plus lots of crashes from users in the Google Play console.
Dynamic features are not used in the application.

As a wild guess, I have downgraded build tools from 31.0.0 to 30.0.3, compileSdk from 31 to 30, and moved all work with Language ID to the service in a separate process (just to be sure that crash can kill secondary process instead of main). This combination is in beta for 2 days by now and I don't see any SIGSEGV crashes.

du...@google.com <du...@google.com> #5Sep 5, 2020 06:32PM

Hmm, I feel the crash might be something related to separate/secondary process.

I also changed compileSdk and targetSDK to 31 but still cannot repro this issue.

du...@google.com <du...@google.com> #6Sep 23, 2020 04:18AM

On the contrary, there was no separate process before, when crashes started.

In the new build (with the aforementioned changes) I can see SIGSEGV crash, but only one instead of dozens and it has a bit different backtrace:

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
liblanguage_id_jni.so (offset 0x11e000)

backtrace:
  #00  pc 000000000003c7c0  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 000000000003b960  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 000000000003bb48  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 000000000003bafc  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 0000000000036c98  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 0000000000032714  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 0000000000031cac  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/split_config.arm64_v8a.apk!lib/arm64-v8a/liblanguage_id_jni.so (offset 0x11e000)
  #00  pc 0000000000057438  /data/app/azagroup.reedy-mF7zTu2bv_ELlbFArwNgqA==/oat/arm64/base.odex (offset 0x57000)

ap...@google.com <ap...@google.com> #7Oct 13, 2020 04:11PM

Marked as fixed.

FYI, ML Kit launched a new language ID SDK in the latest release, which uses a new language ID model. https://developers.google.com/ml-kit/release-notes#august_11_2021

Could you try the new SDK version(17.0.0) to check if you can still repro this native crash? Thanks!

Issue 165313046