InvalidProtocolBufferException during initialization [365889758]

Duplicate

of 346197747

Bug

Status Update

No update yet.

Description

ah...@truecaller.com

created issue #1

Sep 11, 2024 12:40PM

DataStore Component used: Preferences DataStore
DataStore Version used: 1.1.1
Android versions: 11 (21%) 12 (16%) 13 (14%) 14 (33%)
Devices: Vivo (22%) Samsung (21%) Oppo (14%) XIaomi (13%) Other (30%)

Error messages:
Protocol message tag had invalid wire type.
Protocol message contained an invalid tag (zero)
Protocol message end-group tag did not match expected tag.

Stacktrace:
androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.<init> (InvalidProtocolBufferException.java:47)
androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.invalidTag (InvalidProtocolBufferException.java:133)
androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag (CodedInputStream.java:2107)
androidx.datastore.preferences.protobuf.CodedInputStreamReader.getFieldNumber (CodedInputStreamReader.java:82)
androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper (MessageSchema.java:3913)
androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom (MessageSchema.java:3895)
androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom (GeneratedMessageLite.java:1648)
androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom (GeneratedMessageLite.java:1784)
androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom (PreferencesProto.java:196)
androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom (PreferencesMapCompat.java:32)
androidx.datastore.preferences.core.PreferencesSerializer.readFrom (PreferencesSerializer.jvm.kt:46)
androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl (OkioStorage.kt:180)
androidx.datastore.core.okio.OkioReadScope.readData (OkioStorage.kt:1)
androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend (StorageConnection.kt:74)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:20)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:20)
androidx.datastore.core.okio.OkioStorageConnection.readScope (OkioStorage.kt:113)
androidx.datastore.core.StorageConnectionKt.readData (StorageConnection.kt:74)
androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault (DataStoreImpl.kt:331)
androidx.datastore.core.DataStoreImpl.readDataOrHandleCorruption (DataStoreImpl.kt:373)
androidx.datastore.core.DataStoreImpl.access$readDataOrHandleCorruption (DataStoreImpl.kt:53)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend (DataStoreImpl.kt:445)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:11)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:11)
androidx.datastore.core.SingleProcessCoordinator.lock (SingleProcessCoordinator.kt:41)
androidx.datastore.core.DataStoreImpl$InitDataStore.doRun (DataStoreImpl.kt:442)
androidx.datastore.core.RunOnce.runIfNeeded (DataStoreImpl.kt:505)
androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:274)
androidx.datastore.core.DataStoreImpl.handleUpdate (DataStoreImpl.kt:251)
androidx.datastore.core.DataStoreImpl.access$handleUpdate (DataStoreImpl.kt:53)
androidx.datastore.core.DataStoreImpl$writeActor$3.invokeSuspend (DataStoreImpl.kt:215)
androidx.datastore.core.DataStoreImpl$writeActor$3.invoke (DataStoreImpl.kt:13)
androidx.datastore.core.DataStoreImpl$writeActor$3.invoke (DataStoreImpl.kt:13)
androidx.datastore.core.SimpleActor$offer$2.invokeSuspend (SimpleActor.kt:121)

First of all, it's not clear why preferences are corrupted. However, regardless of that, the corruption handler does not seem to kick in. The expected path would be that the preferences are wiped and recreated. That does not seem to happen. If there was in issue during the recreation, we would expect the surpressed stacktrace to appear according to the code here:

https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:datastore/datastore-core/src/commonMain/kotlin/androidx/datastore/core/DataStoreImpl.kt;l=395

Comments

zh...@google.com <zh...@google.com> #2Sep 11, 2024 05:53PM

This is probably happening because there are multiple instances of DataStore active. You should consider managing the DataStore as a singleton. See the attached bug for more info.

ah...@truecaller.com <ah...@truecaller.com> #3Sep 11, 2024 06:35PM

Yes, this is our configuration:

PreferenceDataStoreFactory.create(
            scope = CoroutineScope(coroutineContext + SupervisorJob()),
            migrations = migrations,
            corruptionHandler = ReplaceFileCorruptionHandler(
                produceNewData = { emptyPreferences() } // Use an empty preferences in case of file corruption
            )
        ) {
            applicationContext.preferencesDataStoreFile(name)
        }

zh...@google.com <zh...@google.com> #4Sep 11, 2024 08:13PM

Accepted by zh...@google.com.

Thanks for the inputs, I'll take a deeper look at it.

ah...@truecaller.com <ah...@truecaller.com> #5Sep 12, 2024 11:01AM

It looks like we found the culprit. It was a third party library that didn't provide a corruption handler. We have two improvement suggestions:

Make the corruption handler mandatory or provide a default value that recreates an empty file. Otherwise any kind of corruption results in unrecoverable scenario.
Add data store file name to logs. Otherwise it's impossible to figure out the culprit from the stack trace. It would be helpful to have more details in the stacktrace.

ni...@gmail.com <ni...@gmail.com> #6Sep 16, 2024 12:36PM

I am facing the similar issue. I don't have enough stacktrace in firebase, #5 which library was causing it?

ah...@truecaller.com <ah...@truecaller.com> #7Sep 17, 2024 08:35AM

Unfortunately, I cannot share the name of the SDK, but here are a couple of hints to narrow down the problem:

Add logs around the creation of your data store to make sure it's not your code. Ensure you've implemented corruption handlers.
Go through recent library updates that you've made.

zh...@google.com <zh...@google.com> #8Sep 17, 2024 07:05PM

Thanks for your inputs, re #comment5,

Make the corruption handler mandatory or provide a default value that recreates an empty file. Otherwise any kind of corruption results in unrecoverable scenario.

This would be an API change and needs careful reviews. I'll discuss with other owners of this project to see if it's possible.

Add data store file name to logs. Otherwise it's impossible to figure out the culprit from the stack trace. It would be helpful to have more details in the stacktrace.

Right now the place where the exception is thrown has no access to the file name information, I'll try to plumb the info through if there's no API reviews needed.

For the trace stack you provided in #1, is it suppressed in a CorruptionException or thrown directly?

ah...@gmail.com <ah...@gmail.com> #9Sep 18, 2024 07:31AM

Comment has been deleted.

Message last modified on Sep 18, 2024 07:31AM

ah...@truecaller.com <ah...@truecaller.com> #10Sep 18, 2024 07:32AM

For the trace stack you provided in #1, is it suppressed in a CorruptionException or thrown directly?

It's suppressed in CorruptionException.

zh...@google.com <zh...@google.com> #11Sep 18, 2024 06:47PM

Thanks for the contexts in #comment10, that explains the source of this exception. I'll follow up for the feature requests you mentioned in #comment7.

ah...@truecaller.com <ah...@truecaller.com> #12Sep 19, 2024 08:53AM

Awesome, thank you for taking a look so promptly!

el...@google.com <el...@google.com> #13Feb 20, 2025 05:20PM

Status: Duplicate of

346197747

Hi there, we have multiple issues open for this same issue, so to consolidate the info on 1 issue tracker, I will be closing this issue "as duplicate". Here is the public tracker for the main issue.