IssueTracker

Update on this - we are continuing to see it affect a relatively small number of users but it is affecting those users quite frequently.

We have 12 users now who have collectively seen this crash 2,500 times. There don't appear to be many commonalities - these 12 users are spread across a few different devices on Android 10, 11, and 12.

sorry for the very late response. But without getting more details, it is very hard for us to diagnose this.

Are you maybe accessing the same file across multiple processes? If yes, please use the multi-process datastore in version 1.1

If no, can you provide any more details on how we can try to reproduce this?

Thanks for following up and reminding me to come back and follow up too!

We tackled this on two fronts at once - First, while we couldn't reproduce this issue locally, we did notice logs indicating there were multiple datastores open at once:

java.lang.IllegalStateException: There are multiple DataStores active for the same file: /data/user/0/com.our.app/files/datastore/medallia.preferences_pb We also noted this issue calling out similar exceptions with multiple datastores open at once

So part one was ensuring that our datastores were being created as singletons.

At the same time, we configured a corruption handler that replaced the datastore with an empty one.

I suspect the first piece played a larger part in resolving this issue - we added logging that should let us know if we are hitting the corruption handler and I'm not seeing any recent hits there. In either case, we aren't seeing this issue now.

Thanks for the update: Maybe you were hitting b/273885132 ?

I suspect so!

Thanks for the update.

I'll close this as not reproducible and created a fix for b/273885132

https://android-review.googlesource.com/c/platform/frameworks/support/+/2729273

This crash still happens in version 1.1.0-alpha05. We haven't updated to 1.1.0-alpha06 yet, so if nothing related to this crash has changed in 1.1.0-alpha06, it is likely happening for this version as well.

The stack trace seems to be slightly longer this time:

androidx.datastore.preferences.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
       at androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag(CodedInputStream.java:2116)
       at androidx.datastore.preferences.protobuf.CodedInputStreamReader.getFieldNumber(CodedInputStreamReader.java:82)
       at androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper(MessageSchema.java:3913)
       at androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom(MessageSchema.java:3895)
       at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom(GeneratedMessageLite.java:1647)
       at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom(GeneratedMessageLite.java:1783)
       at androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom(PreferencesProto.java:194)
       at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.kt:32)
       at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.jvm.kt:46)
       at androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl(OkioStorage.kt:180)
       at androidx.datastore.core.okio.OkioReadScope.readData(OkioStorage.kt:1)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend(StorageConnection.kt:74)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:10)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:11)
       at androidx.datastore.core.okio.OkioStorageConnection.readScope(OkioStorage.kt:113)
       at androidx.datastore.core.StorageConnectionKt.readData(StorageConnection.kt:74)
       at androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault(DataStoreImpl.kt:289)
       at androidx.datastore.core.DataStoreImpl.access$readDataFromFileOrDefault(DataStoreImpl.kt:46)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.readDataOrHandleCorruption(DataStoreImpl.kt:414)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.access$readDataOrHandleCorruption(DataStoreImpl.kt:329)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend(DataStoreImpl.kt:346)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:1)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:2)
       at androidx.datastore.core.SingleProcessCoordinator.lock(SingleProcessCoordinator.kt:38)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.doRun(DataStoreImpl.kt:343)
       at androidx.datastore.core.RunOnce.runIfNeeded(DataStoreImpl.kt:474)
       at androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure(DataStoreImpl.kt:218)
       at androidx.datastore.core.DataStoreImpl.handleUpdate(DataStoreImpl.kt:195)
       at androidx.datastore.core.DataStoreImpl.access$handleUpdate(DataStoreImpl.kt:46)
       at androidx.datastore.core.DataStoreImpl$writeActor$3.invokeSuspend(DataStoreImpl.kt:159)
       at androidx.datastore.core.DataStoreImpl$writeActor$3.invoke(DataStoreImpl.kt:9)
       at androidx.datastore.core.DataStoreImpl$writeActor$3.invoke(DataStoreImpl.kt:5)
       at androidx.datastore.core.SimpleActor$offer$2.invokeSuspend(SimpleActor.kt:121)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
       at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
       at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:100)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

    androidx.datastore.core.CorruptionException: Unable to parse preferences proto.
       at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.kt:34)
       at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.jvm.kt:46)
       at androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl(OkioStorage.kt:180)
       at androidx.datastore.core.okio.OkioReadScope.readData(OkioStorage.kt:1)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend(StorageConnection.kt:74)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:10)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:11)
       at androidx.datastore.core.okio.OkioStorageConnection.readScope(OkioStorage.kt:113)
       at androidx.datastore.core.StorageConnectionKt.readData(StorageConnection.kt:74)
       at androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault(DataStoreImpl.kt:289)
       at androidx.datastore.core.DataStoreImpl.access$readDataFromFileOrDefault(DataStoreImpl.kt:46)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.readDataOrHandleCorruption(DataStoreImpl.kt:414)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.access$readDataOrHandleCorruption(DataStoreImpl.kt:329)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend(DataStoreImpl.kt:346)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:1)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:2)
       at androidx.datastore.core.SingleProcessCoordinator.lock(SingleProcessCoordinator.kt:38)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.doRun(DataStoreImpl.kt:343)
       at androidx.datastore.core.RunOnce.runIfNeeded(DataStoreImpl.kt:474)
       at androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure(DataStoreImpl.kt:218)
       at androidx.datastore.core.DataStoreImpl.handleUpdate(DataStoreImpl.kt:195)
       at androidx.datastore.core.DataStoreImpl.access$handleUpdate(DataStoreImpl.kt:46)
       at androidx.datastore.core.DataStoreImpl$writeActor$3.invokeSuspend(DataStoreImpl.kt:159)
       at androidx.datastore.core.DataStoreImpl$writeActor$3.invoke(DataStoreImpl.kt:9)
       at androidx.datastore.core.DataStoreImpl$writeActor$3.invoke(DataStoreImpl.kt:5)
       at androidx.datastore.core.SimpleActor$offer$2.invokeSuspend(SimpleActor.kt:121)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
       at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
       at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:100)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

It still happens in version 1.1.0-alpha06

androidx.datastore.core.CorruptionException: Unable to parse preferences proto.
	at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.java:0)
	at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.java:0)
	at androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl(OkioReadScope.java:0)
	at androidx.datastore.core.okio.OkioReadScope.readData(OkioReadScope.java:0)
	at androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend(StorageConnectionKt.java:28)
	at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnectionKt.java:0)
	at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnectionKt.java:0)
	at androidx.datastore.core.okio.OkioStorageConnection.readScope(OkioStorageConnection.java:0)
	at androidx.datastore.core.StorageConnectionKt.readData(StorageConnectionKt.java:0)
	at androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl.access$readDataFromFileOrDefault(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl$InitDataStore.readDataOrHandleCorruption(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl$InitDataStore.access$readDataOrHandleCorruption(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.java:0)
	at androidx.datastore.core.SingleProcessCoordinator.lock(SingleProcessCoordinator.java:93)
	at androidx.datastore.core.DataStoreImpl$InitDataStore.doRun(DataStoreImpl.java:0)
	at androidx.datastore.core.RunOnce.runIfNeeded(RunOnce.java:107)
	at androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure(DataStoreImpl.java:95)
	at androidx.datastore.core.DataStoreImpl.access$readAndInitOrPropagateAndThrowFailure(DataStoreImpl.java:0)
	at androidx.datastore.core.DataStoreImpl$readState$2.invokeSuspend(DataStoreImpl.java:0)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:8)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:105)
	at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.java:4)
	at kotlinx.coroutines.scheduling.TaskImpl.run(TaskImpl.java:2)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:0)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:0)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:0)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:0)
	Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [d2{Cancelling}@c5c2351, Dispatchers.Default]
Caused by: androidx.datastore.preferences.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
	at androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:0)
	at androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag(CodedInputStream.java:0)
	at androidx.datastore.preferences.protobuf.CodedInputStreamReader.getFieldNumber(CodedInputStreamReader.java:12)
	at androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper(MessageSchema.java:0)
	at androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom(MessageSchema.java:0)
	at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom(GeneratedMessageLite.java:0)
	at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom(GeneratedMessageLite.java:0)
	at androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom(PreferencesProto.java:0)
	at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.java:0)
	at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.java:0)

are you able to reproduce this or is this only from field metrics?

I encountered multiple instances of the same crash as #13 but only from field metrics.

any chance this is happening (on the field) when the device is not yet unlocked (e.g. your process runs on boot completed and may not have access to disk yet?)

Sorry for the late response. The process is started in a BOOT_COMPLETED receiver or it can be started when a ACTION_OPEN_AUDIO_EFFECT_CONTROL_SESSION is received. A LOCKED_BOOT_COMPLETED receiver is not used and android:directBootAware is not set (defaults to false).

Can't reproduce the issue, but also getting crashes from users in production with version androidx.datastore:datastore-preferences:1.1.1. 54% of the crashes happen with the app in the background.

Stacktrace:

Fatal Exception: L1.b: Unable to parse preferences proto.
       at androidx.datastore.core.CorruptionException.<init>(CorruptionException.java:293)
       at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.java:293)
       at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.jvm.kt:293)
       at androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl(OkioStorage.kt:89)
       at androidx.datastore.core.okio.OkioReadScope.readData(OkioReadScope.java:97)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend(StorageConnection.kt:97)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:47)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:47)
       at androidx.datastore.core.okio.OkioStorageConnection.readScope(OkioStorage.kt:100)
       at androidx.datastore.core.StorageConnectionKt.readData(StorageConnection.kt:16)
       at androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault(DataStoreImpl.kt:16)
       at androidx.datastore.core.DataStoreImpl.readDataOrHandleCorruption(DataStoreImpl.kt:153)
       at androidx.datastore.core.DataStoreImpl.access$readDataOrHandleCorruption(DataStoreImpl.kt:153)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend(DataStoreImpl.kt:127)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:14)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:14)
       at androidx.datastore.core.SingleProcessCoordinator.lock(SingleProcessCoordinator.kt:94)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.doRun(DataStoreImpl.java:87)
       at androidx.datastore.core.RunOnce.runIfNeeded(RunOnce.java:129)
       at androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure(DataStoreImpl.kt:96)
       at androidx.datastore.core.DataStoreImpl.access$readAndInitOrPropagateAndThrowFailure(DataStoreImpl.kt:55)
       at androidx.datastore.core.DataStoreImpl$readState$2.invokeSuspend(DataStoreImpl.kt:55)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:9)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:95)
       at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:3)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:92)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:92)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:92)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:92)

Caused by androidx.datastore.preferences.protobuf.B: Protocol message contained an invalid tag (zero).
       at androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.<init>(InvalidProtocolBufferException.java:26)
       at androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:26)
       at androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag(CodedInputStream.java:26)
       at androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper(MessageSchema.java:28)
       at androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom(MessageSchema.java:28)
       at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom(GeneratedMessageLite.java:41)
       at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom(GeneratedMessageLite.java:41)
       at androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom(PreferencesProto.java:41)
       at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.java:6)
       at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.jvm.kt:6)
       at androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl(OkioStorage.kt:89)
       at androidx.datastore.core.okio.OkioReadScope.readData(OkioReadScope.java:97)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend(StorageConnection.kt:97)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:47)
       at androidx.datastore.core.StorageConnectionKt$readData$2.invoke(StorageConnection.kt:47)
       at androidx.datastore.core.okio.OkioStorageConnection.readScope(OkioStorage.kt:100)
       at androidx.datastore.core.StorageConnectionKt.readData(StorageConnection.kt:16)
       at androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault(DataStoreImpl.kt:16)
       at androidx.datastore.core.DataStoreImpl.readDataOrHandleCorruption(DataStoreImpl.kt:153)
       at androidx.datastore.core.DataStoreImpl.access$readDataOrHandleCorruption(DataStoreImpl.kt:153)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend(DataStoreImpl.kt:127)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:14)
       at androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke(DataStoreImpl.kt:14)
       at androidx.datastore.core.SingleProcessCoordinator.lock(SingleProcessCoordinator.kt:94)
       at androidx.datastore.core.DataStoreImpl$InitDataStore.doRun(DataStoreImpl.java:87)
       at androidx.datastore.core.RunOnce.runIfNeeded(RunOnce.java:129)
       at androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure(DataStoreImpl.kt:96)
       at androidx.datastore.core.DataStoreImpl.access$readAndInitOrPropagateAndThrowFailure(DataStoreImpl.kt:55)
       at androidx.datastore.core.DataStoreImpl$readState$2.invokeSuspend(DataStoreImpl.kt:55)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:9)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:95)
       at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:3)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:92)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:92)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:92)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:92)

#19, boot completed case is expected as your app won't have access to the file before device is unlocked.

we don't have an easy way to wait for boot completed within the library so it is something you need to handle in the app for now (don't initialize datastore before you have access to the filesystem)

Same here. 99% app in background. Version 1.1.1

   Caused by androidx.datastore.preferences.protobuf.z: Protocol message contained an invalid tag (zero).
   at androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.<init>(InvalidProtocolBufferException.java:47)
   at androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:133)
   at androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag(CodedInputStream.java:2107)
   at androidx.datastore.preferences.protobuf.CodedInputStreamReader.getFieldNumber(CodedInputStreamReader.java:82)
   at androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper(MessageSchema.java:3913)
   at androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom(MessageSchema.java:3895)
   at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom(GeneratedMessageLite.java:1648)
   at androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom(GeneratedMessageLite.java:1784)
   at androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom(PreferencesProto.java:196)
   at androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom(PreferencesMapCompat.java:32)
   at androidx.datastore.preferences.core.PreferencesSerializer.readFrom(PreferencesSerializer.jvm.kt:46)

I downgraded to version 1.0.0 and the number of crashed declined to round 5%.

The issue is happening to us on version 1.1.1. We're using a singleton DataStore via Hilt - and our app is single process. The weird thing is the users affected have 50+ crashes in a relatively small timeframe (less than a couple of minutes).

Caused by androidx.datastore.preferences.protobuf.H
Protocol message contained an invalid tag (zero).
androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.<init> (InvalidProtocolBufferException.java:47)
androidx.datastore.preferences.protobuf.InvalidProtocolBufferException.invalidTag (InvalidProtocolBufferException.java:133)
androidx.datastore.preferences.protobuf.CodedInputStream$StreamDecoder.readTag (CodedInputStream.java:2107)
androidx.datastore.preferences.protobuf.CodedInputStreamReader.getFieldNumber (CodedInputStreamReader.java:82)
androidx.datastore.preferences.protobuf.MessageSchema.mergeFromHelper (MessageSchema.java:3913)
androidx.datastore.preferences.protobuf.MessageSchema.mergeFrom (MessageSchema.java:3895)
androidx.datastore.preferences.protobuf.GeneratedMessageLite.parsePartialFrom (GeneratedMessageLite.java:1648)
androidx.datastore.preferences.protobuf.GeneratedMessageLite.parseFrom (GeneratedMessageLite.java:1784)
androidx.datastore.preferences.PreferencesProto$PreferenceMap.parseFrom (PreferencesProto.java:196)
androidx.datastore.preferences.PreferencesMapCompat$Companion.readFrom (PreferencesMapCompat.java:32)
androidx.datastore.preferences.core.PreferencesSerializer.readFrom (PreferencesSerializer.jvm.kt:46)
androidx.datastore.core.okio.OkioReadScope.readData$suspendImpl (OkioStorage.kt:180)
androidx.datastore.core.okio.OkioReadScope.readData (OkioReadScope.java:31)
androidx.datastore.core.StorageConnectionKt$readData$2.invokeSuspend (StorageConnection.kt:74)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:19)
androidx.datastore.core.StorageConnectionKt$readData$2.invoke (StorageConnection.kt:19)
androidx.datastore.core.okio.OkioStorageConnection.readScope (OkioStorage.kt:113)
androidx.datastore.core.StorageConnectionKt.readData (StorageConnection.kt:74)
androidx.datastore.core.DataStoreImpl.readDataFromFileOrDefault (DataStoreImpl.kt:331)
androidx.datastore.core.DataStoreImpl.readDataOrHandleCorruption (DataStoreImpl.kt:373)
androidx.datastore.core.DataStoreImpl.access$readDataOrHandleCorruption (DataStoreImpl.kt:53)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invokeSuspend (DataStoreImpl.kt:445)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:13)
androidx.datastore.core.DataStoreImpl$InitDataStore$doRun$initData$1.invoke (DataStoreImpl.kt:13)
androidx.datastore.core.SingleProcessCoordinator.lock (SingleProcessCoordinator.kt:41)
androidx.datastore.core.DataStoreImpl$InitDataStore.doRun (DataStoreImpl.java:442)
androidx.datastore.core.RunOnce.runIfNeeded (RunOnce.java:505)
androidx.datastore.core.DataStoreImpl.readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:274)
androidx.datastore.core.DataStoreImpl.access$readAndInitOrPropagateAndThrowFailure (DataStoreImpl.kt:53)
androidx.datastore.core.DataStoreImpl$readState$2.invokeSuspend (DataStoreImpl.kt:226)
kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith (ContinuationImpl.kt:33)
kotlinx.coroutines.DispatchedTask.run (DispatchedTask.kt:104)
kotlinx.coroutines.internal.LimitedDispatcher$Worker.run (LimitedDispatcher.java:111)
kotlinx.coroutines.scheduling.TaskImpl.run (Tasks.kt:99)
kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely (CoroutineScheduler.java:585)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask (CoroutineScheduler.kt:802)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker (CoroutineScheduler.kt:706)
kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run (CoroutineScheduler.kt:693)

I have also been trying to get ridge of this error, and it does not seem to budge. Listening for BOOT_COMPLETED also seem useless because this always happens in the Worker and in the background.

Might consider going back to SharedPreferences 🤔

if it is BOOT_COMPLETED, that is likely about the direct boot support, which we are working on to address.

b/334128266

I added more logging during my investigation.

Like this line: Timber.i("Uptime: ${SystemClock.uptimeMillis()} ms") which printes something like Uptime: 4821851 ms. Which is just over an hour of runtime. Since this value only gives runtime without deep sleep, could this be an issue after returning from standby? Or is this what you mean by "direct boot"?

direct boot is the time frame where device started but user has not unlocked it yet. most apps are not available there but certain apps (like alarms) need to work.

This could be related to standby but still hard to guess from that. do you have more logs you can share? e.g. does it read the file a couple of times successfully then crash or does it crash at first read?

Ok, I do not think direct boot related to the issue I'm seeing, since it only occur in the Android TV. The issue is marked as a crash in Crashlytics, but I'm not even sure the app crashed (see the log file). And it seems like the log is just spammed with these messages (attached to log messages probably from the same device?).

Which is also is in line with what crashlytics says: This issue has 2,446 crash events affecting 8 users

But I can add more log info, but what kind of information are you looking for?

Most of my issues occur on "Chromecast Google TV (HD)" with Android 12. But maybe it is because it "identify" it self as a Phone 🤪

by any chance, are you able to pull a corrupted file out of a client? might be too much to ask but i thought i would ask :) if we could see the corrupted file after this exception happens, it might give ideas on what is happening.

I can give it a go, but can you provide some code to guide me in the right direction? Reading the file from correct path, permissions and so.

this might change based on how you intialize it but by default it should be here

File(applicationContext.filesDir, "datastore/$fileName")

where fileName is "$name.preferences_pb" (so whatever name you have for your preferences with .preferences_pb extension).

since this is your app's folder, you shouldn't need any additional permissions to read it. (assuming the device is unlocked, but if it is locked, that would mean the direct boot issue already).

Thanks!

Alright, the coming release contains code to extract the file. A bit of a gamble to find the correct DataStore because the crash happens inside the package and seems to be consumed by the corruption handler (I think). 🤞

I do not think it is because of direct boot. The following code gives "Encryption Unsupported". But that is also logged in crashlytics now, so lets see in the actual crash.

val status = (getSystemService(Context.DEVICE_POLICY_SERVICE) as DevicePolicyManager).getStorageEncryptionStatus()
when(status) {
    DevicePolicyManager.ENCRYPTION_STATUS_* -> "Encryption *"
}

wow thats great, thank you! looking forward to it.

Ok, seems like there are different failures. I'll just list my findings.

getStorageEncryptionStatus() = 5, which is Encryption Active Per User. This is only logged in the beginning in Crashlytics, but uncertain if that would change during the work of the CoroutineWorkers short lifespan.

Exception: Failed to preload data from datastore prefs. androidx.datastore.core.CorruptionException: Unable to parse preferences.

The prefs-file read out bytes to Base64 encoded (with URL safe) is

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==

I was expecting something like

ChoKFExBU1RfV0FUQ0hFRF9DSEFOTkVMEgIYAQodChJMQVNUX1NZTkNfRVBHX0RBVEUSByCn543c uTIKIQoZUFJFRkVSRU5DRV9BVURJT19MQU5HVUFHRRIEKgJlbgoPCglBR0VfTElNSVQSAhgA

Code to print file

Timber.i(context.preferencesDataStoreFile(pref).readBytes().toBase64())

fun ByteArray.toBase64(): String = String(Base64.encode(this, Base64.URL_SAFE))

Which as you indicate the disk is in fact encrypted. The Worker is use to sync EPG to a SQL lite database, and that does not fail.

There is also failure from another datastore, basically the same

Failed to preload data from datastore user_app_data. androidx.datastore.core.CorruptionException: Unable to parse preferences proto.,

and disk storage is getStorageEncryptionStatus() = 5 = Encryption Active Per User.

so the file is definitely encrypted :/. So the context doesn't seem to have access to the file.

Any chance you have access to the device protected storage state of the Context?

https://developer.android.com/reference/android/content/Context#isDeviceProtectedStorage()

No, but I can add to next release (give it another week then).

Anything else I should add while I'm at it?

actually if you can log that information when you first create the DataStore (first time it gets accessed) and when you later try to collect the log. (log the file contents before you open it, along with the device protected state). If the data is already corrupted at the beginning, we cannot gain more information. (you are probably better of just resetting it in that case, like delete the file before opening datastore)

But if we can get a case where data was not corrupted at open, then corrupted later; we might be able to understand more from the logs in between those two events.

i'm also pinging the team to see if there is anything else they think can help.

Ok, I thought the CorruptionHandler took care of file handling on corruption. But do you think it would be best to explicitly delete it in the CorruptionHandler on errors? If the disk encryption is active, then it would probably be best just to abort!?

How do I check if the datastore is created? Check if the file exists or not?

(I use private val Context.dataStore by preferencesDataStore(..), since this is same method when creating and reading the file. )

yea i think you are right on the CoorruptionHandler.

Here is an alternative to also try, using the FileStorage instead of OkioStorage.

Unfortunately, we didn't have the FileSerializer public in 1.1.1 so you cannot use that. It will be in 1.2 (change)

But I'm attaching 2 files: JvmSerializer -> copies the internal serializer (you can also get this by using a snapshot build from androidx.dev) FilePreferences -> creates the delegate that is similar to preferencesDataStore but uses FileStorage instead of OkioStorage.

If you have some AB testing infra, it would be great to use that version as well to see if that works well (would also at least fix your app). these two implementations are compatible with eachother so it should be safe to swap them on the same device.

Now, going back to the reading file before you load.

You cannot if you use the delegate, but you can use a similar delegate like the one i attached (but use OkioStorage, which be believe is potentially the problem here)

For that, I'm attaching OkioInitialValuePreferences.kt which creates the same delegate but stashes the bytes of the file when it is created for the first time. When you detect the error, you should be able to find the contents of it and attach with the error.

I've not tried these btw but they should mostly be correct or at least give you enough pointers.

Thank you very much btw, we really appreciate this help.

I had to attend other matters, so the code change you suggested will not be worked on until next year.

However, result of isDeviceProtectedStorage() is false, but it could have been when the store was created - the log does not tell.

Had some time to test code for a beta release and discovered context.dataStore.edit throws Exceptions not picked up by the corruption handler...

So I'll add the following code in a upcoming beta release.

    private fun edit(block: suspend (MutablePreferences) -> Unit) =
        launch(dispatcherProvider.io()) {
            context.dataStore.runCatching {
                edit { block(it) }
            }.onFailure { e ->
                Timber.w("Failed to edit datastore ${name}. ${e::class.simpleName}(${e.message})")
                replacePreferenceFile()
            }
        }

    private fun replacePreferenceFile() = runCatching {
        Timber.i(if (context.isDeviceProtectedStorage) "Device storage protected" else "Device storage not protected")
        context.preferencesDataStoreFile(name).apply {
            delete()
        }.createNewFile()
    }.onSuccess {
        Timber.i("Recreated corrupted file for datastore ${name}.")
    }.onFailure { e ->
        Timber.e(e, "Failed to delete corrupted file for ${name}.")
    }

Managed to reproduce the exception locally by writing garbage to the preference file before accessing the datastore

        suspend fun generatePreferenceFile(context: Context, name: String) = withContext(Dispatchers.Default) {
            try {
                context.preferencesDataStoreFile(name).writeBytes(Base64.decode("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==", Base64.URL_SAFE))
            } catch (e: Exception) {
                Timber.w("Failed to write preference file for ${name}. ${e.message}")
                null
            }
        }

Corruption exception is caught only during the first read init

When we write we don't read it back after the write. Any exception thrown during the write (e.g. the block throwing etc) will be dispatched back to the code where updateData was called.

So exceptions throwing during a write are not considered corruption since they would end up not writing to the file.

Yes, that makes sense, but the corruption handler failed to replace the preferences/file during the initial read, so I'm left with doing this during the write process.

And why does the corruption handler not handle writes, would have been easier to avoid these mistakes - and not forcing developers to read the code. I'm puzzled how complex this API is and how much is left for the developer to handle for them self.

Crash has now disappeared after the file is deleted/recreated. So I think the corruption handler fails somehow in the initialisation phase.

Hi there, we have multiple issues open for this same issue, so to consolidate the info on 1 issue tracker, I will be closing this issue "as duplicate". Here is the public tracker for the main issue.