IssueTracker

Can you show me the code you are using to build/enqueue your work?  This is definitely surprising - I wonder if you have an incredibly deep hierarchy, or if there's some kind of circular dependency (which we should be catching).

Unfortunately, the stack trace isn't telling us *which* work is causing the problem. My best guess is that it's this one, as it's the newly introduced worker on the build that is crashing, and other builds that don't include it aren't reporting the same crash:

fun enqueueWork(objects: List<MyType>) {
        val inputData = Data.Builder().putString(MY_EXTRA, objects.serializeToJson())
        val request = OneTimeWorkRequest.Builder(WORKER_CLASS)
                .setConstraints(Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build())
                .setInputData(inputData.build())
                .addTag(WORK_TAG)
                .build()

        val workManager = WorkManager.getInstance()
        workManager.beginUniqueWork(WORK_ID, ExistingWorkPolicy.APPEND, request).enqueue()
}

The enqueueWork method is being called at the end of an RxJava chain that's using a buffer that's flushed every 5 minutes, or when an action is called.

How often are those actions being called?  Any idea how big this chain is?

The manual flushing is called when the user takes a particular UX action or closes a fragment that has attached to the object generator. In typical usage, this might happen a couple times in the first few minutes, but then would be unlikely to be called again.

Here is the chain:

val timer = Observable.interval(5, TimeUnit.MINUTES, scheduler)
eventSubject
        .buffer( // Flush buffer on timer, or when signal is called
                    Observable.merge(
                            timer.map { Unit }, // We don't care what types are emitted, we just care that a value came out
                            flushObservable.map { Unit }
                    ).map { // Debug log }
            )
            .filter {
                it.isNotEmpty()
            }
            .observeOn(scheduler)
            .subscribe(
                    { events ->
                        enqueueWork(events)
                    },
                    { // Log error })

Gg

Fixed in https://android-review.googlesource.com/c/platform/frameworks/support/+/932127

Thanks for addressing that so quickly. Do you know approximately how long it will take for it to make it into a release? And is it at all possible that this will be incorporated into the non-Android-x version of WorkManager? It's a little frustrating that two days after 1.0.0 was released, it was announced that there would be no more development on the non-Android-x version.

This will probably be available next week, and so far the plan is to port it to the 1.x branch as well.

That being said, please be aware that all active Jetpack and Support Library development is happening in the AndroidX branches, so I would highly recommend switching your app when you get a chance.  Thanks!

Thanks for the info. We're definitely planning on moving to AndroidX, but until we do, I appreciate that you're porting fixes to the old version.

Thanks for looking into this more.
I've attached the logs I captured when reproducing the problem as follows:

1. Create a brand new Nexus 7 (2012) API 22 emulator to make sure the system does not have any old state that affects behavior, and start it, and let it boot up and rest a bit. Double-check that in Settings -> Storage, it doesn't say "running out of storage". If it does say that, reboot the emulator. That should get rid of the message. If you try to install the app when the device is in its strange "running out of storage state", you will get "stale dex" errors when installing the app. I'm using the x86 emulator on a 2018 MacBook Pro.

2. Start log capturing with this command: adb -s emulator-5554 logcat -T 0 -v threadtime -s '*:V' | tee ~/log-workmanager-repro-verbose-3.txt

3. Start the app, and after ~10 seconds, close it (I've increased sleep to 20 s)

4. Wait for a long enough time to convince yourself of that the Worker will never be scheduled again (I waited for > 10 minutes)

5. Stop logging.

For your convenience, I have inlined the log filtered with: grep -e Repro -e WM- -e Killing.*workmanager ~/log-workmanager-repro-verbose-3.txt
at the end of the mail. For full logs, see attached file.

The last line with 'Killing' is from me swiping away the app in the App Switcher.

Note that the logs have this line:
WM-SystemAlarmService: All commands completed in dispatcher
So it looks like even though SystemAlarmService was used to dispatch the Worker, the bug happened. ("All commands completed in dispatcher" means SystemAlarmService is about to do Service.stopSelf() as you probably know.)

Let me know if I can help you in any other way.

BR Martin



01-30 09:42:31.954  3613  3613 D WM-Schedulers: Created SystemAlarmScheduler
01-30 09:42:31.960  3613  3613 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.SystemAlarmService enabled
01-30 09:42:31.970  3613  3632 D WM-ForceStopRunnable: Application was force-stopped, rescheduling.
01-30 09:42:31.993  3613  3613 E Repro   : Calling enqueueUniqueWork(1548837751988, ...)
01-30 09:42:32.040  3613  3632 D WM-PackageManagerHelper: androidx.work.impl.background.systemalarm.RescheduleReceiver enabled
01-30 09:42:32.043  3613  3632 D WM-SystemAlarmScheduler: Scheduling work with workSpecId 576d2452-a559-4c85-b217-c954c6905d7e
01-30 09:42:32.046  3613  3632 D WM-Processor: Processor: processing 576d2452-a559-4c85-b217-c954c6905d7e
01-30 09:42:32.054  3613  3613 D WM-SystemAlarmDispatche: Adding command Intent { act=ACTION_SCHEDULE_WORK cmp=com.github.enselic.workmanagerfailedscheduling/androidx.work.impl.background.systemalarm.SystemAlarmService (has extras) } (1)
01-30 09:42:32.055  3613  3632 D WM-SystemAlarmDispatche: Processing command Intent { act=ACTION_SCHEDULE_WORK cmp=com.github.enselic.workmanagerfailedscheduling/androidx.work.impl.background.systemalarm.SystemAlarmService (has extras) }, 1
01-30 09:42:32.055  3613  3632 D WM-SystemAlarmDispatche: Acquiring operation wake lock (ACTION_SCHEDULE_WORK) WakeLock{4e8f358 held=false, refCount=0}
01-30 09:42:32.055  3613  3632 D WM-CommandHandler: Handling schedule work for 576d2452-a559-4c85-b217-c954c6905d7e
01-30 09:42:32.056  3613  3637 E Repro   : doWork() called. Now swipe away app in App Switcher. Tags: [com.github.enselic.workmanagerfailedscheduling.LogPrinter, 1548837751988]
01-30 09:42:32.056  3613  3632 W WM-CommandHandler: Skipping scheduling 576d2452-a559-4c85-b217-c954c6905d7e because it is no longer enqueued
01-30 09:42:32.056  3613  3632 D WM-SystemAlarmDispatche: Releasing operation wake lock (ACTION_SCHEDULE_WORK) WakeLock{4e8f358 held=true, refCount=1}
01-30 09:42:32.096  3613  3613 D WM-SystemAlarmDispatche: Checking if commands are complete.
01-30 09:42:32.096  3613  3613 D WM-SystemAlarmDispatche: Removing command Intent { act=ACTION_SCHEDULE_WORK cmp=com.github.enselic.workmanagerfailedscheduling/androidx.work.impl.background.systemalarm.SystemAlarmService (has extras) }
01-30 09:42:32.096  3613  3613 D WM-SystemAlarmDispatche: No more commands & intents.
01-30 09:42:32.096  3613  3613 D WM-SystemAlarmService: All commands completed in dispatcher
01-30 09:42:43.607  1647  2397 I ActivityManager: Killing 3613:com.github.enselic.workmanagerfailedscheduling/u0a58 (adj 9): remove task

Thanks for providing more information.. This is in line with a fix that I am working on. This should be fixed in the next beta.

https://android-review.googlesource.com/c/platform/frameworks/support/+/889837

Project: platform/frameworks/support
Branch: androidx-master-dev

commit ebb4f87d732f66d01d1a4cf362bb11a815e80b74
Author: Rahul Ravikumar <rahulrav@google.com>
Date:   Tue Jan 29 16:16:17 2019

    Correctly retry work on process death when using AlarmManager based implementation.
   
    * AlarmManager now schedules a Worker unless the WorkSpec#isFinished() is true.
    * When AlarmManager gets worken up by ActivityManager to resume processing, we
      correctly execute the Worker after a backoff period.
    * We now correctly handle rescheduling in DelayMetCommandHandler. Previously the
      only reason why this worked before was the call to Schedulers.schedule() in
      WorkerWrapper inside the onWorkFinished() method.
    * When delaying execution of Workers in WorkerWrapper we always reschedule givem
      SystemJobScheduler does not schedule jobs where JobScheduler is already aware
      of the jobId.
    * Added a lot more logging in GreedyScheduler, SystemJobScheduler and SystemAlarmScheduler
      to make debugging easier.
    * Ran a lot of integration tests on API 22, 23 (double scheduling) and 28.
      Look at https://paste.googleplex.com/5709211287683072?raw
      for more information. I also ran integration tests that cancel
      workers to sanity check.
   
    Fixes: b/123329850
    Test: Added additional test cases in SystemAlarmDispatcherTest.
   
    Change-Id: I1bcff1ebf5f60928a02e3e94e2e6ccaed0de47b1

M       work/integration-tests/testapp/src/main/java/androidx/work/integration/testapp/MainActivity.java
A       work/integration-tests/testapp/src/main/java/androidx/work/integration/testapp/RetryWorker.java
M       work/integration-tests/testapp/src/main/res/layout/activity_main.xml
M       work/integration-tests/testapp/src/main/res/values/strings.xml
M       work/workmanager/src/androidTest/java/androidx/work/impl/WorkerWrapperTest.java
M       work/workmanager/src/androidTest/java/androidx/work/impl/background/systemalarm/SystemAlarmDispatcherTest.java
M       work/workmanager/src/main/java/androidx/work/impl/WorkerWrapper.java
M       work/workmanager/src/main/java/androidx/work/impl/background/greedy/GreedyScheduler.java
M       work/workmanager/src/main/java/androidx/work/impl/background/systemalarm/CommandHandler.java
M       work/workmanager/src/main/java/androidx/work/impl/background/systemalarm/DelayMetCommandHandler.java
M       work/workmanager/src/main/java/androidx/work/impl/background/systemjob/SystemJobScheduler.java

https://android-review.googlesource.com/889837
https://goto.google.com/android-sha1/ebb4f87d732f66d01d1a4cf362bb11a815e80b74