IssueTracker

This should fixed by https://android-review.googlesource.com/c/platform/frameworks/support/+/1226859.

This should land in WorkManager 2.3.2.

We fixed a bug recently, where there is a mismatch between WorkManagers tracking of jobs in JobScheduler and the actual state of the jobs. This happens in very rare cases, and I think that's what is going on in your case.

https://issuetracker.google.com/issues/149092520 was a recently reported bug. I fixed it before the bug was reported on the public tracker, because one of Google's apps ran into the same edge case.

This change (https://android-review.googlesource.com/c/platform/frameworks/support/+/1226859) has the fix. We should release this as part of WorkManager 2.3.2 very soon.

If you cannot wait, you can use a snapshot build that should be identical to the eventual release.

Here is the gradle snippet you will need:

repositories {
    google()
    maven { url 'https://ci.android.com/builds/submitted/6195753/androidx_snapshot/latest/repository/' }
}

dependencies {
  implementation "androidx.work:work-runtime:2.4.0-SNAPSHOT"
}

One other thing you might need to do is to get rid of stale jobs. It is as simple as calling JobScheduler.cancelAllJobs() before WorkManager is initialized. You can do it inside Application.onCreate() in your Application subclass.

My apologies for the delay. Try using this. This is from our dev branch.

repositories {
    google()
    maven { url 'https://ci.android.com/builds/submitted/6215972/androidx_snapshot/latest/repository/' }
}

dependencies {
  implementation "androidx.work:work-runtime:2.4.0-SNAPSHOT"
}

My apologies, it looks like the artifacts were not generated for that build. I have verified that this one works:

https://ci.android.com/builds/submitted/6217200/androidx_snapshot/latest/repository/

repositories { google() maven { url 'https://ci.android.com/builds/submitted/6217200/androidx_snapshot/latest/repository/' } }

dependencies { implementation "androidx.work:work-runtime:2.4.0-SNAPSHOT" }

My apologies, it looks like the artifacts were not generated for that build. I have verified that this one works:

https://ci.android.com/builds/submitted/6217200/androidx_snapshot/latest/repository/

repositories {
    google()
    maven { url 'https://ci.android.com/builds/submitted/6217200/androidx_snapshot/latest/repository/' }
}

dependencies {
  implementation "androidx.work:work-runtime:2.4.0-SNAPSHOT"
}

Re: shahbaz.hussain@magicpin.in

Your app is probably using JobScheduler directly, besides WorkManager. That is when this happens.

Yes, AppMeasurementJobService would be an example.

Hi,

I am using the version 2.3.4 of the work manager library and I am still having the same problem. I didn't do JobScheduler.cancelAllJobs() before WorkManager is initialized. You can do it inside Application.onCreate() in your Application subclass. because we are using a third party library that is scheduling jobs and we don't really know how could affect the behaviour of the app if we cancel everything.

Is there anyway to get the only stale jobs that can be cancelled?

Regards!

You have another JobService in your application which is scheduling too many jobs.

The exception from WorkManager clearly states that we only scheduled 20 jobs.

Re: #23.

The error is very similar. You have another job service running & another part of the application is scheduling jobs directly using JobScheduler's APIs. There are 30 jobs being tracked, and you have 48 jobs attributed to WorkManager based on the error message.

Someone else is scheduling 52+ jobs.

That is because, the bug is in another library and not WorkManager. Just because you see it does not mean WorkManager caused it.

Why can't you at least show the stack trace of which function tried to create a new task? Or better: the place of each of the tasks that were created?

How can we do that if its not our code scheduling work with JobScheduler ?

Also, how can you be sure it's not WorkManager, if multiple developers here reported about it?

The error message makes it very clear how many jobs WorkManager actually knows about. In the attached screenshot, WorkManagers configuration limit is 50, and there are 48 known jobs. The app however is exceeding scheduling limits. So some jobs are not managed by WorkManager.

@28 You can always access the stack trace. And, when there is about to be a crash, you can query which tasks are pending and print all the information about them.

You could do that too, if you were really curious. And if you find bugs in other Google libraries, please file bugs.

Hi,

This issue is marked as fixed but to me it appears that some of the comments are NOT clearly addressed. We are also facing this issue using v2.5.0 and the stack trace is very similar to the #20:

Fatal Exception: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded.  We count 101 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.
       at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:204)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.schedule(SystemJobScheduler.java:132)
       at androidx.work.impl.Schedulers.schedule(Schedulers.java:108)
       at androidx.work.impl.utils.ForceStopRunnable.forceStopRunnable(ForceStopRunnable.java:176)
       at androidx.work.impl.utils.ForceStopRunnable.run(ForceStopRunnable.java:102)
       at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:91)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:923)

It explicitly says that 101 jobs belongs to the WorkManager. This rules out the possibility that the jobs may be scheduled by another library that directly use JobScheduler APIs. In fact, I created a sample project to test this and verified that when JobScheduler APIs are used, the number in the stack trace becomes 0 as jobs are not scheduled by WorkManager.

From what I could gather after reading the source code where this exception occurs, it is impossible to create a scenario that would result in this exception using WorkManager APIs since the stack trace says the configuration limit is 20 and it should not be possible for WorkManager to schedule more than 20 jobs. (see: Schedulers#schedule, eligibleWorkSpecsForLimitedSlots)

Fatal Exception: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded. We count 101 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.

This means that you have only scheduled 20 jobs that WorkManager is aware of. There is another JobService being used in the app (I would look at your merged app-manifest) that is the offender here.

This means that you have only scheduled 20 jobs that WorkManager is aware of

I am not sure if this inference is correct for the stack trace I've shared.

Looking at the source code for the SystemJobScheduler#getPendingJobs where the number 101 in the stack trace comes from, I can see that it is the number of jobs that JobScheduler is aware of and uses SystemJobService as the service component. SystemJobService is the service used by WorkManager to schedule jobs. Even if the JobScheduler is used directly by another library as you pointed out, it would not use the SystemJobService and therefore would not show up or add up to the number we see in the stack trace.

No, we tabulate both. Jobs we own (like you suggested) and jobs scheduled overall. Like I said, this is not something WorkManager can help with. You need to figure out the actual component to blame. Even something as simple as adb shell dumpsys jobscheduler will tell you what's going on.

No, we tabulate both. Jobs we own (like you suggested) and jobs scheduled overall.

Again, I would have to disagree. Below is the related part of the source code:

    @Nullable
    private static List<JobInfo> getPendingJobs(
            @NonNull Context context,
            @NonNull JobScheduler jobScheduler) {
        List<JobInfo> pendingJobs = null;
        try {
            // Note: despite what the word "pending" and the associated Javadoc might imply, this is
            // actually a list of all unfinished jobs that JobScheduler knows about for the current
            // process.
            pendingJobs = jobScheduler.getAllPendingJobs();
        } catch (Throwable exception) {
            // OEM implementation bugs in JobScheduler cause the app to crash. Avoid crashing.
            Logger.get().error(TAG, "getAllPendingJobs() is not reliable on this device.",
                    exception);
        }

        if (pendingJobs == null) {
            return null;
        }

        // Filter jobs that belong to WorkManager.
        List<JobInfo> filtered = new ArrayList<>(pendingJobs.size());
        ComponentName jobServiceComponent = new ComponentName(context, SystemJobService.class);
        for (JobInfo jobInfo : pendingJobs) {
            if (jobServiceComponent.equals(jobInfo.getService())) {
                filtered.add(jobInfo);
            }
        }
        return filtered;
    }

The first number does NOT contain all the scheduled jobs as there is an explicit filtering for counting only the jobs that belong to WorkManager. Wouldn't this mean that 101 jobs in the stack trace I've shared were actually scheduled by the WorkManager at some point? Otherwise it would've been filtered out in the last part above.

I think what you suggest is a separate scenario where bug with a similar stack trace could be encountered but in that case I would expect a different stack trace from what I've shared. What you suggest would look more like the stack trace in the #15 where it says there are only 3 jobs scheduled by WorkManager whereas in my case it says 101.

WorkManager 2.5.x is over 2 years old at this point. Please upgrade to the latest stable version and then file a new bug report. In the latest alpha version, you can go further and use a SchedulingExceptionHandler (an @Restricted API) to get better signals on where the underlying problem might be.

Also, 2.5.x is not supported on Android S+ devices at all. You will need a minimum of 2.7.1.

You are very likely using another library that is scheduling jobs directly. I would look at the merged AndroidManifest.xml to find the other JobService instance.

We are using androidx.work:work-runtime:2.9.0 and getting the same exception in Firebase Crashlytics.

Fatal Exception: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded.  We count 100 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.
       at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:223)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.schedule(SystemJobScheduler.java:140)
       at androidx.work.impl.Schedulers.schedule(Schedulers.java:133)
       at androidx.work.impl.utils.ForceStopRunnable.forceStopRunnable(ForceStopRunnable.java:266)
       at androidx.work.impl.utils.ForceStopRunnable.run(ForceStopRunnable.java:135)
       at androidx.work.impl.utils.SerialExecutorImpl$Task.run(SerialExecutorImpl.java:96)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)
Caused by java.lang.IllegalStateException: Apps may not schedule more than 100 distinct jobs
       at android.os.Parcel.createException(Parcel.java:2096)
       at android.os.Parcel.readException(Parcel.java:2056)
       at android.os.Parcel.readException(Parcel.java:2004)
       at android.app.job.IJobScheduler$Stub$Proxy.schedule(IJobScheduler.java:324)
       at android.app.JobSchedulerImpl.schedule(JobSchedulerImpl.java:43)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:194)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.schedule(SystemJobScheduler.java:140)
       at androidx.work.impl.Schedulers.schedule(Schedulers.java:133)
       at androidx.work.impl.utils.ForceStopRunnable.forceStopRunnable(ForceStopRunnable.java:266)
       at androidx.work.impl.utils.ForceStopRunnable.run(ForceStopRunnable.java:135)
       at androidx.work.impl.utils.SerialExecutorImpl$Task.run(SerialExecutorImpl.java:96)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)

More information:

• All of the places we enqueue work are using the enqueueUnique version, with the KEEP flag
• We do not bind directly to the JOB_SCHEDULER_SERVICE anywhere in our code.
• I checked the merged manifest, here are the list of services to bind to the JOB_SCHEDULER_SERVICE, they are all part of Firebase: com.google.android.datatransport.runtime.scheduling.jobscheduling.JobInfoSchedulerService, com.google.android.gms.measurement.AppMeasurementJobService, com.google.android.gms.analytics.AnalyticsJobService, androidx.work.impl.background.systemjob.SystemJobService

Any ideas?

I am also seeing this error state with WorkManager 2.8.1 and 2.9.0, possibly arising in connection with crashes in the app using WorkManager.

2024-05-29 18:52:43.792824737 +0000 10041 23649 23685 E AndroidRuntime: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded.  We count 151 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.
2024-05-29 18:52:43.792824737 +0000 10041 23649 23685 E AndroidRuntime: 	at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:220)

There is no direct JobScheduler API use at all in the codebase, only WorkManager.

Are you seeing reference to other job services when you run adb shell dumpsys jobscheduler?

The error message is more confusing than it needs to be. The number of WorkRequests that WorkManager is aware of is 20. Even though the error message uses the name WorkManager jobs, its actually referring to the total number of jobs that JobScheduler knows about. There seem to be other jobs that a dependency might be scheduling.

In the incident quoted at #42, there are no jobs scheduled for the app other than WorkManager jobs.

We've gone so far as to decompile and audit the deployed APK for any Job Scheduler use that might have been brought in "silently" via libraries. I can confirm that the only scheduled job access by this app is via WorkManager. (And WM's androidx.work.impl.background.systemjob.SystemJobService is the only job service declared in the APK, also verified via direct inspection with aapt.)

That is extremely strange. Can you try and turn on verbose logging using the instructions in:

https://developer.android.com/develop/background-work/background-tasks/testing/persistent/debug

Could you please Log.DEBUG as your log level so we get more information. Also, can you consistently reproduce this?

We are also seeing this error in the crashlytics.

#47, How did you resolve the issue, are you doing pruneWork in Application.onCreate? Were you able to replicate the issue?

For posterity: If you are seeing this on a new version of WorkManager - some other component in the app is scheduling jobs and that is what is causing this.

Thank you for reverting back.
Yes, we are using the latest version of WM, that is 2.9.0. The error mentions the count of WM jobs scheduled. "JobScheduler 100 job limit exceeded. We count 151 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20."

And internally observed, that WM jobs are filtered from all pending jobs:

// Filter jobs that belong to WorkManager.
List<JobInfo> filtered = new ArrayList<>(pendingJobs.size());
ComponentName jobServiceComponent = new ComponentName(context, SystemJobService.class);
for (JobInfo jobInfo : pendingJobs) {
  if (jobServiceComponent.equals(jobInfo.getService())) {
    filtered.add(jobInfo);
  }
}

Please let me know if I am missing something here.

This is still working on other launcher apps running on Android 14 (Action Launcher, Lawnchair, etc.). Can we confirm that it is really "intended behavior" to make it stop working on the official Google Pixel launcher app?

Even the documentation for developers at

https://developer.android.com/develop/ui/views/notifications/badges

still states the following:

"Starting with Android 8.0 (API level 26), notification badges—also known as notification dots—appear on a launcher icon when the associated app has an active notification. Users can touch & hold the app icon to reveal the notifications, along with any app shortcuts, as shown in figure 1."

:)

The removal of this feature is very disappointing. It's the best way to access notifications from a given app. For those of us that don't care to have a clear notification bar all the time, it was a life saver. Pretty please, bring it back as an option at least 🙏🙏🙏