IssueTracker

this issue on workManager 2.5.0 when initialize WorkManager in background thread, bring it back to UI thread

Still happening with WorkManager 2.7.1 Samsung Galaxy S21 Ultra 5G Android 12 (SDK 31)

java.lang.RuntimeException: 
  at android.app.ActivityThread.handleCreateService (ActivityThread.java:4953)
  at android.app.ActivityThread.access$1900 (ActivityThread.java:310)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:2300)
  at android.os.Handler.dispatchMessage (Handler.java:106)
  at android.os.Looper.loopOnce (Looper.java:226)
  at android.os.Looper.loop (Looper.java:313)
  at android.app.ActivityThread.main (ActivityThread.java:8669)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:571)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1135)
Caused by: java.lang.IllegalStateException: 
  at androidx.work.impl.WorkManagerImpl.getInstance (WorkManagerImpl.java:158)
  at androidx.work.impl.foreground.SystemForegroundDispatcher.<init> (SystemForegroundDispatcher.java:105)
  at androidx.work.impl.foreground.SystemForegroundService.initializeDispatcher (SystemForegroundService.java:96)
  at androidx.work.impl.foreground.SystemForegroundService.onCreate (SystemForegroundService.java:59)
  at android.app.ActivityThread.handleCreateService (ActivityThread.java:4940)
  at android.app.ActivityThread.access$1900 (ActivityThread.java:310)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:2300)
  at android.os.Handler.dispatchMessage (Handler.java:106)
  at android.os.Looper.loopOnce (Looper.java:226)
  at android.os.Looper.loop (Looper.java:313)
  at android.app.ActivityThread.main (ActivityThread.java:8669)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:571)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:1135)

I would check to make sure that the app is not a multi-process app. If it is, you will need your Application to additionally sub-class Configuration.Provider or additionally define a second provider for the other process.

Why can't you at least show the stack trace of which function tried to create a new task? Or better: the place of each of the tasks that were created?

How can we do that if its not our code scheduling work with JobScheduler ?

Also, how can you be sure it's not WorkManager, if multiple developers here reported about it?

The error message makes it very clear how many jobs WorkManager actually knows about. In the attached screenshot, WorkManagers configuration limit is 50, and there are 48 known jobs. The app however is exceeding scheduling limits. So some jobs are not managed by WorkManager.

@28 You can always access the stack trace. And, when there is about to be a crash, you can query which tasks are pending and print all the information about them.

You could do that too, if you were really curious. And if you find bugs in other Google libraries, please file bugs.

Hi,

This issue is marked as fixed but to me it appears that some of the comments are NOT clearly addressed. We are also facing this issue using v2.5.0 and the stack trace is very similar to the #20:

Fatal Exception: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded.  We count 101 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.
       at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:204)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.schedule(SystemJobScheduler.java:132)
       at androidx.work.impl.Schedulers.schedule(Schedulers.java:108)
       at androidx.work.impl.utils.ForceStopRunnable.forceStopRunnable(ForceStopRunnable.java:176)
       at androidx.work.impl.utils.ForceStopRunnable.run(ForceStopRunnable.java:102)
       at androidx.work.impl.utils.SerialExecutor$Task.run(SerialExecutor.java:91)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:923)

It explicitly says that 101 jobs belongs to the WorkManager. This rules out the possibility that the jobs may be scheduled by another library that directly use JobScheduler APIs. In fact, I created a sample project to test this and verified that when JobScheduler APIs are used, the number in the stack trace becomes 0 as jobs are not scheduled by WorkManager.

From what I could gather after reading the source code where this exception occurs, it is impossible to create a scenario that would result in this exception using WorkManager APIs since the stack trace says the configuration limit is 20 and it should not be possible for WorkManager to schedule more than 20 jobs. (see: Schedulers#schedule, eligibleWorkSpecsForLimitedSlots)

Fatal Exception: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded. We count 101 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.

This means that you have only scheduled 20 jobs that WorkManager is aware of. There is another JobService being used in the app (I would look at your merged app-manifest) that is the offender here.

This means that you have only scheduled 20 jobs that WorkManager is aware of

I am not sure if this inference is correct for the stack trace I've shared.

Looking at the source code for the SystemJobScheduler#getPendingJobs where the number 101 in the stack trace comes from, I can see that it is the number of jobs that JobScheduler is aware of and uses SystemJobService as the service component. SystemJobService is the service used by WorkManager to schedule jobs. Even if the JobScheduler is used directly by another library as you pointed out, it would not use the SystemJobService and therefore would not show up or add up to the number we see in the stack trace.

No, we tabulate both. Jobs we own (like you suggested) and jobs scheduled overall. Like I said, this is not something WorkManager can help with. You need to figure out the actual component to blame. Even something as simple as adb shell dumpsys jobscheduler will tell you what's going on.

No, we tabulate both. Jobs we own (like you suggested) and jobs scheduled overall.

Again, I would have to disagree. Below is the related part of the source code:

    @Nullable
    private static List<JobInfo> getPendingJobs(
            @NonNull Context context,
            @NonNull JobScheduler jobScheduler) {
        List<JobInfo> pendingJobs = null;
        try {
            // Note: despite what the word "pending" and the associated Javadoc might imply, this is
            // actually a list of all unfinished jobs that JobScheduler knows about for the current
            // process.
            pendingJobs = jobScheduler.getAllPendingJobs();
        } catch (Throwable exception) {
            // OEM implementation bugs in JobScheduler cause the app to crash. Avoid crashing.
            Logger.get().error(TAG, "getAllPendingJobs() is not reliable on this device.",
                    exception);
        }

        if (pendingJobs == null) {
            return null;
        }

        // Filter jobs that belong to WorkManager.
        List<JobInfo> filtered = new ArrayList<>(pendingJobs.size());
        ComponentName jobServiceComponent = new ComponentName(context, SystemJobService.class);
        for (JobInfo jobInfo : pendingJobs) {
            if (jobServiceComponent.equals(jobInfo.getService())) {
                filtered.add(jobInfo);
            }
        }
        return filtered;
    }

The first number does NOT contain all the scheduled jobs as there is an explicit filtering for counting only the jobs that belong to WorkManager. Wouldn't this mean that 101 jobs in the stack trace I've shared were actually scheduled by the WorkManager at some point? Otherwise it would've been filtered out in the last part above.

I think what you suggest is a separate scenario where bug with a similar stack trace could be encountered but in that case I would expect a different stack trace from what I've shared. What you suggest would look more like the stack trace in the #15 where it says there are only 3 jobs scheduled by WorkManager whereas in my case it says 101.

WorkManager 2.5.x is over 2 years old at this point. Please upgrade to the latest stable version and then file a new bug report. In the latest alpha version, you can go further and use a SchedulingExceptionHandler (an @Restricted API) to get better signals on where the underlying problem might be.

Also, 2.5.x is not supported on Android S+ devices at all. You will need a minimum of 2.7.1.

You are very likely using another library that is scheduling jobs directly. I would look at the merged AndroidManifest.xml to find the other JobService instance.

We are using androidx.work:work-runtime:2.9.0 and getting the same exception in Firebase Crashlytics.

Fatal Exception: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded.  We count 100 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.
       at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:223)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.schedule(SystemJobScheduler.java:140)
       at androidx.work.impl.Schedulers.schedule(Schedulers.java:133)
       at androidx.work.impl.utils.ForceStopRunnable.forceStopRunnable(ForceStopRunnable.java:266)
       at androidx.work.impl.utils.ForceStopRunnable.run(ForceStopRunnable.java:135)
       at androidx.work.impl.utils.SerialExecutorImpl$Task.run(SerialExecutorImpl.java:96)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)
Caused by java.lang.IllegalStateException: Apps may not schedule more than 100 distinct jobs
       at android.os.Parcel.createException(Parcel.java:2096)
       at android.os.Parcel.readException(Parcel.java:2056)
       at android.os.Parcel.readException(Parcel.java:2004)
       at android.app.job.IJobScheduler$Stub$Proxy.schedule(IJobScheduler.java:324)
       at android.app.JobSchedulerImpl.schedule(JobSchedulerImpl.java:43)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:194)
       at androidx.work.impl.background.systemjob.SystemJobScheduler.schedule(SystemJobScheduler.java:140)
       at androidx.work.impl.Schedulers.schedule(Schedulers.java:133)
       at androidx.work.impl.utils.ForceStopRunnable.forceStopRunnable(ForceStopRunnable.java:266)
       at androidx.work.impl.utils.ForceStopRunnable.run(ForceStopRunnable.java:135)
       at androidx.work.impl.utils.SerialExecutorImpl$Task.run(SerialExecutorImpl.java:96)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)

More information:

• All of the places we enqueue work are using the enqueueUnique version, with the KEEP flag
• We do not bind directly to the JOB_SCHEDULER_SERVICE anywhere in our code.
• I checked the merged manifest, here are the list of services to bind to the JOB_SCHEDULER_SERVICE, they are all part of Firebase: com.google.android.datatransport.runtime.scheduling.jobscheduling.JobInfoSchedulerService, com.google.android.gms.measurement.AppMeasurementJobService, com.google.android.gms.analytics.AnalyticsJobService, androidx.work.impl.background.systemjob.SystemJobService

Any ideas?

I am also seeing this error state with WorkManager 2.8.1 and 2.9.0, possibly arising in connection with crashes in the app using WorkManager.

2024-05-29 18:52:43.792824737 +0000 10041 23649 23685 E AndroidRuntime: java.lang.IllegalStateException: JobScheduler 100 job limit exceeded.  We count 151 WorkManager jobs in JobScheduler; we have 20 tracked jobs in our DB; our Configuration limit is 20.
2024-05-29 18:52:43.792824737 +0000 10041 23649 23685 E AndroidRuntime: 	at androidx.work.impl.background.systemjob.SystemJobScheduler.scheduleInternal(SystemJobScheduler.java:220)

There is no direct JobScheduler API use at all in the codebase, only WorkManager.

Are you seeing reference to other job services when you run adb shell dumpsys jobscheduler?

The error message is more confusing than it needs to be. The number of WorkRequests that WorkManager is aware of is 20. Even though the error message uses the name WorkManager jobs, its actually referring to the total number of jobs that JobScheduler knows about. There seem to be other jobs that a dependency might be scheduling.

In the incident quoted at #42, there are no jobs scheduled for the app other than WorkManager jobs.

We've gone so far as to decompile and audit the deployed APK for any Job Scheduler use that might have been brought in "silently" via libraries. I can confirm that the only scheduled job access by this app is via WorkManager. (And WM's androidx.work.impl.background.systemjob.SystemJobService is the only job service declared in the APK, also verified via direct inspection with aapt.)

That is extremely strange. Can you try and turn on verbose logging using the instructions in:

https://developer.android.com/develop/background-work/background-tasks/testing/persistent/debug

Could you please Log.DEBUG as your log level so we get more information. Also, can you consistently reproduce this?