IssueTracker

Same as https://partnerissuetracker.corp.google.com/issues/242248365

Ack, yeah there is a bug in this scenario. We'll try to address it in next release.

Thanks for your information (#10 comment). Can we know the next release plan and schedule?

it will be in 2.9.0-alpha01 release which likely be early next year.

We are facing high number of ANR related to androidx.work.impl.background.systemalarm.RescheduleReceiver too, after we migrate from API 31 to API 33.

The characteristic of such ANR are

1. Only happen in devices with Android 13.
2. Only happen in background, not foreground.

Based on comment from https://issuetracker.google.com/issues/236906724#comment2, we thought we might be doing too many things during app startup.

We also suspect Firebase libraries are the real cause based on some reports - https://issuetracker.google.com/issues/230950647

Even we have upgraded from com.google.firebase:firebase-bom:31.1.0 to com.google.firebase:firebase-bom:31.2.2 (latest to-date), we still observe a high number of ANR in Android 13, background.

We also upgraded to use 2.8.0 WorkManager.

May I know, will 2.9.0-alpha01 able to resolve such ANR? - https://issuetracker.google.com/issues/236906724#comment12

Thank you.

Hi,

Wondering when will 2.9.0-alpha01 be available for us to try out? Thank you very much.

Hi Googlers,

Thank you for taking effort to address this issue. Allow me to provide more info, so that we can track down to the root cause.

Here's my app configuration, with healthy ANR number

API 31
def work_version = "2.7.1"
implementation "androidx.work:work-runtime:$work_version"
implementation "androidx.work:work-gcm:$work_version"

After we upgrade to API 33, we saw a huge spike in ANR (In Android 33 T, all happens in background)

API 33
def work_version = "2.7.1"
implementation "androidx.work:work-runtime:$work_version"
implementation "androidx.work:work-gcm:$work_version"

Here's the ANR spike we are seeing.

Native method - android.os.MessageQueue.nativePollOnce Broadcast of Intent { act=android.intent.action.TIME_SET cmp=com.xxx.yyy/androidx.work.impl.background.systemalarm.RescheduleReceiver }

android.os.MessageQueue.nativePollOnce Broadcast of Intent { act=android.intent.action.BOOT_COMPLETED cmp=com.xxx.yyy/androidx.work.impl.background.systemalarm.RescheduleReceiver }

Occurrence of TIME_SET ANR is having 3 to 4 times more than BOOT_COMPLETED ANR. A device will either hit TIME_SET ANR, or BOOT_COMPLETED ANR, but not both.

We make sure we upgrade every libraries to latest, including WorkManager.

API 33
def work_version = "2.8.0"
implementation "androidx.work:work-runtime:$work_version"
implementation "androidx.work:work-gcm:$work_version"

However, we are still facing high ANR in Android T.

Interestingly, I after placing my app in background, and trying to change the emulator time manually. Hopefully able to trigger RescheduleReceiver with TIME_SET. But, I didn't observe any logging from RescheduleReceiver.

I do have some questions regarding RescheduleReceiver call on goAsync

1. I expect when RescheduleReceiver call goAsync on main thread, it will start to schedule to run some "busy" task on user thread. But, I do not see such logic in RescheduleReceiver. Am I missing something?

2. I was pondering, if 1st goAsync occur, and then 2nd goAsync occur again before the mRescheduleReceiverResult.finish() associated with 1st goAsysnc is executed, can 2nd goAsync causes ANR?

Thank you for taking effort for tackling this issue. I attach some screenshots for your reference, hoping able to help in tracking down the root cause.

This week, I will roll out a remote config change, to disable WorkManager enqueuing in all Android 13 device (For few days). My purpose is to monitor how WM affects ANR rate. I will update more on my findings.

Please let me know, if you need more info from my side.

A missing attachment for above thread.

But, I do not see such logic in RescheduleReceiver. Am I missing something? I was pondering, if 1st goAsync occur, and then 2nd goAsync occur again before the mRescheduleReceiverResult.finish() associated with 1st goAsync is executed, can 2nd goAsync causes ANR?

We think that it is highly likely that second goAsync triggers ANR. We actually even tried to address it in aosp/2401352. However apps that use nightly build of androidx (you can use it too, see) reported huge spike in volume of ANRs specifically on android T. So that fix was reverted. Once we confirm that ANR goes back to previous number, we'll explore why that CL was triggering more ANRs on T (likely platform bug) and how else we can address ANRs.

Hi Googlers,

It isn't clear on why the initial proposed fix (which is now reverted) can cause spike in ANR.

It isn't totally clear too, on why current 2.8.0 WM is causing ANR. Hypothesis provided by Samsung engineer might hold true.

I am not able to test BOOT_COMPLETED as it is not easy to test - https://stackoverflow.com/questions/40698450/android-trying-to-test-a-service-on-boot-java-lang-securityexception-permiss

I however try to do a "fast" manual time adjustment, to fire 2 TIME_SET.

Using the following code

public class MyBroadcastReceiver extends BroadcastReceiver {
    private static final Executor dateTimeReminderExecutor = Executors.newSingleThreadExecutor();

    @Override
    public void onReceive(Context context, Intent intent) {
        Log.i("CHEOK", intent.getAction() + " : onReceive -> " + this);

        final PendingResult result = goAsync();

        dateTimeReminderExecutor.execute(() -> {
            Log.i("CHEOK", "Begin sleep -> " + this);
            try {
                Thread.sleep(20000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            Log.i("CHEOK", "Finish sleep -> " + this);

            try {
            } finally {
                if (result != null) {
                    result.finish();
                    Log.i("CHEOK", "result.finish() la -> " + this);
                }
            }
        });
    }
}

I obtain the following outcome, in both API 33 and API 32.

android.intent.action.TIME_SET : onReceive -> com.yocto.myapplication.MyBroadcastReceiver@9140719
Begin sleep -> com.yocto.myapplication.MyBroadcastReceiver@9140719
Finish sleep -> com.yocto.myapplication.MyBroadcastReceiver@9140719
result.finish() la -> com.yocto.myapplication.MyBroadcastReceiver@9140719
android.intent.action.TIME_SET : onReceive -> com.yocto.myapplication.MyBroadcastReceiver@92a09bf
Begin sleep -> com.yocto.myapplication.MyBroadcastReceiver@92a09bf
Finish sleep -> com.yocto.myapplication.MyBroadcastReceiver@92a09bf
result.finish() la -> com.yocto.myapplication.MyBroadcastReceiver@92a09bf

Based on the observation

1. 1st TIME_SET will trigger BroadcastReceiver's onReceive.
2. 2nd TIME_SET will not trigger BroadcastReceiver's onReceive, after 1st TIME_SET has called goAsync and before calling result.finish. 2nd TIME_SET will only trigger BroadcastReceiver's onReceive, after 1st TIME_SET call result.finish.
3. Different instances of BroadcastReceiver are used to serve 1st and 2nd TIME_SET

If that is the case, 2nd action are still "waiting" to enter RescheduleReceiver's onReceive.

2nd action might not have chance to call WorkManagerImpl's setReschedulePendingResult to "accidentally" override member variable mRescheduleReceiverResult, because the 1st action has not called mRescheduleReceiverResult.finish()

This is my very rough initial observation, and they might not hold true.

I am not able to reproduce any ANR even I have "Enable background ANR dialogs". Can you?

Thank you for your efforts.

May I know when will be released for testing? Thank you.

It will be released as 2.8.1 in upcoming days.

Thank you. I am looking forward the release.

Hi, after releasing 2.8.1 to production for more than 1 week, I can confirm that this ANR is being reduced significantly.

Thank you for being responsive in this issue.

The fix doesn't seem to work. I can see see ANRs happening, mostly on Samsung devices.

Edit: screenshot from Play Console "Crashes and ANRs" tab.