IssueTracker

Which device is issue being reported on? Can you get any logs from the device for us after you turn on verbose logging ?

A few things in addition to the questions in #2:

1. Can you verify that this problem still occurs in beta03?  This is available now.
2. Can you check your merged manifest has a network permission (ACCESS_NETWORK_STATE)?  It should get merged in automatically.

#3: have now tried beta03 and it's the same behaviour... periodic work schedule breaks down when network constraint is applied (after a period of time with no network).  And yes, ACCESS_NETWORK_STATE is in the merged manifest... it's actually in the main manifest anyway.

#2: I'll try...

Re #2 getting logs... I have tried previously to get logs from the user, and on each occasion the logs started *after* the point of interest, as if the logs had been cleared just before (e.g. just after a problem), so they've not been of any use.  

The way I'm getting logs is via a button in the widget configuration activity which runs the following:

Runtime.getRuntime().exec("logcat -f " + logFile + " *:V");

Is there a better way of getting logs from a user (where the user is not an advanced user and can only just about handle pressing a button and sending a text file)?

Looks like this issue isn't specific to Unique**Periodic**Work ... with a network constraint applied to UniqueWork, if it is scheduled for a time when there is no network, it doesn't go off, even when network is restored.  Again, on this user's Android 4.3 device... on the devices I've tested, it works fine.

Do I need to do anything special to make this work on Android 4.3?

And periodic work is running just fine on the problem device without the network constraint applied... runs like clockwork.

Can you send us a sample app?

Does your app have verbose logging turned on for WorkManager. You will need to initialize WorkManager with a custom configuration for that.

* Remove the default WorkManager initializer using:

 <provider
        android:name="androidx.work.impl.WorkManagerInitializer"
        android:authorities="${applicationId}.workmanager-init"
        tools:node="remove"/>

* Initialize WorkManager in your own content provider or Application#onCreate() with a custom configuration

WorkManager.initialize(
                applicationContext,
                new Configuration.Builder().setMinimumLoggingLevel(Log.VERBOSE).build());

You will see a lot more WorkManager logs now.

Typically when there are changes in constraints on API <=22 and you are using the AlarmManager implementation you should see logs with a log tag "WM-ConstraintsCmdHandler".

Do you have any comment or suggestion regarding mine at #5?  Is that an OK way to get logs from a user, particularly one that has a non-rooted device and where I don't particularly want to ask them to go to the trouble of install logcat apps etc... just a simple button in the app which will dump logs would be ideal.

We don't really have any good advice for you there; it touches on potentially sensitive data so getting data directly from the user is between what you and the user deem acceptable.

What is the user's 4.3 device name/model?

When I use:

WorkManager.initialize(
                applicationContext,
                new Configuration.Builder().setMinimumLoggingLevel(Log.VERBOSE).build());

The 'Builder' is highlighted in red with Cannot resolve symbol 'Builder'

Sorry, hard to tell without a sample app.  It sounds like you may have a missing import or something, but I can't diagnose that.

OK I used androidx.work.Configuration.Builder() instead

User's device is a Samsung Galaxy S3

Where can I send a logcat from the user?  There isn't anything of any sensitive nature so far as I can tell, but best not to post it here.

You can send the logs here; I just mean I can't advise you about collection.

I'm attaching some logs which may be of interest.  It includes the extra WorkManager logging you describe above, plus some random bits of logging from my own code (though I've removed quite a bit to avoid overkill here).  I've added some annotations to note where the user did various actions (I'm guessing a little but fairly sure) to add some context.

The interest bit starts from "01-29 20:51:44.715"

I looked at the logs, and there are a lot of Worker cancellations happening. Are you explicitly cancelling work or using Existing{Periodic}WorkPolicy.REPLACE ?
Can you send us a trimmed down sample of how Workers are being enqueued for your app ?

Sorry, I see that you are using unique work. Looking into this further.

I don't think I'm doing much out of the ordinary when scheduling the periodic work... basically as follows (leaving in the logging that corresponds to what you see in the logcat, and removing some of the value fetching logic, e.g. fetching values from SharedPrefs etc).  Entry point is e.g. schedulePeriodicWork(1234, "some-trigger");

    static void schedulePeriodicWork(int appWidgetId, String trigger) {
        Log.d(TAG, "schedulePeriodicWork for " + appWidgetId);
        String name = getPeriodicName(appWidgetId);
        long interval = getInterval();
        Constraints constraints = getConstraints("periodic");
        Data inputData = getPeriodicInputData(appWidgetId, trigger);
        PeriodicWorkRequest workRequest = getPeriodicWorkRequest(appWidgetId, interval, constraints, inputData);
        WorkManager workManager = WorkManager.getInstance();
        workManager.enqueueUniquePeriodicWork(name, ExistingPeriodicWorkPolicy.REPLACE, workRequest);
    }

    private static String getPeriodicName(int appWidgetId) {
        return "periodic-" + appWidgetId;
    }

    private static long getInterval() {
        return 15 * 60 * 1000;
    }

    private static Constraints getConstraints(String method) {
        Log.d(TAG, "creating Constraints for " + method);
        Constraints.Builder constraintsBuilder = new Constraints.Builder();
        Log.d(TAG, "adding network constraint");
        constraintsBuilder.setRequiredNetworkType(NetworkType.CONNECTED);
        return constraintsBuilder.build();
    }

    private static PeriodicWorkRequest getPeriodicWorkRequest(int appWidgetId, long interval, Constraints constraints, Data inputData) {
        String workTag = getWorkTag(appWidgetId);
        Log.d(TAG, "creating PeriodicWorkRequest with tag " + workTag + " and with interval " + interval + " ms");
        PeriodicWorkRequest.Builder workRequestBuilder = new PeriodicWorkRequest.Builder(MeteogramWorker.class, interval, TimeUnit.MILLISECONDS);
        workRequestBuilder.setConstraints(constraints);
        workRequestBuilder.setInputData(inputData);
        workRequestBuilder.addTag(workTag);
        workRequestBuilder.addTag(PERIODIC);
        return workRequestBuilder.build();
    }

    private static Data getPeriodicInputData(int appWidgetId, String trigger) {
        Log.d(TAG, "creating Data");
        Data.Builder dataBuilder = new Data.Builder();
        dataBuilder.putInt(AppWidgetManager.EXTRA_APPWIDGET_ID, appWidgetId);
        dataBuilder.putString(trigger);
        dataBuilder.putString("method", "periodic");
        return dataBuilder.build();
    }

    private static String getWorkTag(int appWidgetId) {
        return "widget-" + appWidgetId;
    }

Presumably there is some significance in the log entries around the time that the 15 min periodic work is due to run -- if there had been network available -- i.e. around 21:07 ?  i.e. the ones that refer to failed linking and resolving of work-related classes?  Examples:

01-29 21:07:01.245 W/dalvikvm(29185): Unable to resolve superclass of Landroidx/work/impl/background/systemjob/SystemJobService; (90)
01-29 21:07:01.245 W/dalvikvm(29185): Link of class 'Landroidx/work/impl/background/systemjob/SystemJobService;' failed
01-29 21:07:01.245 E/dalvikvm(29185): Could not find class 'androidx.work.impl.background.systemjob.SystemJobService', referenced from method androidx.work.impl.Schedulers.createBestAvailableBackgroundScheduler
01-29 21:07:01.245 W/dalvikvm(29185): VFY: unable to resolve const-class 2971 (Landroidx/work/impl/background/systemjob/SystemJobService;) in Landroidx/work/impl/Schedulers;
01-29 21:07:01.345 W/dalvikvm(29185): Unable to resolve superclass of Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback; (341)
01-29 21:07:01.345 W/dalvikvm(29185): Link of class 'Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback;' failed
01-29 21:07:01.350 E/dalvikvm(29185): Could not find class 'androidx.work.impl.constraints.trackers.NetworkStateTracker$NetworkStateCallback', referenced from method androidx.work.impl.constraints.trackers.NetworkStateTracker.<init>
01-29 21:07:01.350 W/dalvikvm(29185): VFY: unable to resolve new-instance 2994 (Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback;) in Landroidx/work/impl/constraints/trackers/NetworkStateTracker;
01-29 21:07:01.355 I/dalvikvm(29185): Could not find method android.net.ConnectivityManager.getActiveNetwork, referenced from method androidx.work.impl.constraints.trackers.NetworkStateTracker.isActiveNetworkValidated
01-29 21:07:01.355 W/dalvikvm(29185): VFY: unable to resolve virtual method 1865: Landroid/net/ConnectivityManager;.getActiveNetwork ()Landroid/net/Network;
01-29 21:07:01.355 D/dalvikvm(29185): VFY: replacing opcode 0x6e at 0x000a
01-29 21:07:01.360 W/dalvikvm(29185): Unable to resolve superclass of Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback; (341)
01-29 21:07:01.360 W/dalvikvm(29185): Link of class 'Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback;' failed
01-29 21:07:01.360 I/dalvikvm(29185): Could not find method android.net.ConnectivityManager.registerDefaultNetworkCallback, referenced from method androidx.work.impl.constraints.trackers.NetworkStateTracker.startTracking
01-29 21:07:01.360 W/dalvikvm(29185): VFY: unable to resolve virtual method 1871: Landroid/net/ConnectivityManager;.registerDefaultNetworkCallback (Landroid/net/ConnectivityManager$NetworkCallback;)V
01-29 21:07:01.365 W/dalvikvm(29185): Unable to resolve superclass of Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback; (341)
01-29 21:07:01.365 W/dalvikvm(29185): Link of class 'Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback;' failed
01-29 21:07:01.365 I/dalvikvm(29185): Could not find method android.net.ConnectivityManager.unregisterNetworkCallback, referenced from method androidx.work.impl.constraints.trackers.NetworkStateTracker.stopTracking
01-29 21:07:01.365 W/dalvikvm(29185): VFY: unable to resolve virtual method 1872: Landroid/net/ConnectivityManager;.unregisterNetworkCallback (Landroid/net/ConnectivityManager$NetworkCallback;)V
01-29 21:07:01.365 D/dalvikvm(29185): VFY: replacing opcode 0x6e at 0x0018
01-29 21:07:01.370 W/dalvikvm(29185): Unable to resolve superclass of Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback; (341)
01-29 21:07:01.370 W/dalvikvm(29185): Link of class 'Landroidx/work/impl/constraints/trackers/NetworkStateTracker$NetworkStateCallback;' failed
01-29 21:07:01.370 D/dalvikvm(29185): DexOpt: unable to opt direct call 0x593e at 0x17 in Landroidx/work/impl/constraints/trackers/NetworkStateTracker;.<init>

And at the time the logs are taken by the user, the app dumps the current work queue, and sure enough there is a periodic work still enqueued (but this periodic work doesn't run when network has been restored and never seems to run again)... this is the log entry here:

01-29 21:11:33.875 D/Meteogram_ConfigActvty(29556): ********** LOG OF WORK QUEUE **********
01-29 21:11:34.175 D/Meteogram_Worker(29556): workInfo: WorkInfo{mId='8eaea234-7001-479e-ae88-2d628f57a62b', mState=ENQUEUED, mOutputData=androidx.work.Data@0, mTags=[widget-43443, com.cloud3squared.meteogram.MeteogramWorker, periodic]}
01-29 21:11:34.175 D/Meteogram_Worker(29556): workInfo: WorkInfo{mId='c43771c7-6b71-49fb-a2ee-c8486b37b798', mState=SUCCEEDED, mOutputData=androidx.work.Data@0, mTags=[widget-43443, com.cloud3squared.meteogram.MeteogramWorker, onetime]}
01-29 21:11:34.175 D/Meteogram_ConfigActvty(29556): ********** END OF WORK QUEUE **********

The above is logged by the following function, with the widget id passed in to get the tag (same method as in previous post), and filtering the work on this tag:

    static void logWorkQueue(int appWidgetId) {
        String tag = getWorkTag(appWidgetId);
        WorkManager workManager = WorkManager.getInstance();
        ListenableFuture<List<WorkInfo>> workInfos = workManager.getWorkInfosByTag(tag);
        try {
            List<WorkInfo> workInfoList = workInfos.get();
            for (WorkInfo workInfo : workInfoList) {
                Log.d(TAG, "workInfo: " + workInfo.toString());
            }
        } catch (ExecutionException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

Further update: it's just possible that this isn't specifically related to the network constraint.  This user has now reported that a even a non-network-constrained periodic work seems to have stopped (as manifested by the associated widget no longer auto updating).  I don't have logs for this, but maybe some of those log entries around 21:07 from the previous logcat might also be occurring even for periodic work that is not network constrained?  Until now, the non-network-constrained periodic work does however seem to be more reliable... network constrained periodic work seems consistently to stop after a period without network.

Thanks for the details. I should have updated the thread yesterday. I think I know what might be going on.
I have a pending CL as well. It should be fixed in the next beta.

https://android-review.googlesource.com/c/platform/frameworks/support/+/889839

This is excellent news... thanks for checking it out so carefully, and fingers crossed... look forward to the next beta and will report back either way.  

Any ETA on the next beta?

Keep up the good work on this really useful library!

Project: platform/frameworks/support
Branch: androidx-master-dev

commit 177837514245d5bb24cc8e9bc27f20503e23e308
Author: Rahul Ravikumar <rahulrav@google.com>
Date:   Tue Jan 29 17:50:14 2019

    Fixes ConstraintsCommandHandler that correctly enables and disables proxies.
   
    * ConstraintsCommandHandler was only looking at work that was eligible to be scheduled.
      This is incorrect, as we may have Workers that are scheduled, but are pending execution due
      to an unmet constraint. We need to look at all scheduled work which is unfinished.
    * The only reason why this worked before with integration tests was due to one of 2 reasons:
      * The process was in the foreground anyway, and there was an active WorkConstraintTracker.
      * Enabling / disabling airplane mode (to test Network Constraints) would have caused a
        TIME_SET broadcast in some cases where we reschedule everything (RescheduleReceiver).
   
    Test: Updated SystemAlarmDispatcherTests & integration tests on API 22 with the Battery charging
          Constraint that has no other side effects.
   
    Change-Id: Ib973931b47be915dd8f533e1e651b6c1a29843f9
    Fixes: b/123379508

M       work/workmanager/src/androidTest/java/androidx/work/impl/background/systemalarm/SystemAlarmDispatcherTest.java
M       work/workmanager/src/main/java/androidx/work/impl/background/systemalarm/ConstraintsCommandHandler.java
M       work/workmanager/src/main/java/androidx/work/impl/background/systemalarm/RescheduleReceiver.java
M       work/workmanager/src/main/java/androidx/work/impl/model/WorkSpecDao.java

https://android-review.googlesource.com/889839
https://goto.google.com/android-sha1/177837514245d5bb24cc8e9bc27f20503e23e308

Any ETA for beta04?  Am desperate to try it out..

Looking at early next week.

OK, thanks!

I'm still seeing this problem, or seeing it again.  I'm monitoring (at a server) requests from the device of the same user (Android 4.3), and can see that a periodic work request seems to run OK for a time, but when the device is idle and without network, when the device is given network again, not only does the late work not run, but no work ever seems to run again.  And a log output of all work does still show that a periodic work is enqueued, but it just never seems to run again.

Are the tests you ran when fixing this before still working?  Or has something subsequently messed it up?  Or maybe I'm seeing another issue altogether.

Please give us a sample project.  I think that's the simplest way for us to take a look at exactly what's happening for you.  Which device are you running this on?

Device is as per #16 -- Samsung Galaxy S3 running Android 4.3

Can you reproduce the same behavior using a stock Android device/emulator?

Not as yet; am trying now.

Is there any way of querying a bit more information about enqueued work requests?  The most information I can get at the moment is something like:

02-15 17:37:31.535 D/My_Worker(11599): workInfo: WorkInfo{mId='00db2ea7-07d6-42a1-a506-a26ff8e3217d', mState=ENQUEUED, mOutputData=androidx.work.Data@0, mTags=[com.example.myapp.MyWorker, widget-43806, periodic]}
02-15 17:37:31.535 D/My_Worker(11599): workInfo: WorkInfo{mId='08bce448-5d4a-4a17-89d2-6fdce6ef1091', mState=SUCCEEDED, mOutputData=androidx.work.Data@0, mTags=[com.example.myapp.MyWorker, widget-43806, backstop]}

And the only reason I can even tell if the work item is periodic or not is because I tag it with 'periodic' if it is.  Otherwise, all I get is a work ID and a state... would be really handy to know when it last ran, when it is due to run next, etc.  Would be very useful not least for debugging purposes.

As suggested: https://issuetracker.google.com/issues/123222823

You need to enable verbose logs. That will tell us a lot more.

Also, if you are going into a SUCCEEDED state, that mean your Worker is actually not periodic. PeriodicWork never goes into a terminal state unless it has been cancelled.
It always goes back to ENQUEUED for the next interval.

As per #9 above.  Yes, the 4.3 user is currently running a test version with this verbose logging enabled to see if I can get something interesting from it.

For reference all WorkManager logs are prefixed with a "WM-".

#39: indeed, but the SUCCEEDED entry is a onetime worker... it is not tagged with 'periodic' but only 'backstop' (which is a onetime request).

I've seen this behaviour now on the 4.3 emulator.  I started a periodic work request (for updating a widget) with network constraint and interval of one hour (flex of 30 mins).  I put the emulator into airplane mode and came back some time later.  On disabling airplane mode the widget did update immediately but the trigger was ON_ENABLED for the widget and not the late work request.  Thereafter, I left the widget for an hour, and nothing happened... the periodic work that was meant to trigger an update hasn't happened.  I meant to dump the alarm state with "adb shell dumpsys alarm" but unfortunately I reset the widget before doing so.  Will next time.  Sorry it's a bit nebulous thus far... I will try to bring more concrete information next time but there's definitely still something not right with network-constrained periodic work on Android 4.3.

Yes, please send us a sample project when you have it.

I'll try, but am short of time over next few days.  Tempting just to set minsdk to 21 and be done with it!

I have some logs to post... would that be of assistance in lieu of a sample project for now?

Yes, please send them

Logs attached, from the user's Android 4.3 device.  During the period in which the logs were generated, the periodic work concerned should have fired (to cause a widget to update) at least twice but failed to do so.  The periodic work was on a 30 minute interval, with 15 minute flex interval.  With network constraint.  After enqueuing the periodic work, the device was without network for a time, then network was restored.  But periodic work failed to trigger.  I schedule the periodic work basically as described in #22 *except* that I now use a flex interval (to address a double running of work as per another issue).

There seems to be lots of "D/WM-" stuff in there that refers to constraints... hopefully something of interest.

Some further information which may be of interest...

To see if it made a difference, I went back to NOT using a flexInterval, i.e. using:

    workRequestBuilder = new PeriodicWorkRequest.Builder(MeteogramWorker.class, interval, TimeUnit.MILLISECONDS);

INSTEAD of:

    workRequestBuilder = new PeriodicWorkRequest.Builder(MeteogramWorker.class, interval, TimeUnit.MILLISECONDS, flexInterval, TimeUnit.MILLISECONDS);

And without using flexInterval, behaviour seems to be FAR more reliable so far.  The periodic work (with network constraint) has so far survived some degree of testing (on the user device): letting the periodic work fire a few times, then turning off network for one or more scheduled firings, then network back on... and each time so far the "missed" work runs straight away.

It's far from an extensive test yet, but it seems like it can't be coincidence... the above change (to NOT use flexInterval) is all I've changed, and it is performing better so far.

After *this current* issue (i.e. #123379508) was marked fixed on 30 Jan, a change was made to WorkManager flexInterval behaviour on 13 Feb as per the following:

https://issuetracker.google.com/issues/124274584

Would it have anything to do with that?  Between #123379508 being fixed (30 Jan) and #124274584 being fixed (13 Feb), I think that things were going well.  It seemed to be after #124274584 was fixed that things went downhill again.  Again, I can't be certain, but it's a lead anyway.

P.S. the only reason I started using flexInterval in the first place was to address the issue reported here:

https://issuetracker.google.com/issues/123804615#comment8

BTW the logs in #48 were BEFORE I made the change mentioned in #49, i.e. using flexInterval

I looked at the logs. It looks like the ConstraintTracker correctly tries to start the work, but the device subsequently reports that the constraints are unmet when trying to run the Worker.

02-17 14:02:25.255 D/WM-ConstraintTracker( 1608): NetworkStateTracker: initial state = [ Connected=false Validated=false Metered=false NotRoaming=false ]. This happens twice, like you described.

This is expected behavior. I think your disabling of flex has nothing to do with why the Worker was running when you had flex. I think the device does not have access to a reliable network as far as I can tell.