Espresso semi-parallel test execution fails on static objects [37082857]

Fixed

Bug

[AOSP] duplicate

[AOSP] assigned

Status Update

No update yet.

Description

v....@gmail.com

created issue #1

Feb 22, 2016 10:15AM

Library used: com.android.support.test.espresso:espresso-core:2.2.1
Version used: 2.2.1

What steps will reproduce the problem?
1. Have atleast 3 tests
2. Log messages in activity onCreate and onDestroy
3. Observe onCreate -> onDestroy messages not going in order (onCreate -> onDestroy, onCreate -> onDestroy, onCreate -> onDestroy)

How are you running your tests (via Android Studio, Gradle, adb, etc.)?
Android Studio 1.5.1

What is the expected output? What do you see instead?
Log messages going sequentially (onCreate -> onDestroy, onCreate -> onDestroy, onCreate -> onDestroy)

I am trying to use espresso and junit4. The problem seems to be that I have code that is statically initialized in activity.onCreate and deinitialized in activity.onDestroy to prevent leaks, etc.

Now when I run espresso, the tests seem to run in "parallel". I added log class to activity onCreate and onDestroy.

What I am seeing is

onCreate = example.package.MainActivity@ABC
onCreate = example.package.MainActivity@JKL
onDestroy = example.package.MainActivity@ABC
onCreate = example.package.MainActivity@XYZ
onDestroy = example.package.MainActivity@JKL
onDestroy = example.package.MainActivity@XYZ

**and ofcourse it fails on NPE in third test because the second test's onDestroy ran after third's onCreate (which null-ed out the static code)**

Is this behaviour normal? Can I force espresso to teardown activity instance first and only then start a new one? Thanks!

Comments

sl...@google.com <sl...@google.com> #2Feb 29, 2016 09:11PM

Thank you for reporting this issue. For us to further investigate this issue, please provide the following additional information:

Android build
Which Android build are you using? (e.g. OPP1.170223.012)

Please provide sample project or apk to reproduce the issue. Also mention the steps to be followed for reproducing the issue with the given sample project or apk.

Frequency
How frequently does this issue occur? (e.g 100% of the time, 10% of the time)

Android bug report capturing (kindly share complete bugreport)
After reproducing the issue, press the volume up, volume down, and power button simultaneously. This will capture a bug report on your device in the “bug reports” directory.

Alternate method
After reproducing the issue, navigate to “developer settings”, ensure “USB debugging” is enabled, then enable “Bug report shortcut”. Capture bug report by holding the power button and selecting the “Take bug report” option.

Screen capture of the issue
Press the volume down and power buttons simultaneously. The image will appear in your gallery. Attach the screenshot file to this issue.

Note: Please upload the files to google drive and share the folder to android-bugreport@google.com, then share the link here.

v....@gmail.com <v....@gmail.com> #3Feb 29, 2016 09:23PM

Android build: N/A - firebase crashlytics doesn't provide that information. It happens on both Android 9 and Android 10.
Sample: N/A - it's a race condition can't reproduce it consistently
Frequency: ~1 %
Android bug report capturing: N/A - firebase crashlytics doesn't provide that information

Problem:
In BiometricFragment the mBundle is not always set - so when isDeviceCredentialAllowed is called the mBundle.getBoolean(BiometricPrompt.KEY_ALLOW_DEVICE_CREDENTIAL, false) throws the NullPointerException.

ai...@gmail.com <ai...@gmail.com> #4Apr 4, 2016 05:49AM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit 0b2dee89f68e9ed41c9552a93fee72364253af41
Author: Curtis Belmonte <curtislb@google.com>
Date: Wed Nov 06 10:00:17 2019

Fix possible NPE with BiometricFragment method

The current implementation of
BiometricFragment#setDeviceCredentialAllowed() tries to get a boolean
from mBundle without first checking to see if mBundle is null,
potentially resulting in a NullPointerException. This commit adds in the
null check in order to make calling this method safer. It also adds a
unit test case to exercise this behavior.

Test: ./gradlew biometric:connectedAndroidTest
Fixes: 142599311
Change-Id: I18e03f926ff03c53f8cf1812fbad46ea75db6a32

M biometric/src/androidTest/java/androidx/biometric/BiometricFragmentTest.java
M biometric/src/main/java/androidx/biometric/BiometricFragment.java

https://android-review.googlesource.com/1159943

nk...@google.com <nk...@google.com> Apr 4, 2016 06:23PM

Assigned to nk...@google.com.

ar...@gmail.com <ar...@gmail.com> #5Jul 4, 2016 02:46PM

The following changes were cherrypicked through Coastguard (

https://android-build.googleplex.com/coastguard/)
Release Track: (

https://android-build.googleplex.com/coastguard/dashboard/5176090922385408/#/history/all)
Created: 2019-11-26 18:05:22.006209+00:00
Changes: aosp/1159943

nk...@google.com <nk...@google.com> #6Jul 6, 2016 08:51PM

This will be fixed in next release. Meanwhile, look at

comment #3

for a horrible but temporary solution. Thanks!

be...@gmail.com <be...@gmail.com> #8Aug 10, 2016 02:21AM

Similar or related issue to this and

https://code.google.com/p/android/issues/detail?id=216442 ? Executing a set of tests in a single class seem to work but running a suite of tests (@Suite junit4) seems to have similar failures.

fr...@gmail.com <fr...@gmail.com> #9Feb 3, 2017 09:50AM

Hi, is there any news when the next release will be available? It's been almost a year since 2.2.2 and half a year since #6.

xa...@gmail.com <xa...@gmail.com> #10Jun 19, 2017 01:10PM

Hi, is there any update on this bug?

vi...@gmail.com <vi...@gmail.com> #11Jun 20, 2017 11:35AM

Hi guys,
please give us an update about the fix for this :)

It's making making using Espresso extremely unpleasant

nk...@google.com <nk...@google.com> #12Jun 26, 2017 07:39PM

Marked as fixed.

Fixed in the next Runner/Rules v1.0 release.

to...@gmail.com <to...@gmail.com> #13Jul 20, 2017 09:27PM

It's been about a month. When will the v1.0 release of Rules/Runner be released?

su...@gmail.com <su...@gmail.com> #14Jul 26, 2017 05:01AM

1.0 is out. Anyone tests that out yet?

https://android-developers.googleblog.com/2017/07/android-testing-support-library-10-is.html?m=1

to...@gmail.com <to...@gmail.com> #15Jul 26, 2017 04:33PM

We just tried to update our runner/rules, but test suite crashed :) I guess we may need to update Espresso to 3.0 as well.

jc...@gmail.com <jc...@gmail.com> #16Apr 17, 2018 06:31AM

Any updates on this bug? Is this fixed?

nk...@google.com <nk...@google.com> #17Apr 17, 2018 07:28PM

Yes, this was fixed for a while now. Please give it a try with the latest version of our libraries.

nk...@google.com <nk...@google.com> #18Apr 17, 2018 07:28PM

Also we just made another public release, see release notes here:

https://developer.android.com/topic/libraries/testing-support-library/release-notes.html

da...@gmail.com <da...@gmail.com> #19May 23, 2018 10:22AM

Any other updates about this bug? I'm still issuing it with:
- androidTestImplementation 'com.android.support.test:runner:1.0.2'
- androidTestImplementation 'com.android.support.test:rules:1.0.2'
- androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
Tests always passes individually.
Running the suite instead, they have an unpredictable behavior: sometimes they passes, sometimes do not.

gu...@gmail.com <gu...@gmail.com> #20Jul 9, 2018 05:31PM

For some reason with databinding and static class I cannot debug code in ViewModel at every 2 test. Individually everything is just fine. espresso 3.0.2

gu...@gmail.com <gu...@gmail.com> #21Jul 10, 2018 11:35AM

This is works like this (everything is without package). In my View which is Activity I call onStart() and onPause() of ViewModel. In xml I have button with enabled set to MutableLiveData<Boolean> property in ViewModel. I write in Kotlin

//This is button enabled field update, each time when this MutableLiveData<Boolean> is called it prints this.toString()
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180

//Second test start onStart() of ViewModel
START: PersonalViewModel@43321e0

PersonalViewModel@43321e0 //Calls button MutableLiveData<Boolean> for enabled property only ONCE

//Second test onPause() of ViewModel
PAUSE: PersonalViewModel@43321e0

//Third test start onStart() of ViewModel
START: PersonalViewModel@dc72c50
PersonalViewModel@dc72c50 //Calls button MutableLiveData<Boolean> for enabled property only ONCE
PAUSE: PersonalViewModel@dc72c50

When the second test starts it seems like it calls MutableLiveData<Boolean> property only once for button and then it fails to bind or something, because button is not updating anymore. Individually all tests work.