Espresso semi-parallel test execution fails on static objects [37082857]

Fixed

Bug

[AOSP] duplicate

[AOSP] assigned

Status Update

No update yet.

Description

v....@gmail.com

created issue #1

Feb 22, 2016 10:15AM

Library used: com.android.support.test.espresso:espresso-core:2.2.1
Version used: 2.2.1

What steps will reproduce the problem?
1. Have atleast 3 tests
2. Log messages in activity onCreate and onDestroy
3. Observe onCreate -> onDestroy messages not going in order (onCreate -> onDestroy, onCreate -> onDestroy, onCreate -> onDestroy)

How are you running your tests (via Android Studio, Gradle, adb, etc.)?
Android Studio 1.5.1

What is the expected output? What do you see instead?
Log messages going sequentially (onCreate -> onDestroy, onCreate -> onDestroy, onCreate -> onDestroy)

I am trying to use espresso and junit4. The problem seems to be that I have code that is statically initialized in activity.onCreate and deinitialized in activity.onDestroy to prevent leaks, etc.

Now when I run espresso, the tests seem to run in "parallel". I added log class to activity onCreate and onDestroy.

What I am seeing is

onCreate = example.package.MainActivity@ABC
onCreate = example.package.MainActivity@JKL
onDestroy = example.package.MainActivity@ABC
onCreate = example.package.MainActivity@XYZ
onDestroy = example.package.MainActivity@JKL
onDestroy = example.package.MainActivity@XYZ

**and ofcourse it fails on NPE in third test because the second test's onDestroy ran after third's onCreate (which null-ed out the static code)**

Is this behaviour normal? Can I force espresso to teardown activity instance first and only then start a new one? Thanks!

Comments

sl...@google.com <sl...@google.com> #2Feb 29, 2016 09:11PM

Further testing shows 11.0.0 works in this setup and 11.2.0 is the first version this appears.

v....@gmail.com <v....@gmail.com> #3Feb 29, 2016 09:23PM

This is expected. Both libraries have a java resource called 'protobuf.meta', which either the library authors need to stop shipping or the build author needs to pick one. See

https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html

However, if both libraries actually depend on that java resource you are in trouble.

ai...@gmail.com <ai...@gmail.com> #4Apr 4, 2016 05:49AM

I can confirm that specifying exclude or pick-first on protobuf.meta results in a successful build and no *immediate* problems. However, the contents of these two files are completely different and without understanding the purpose of these files (I assume related to protocol buffer data structure) it feels very treacherous to remove them.

nk...@google.com <nk...@google.com> Apr 4, 2016 06:23PM

Assigned to nk...@google.com.

ar...@gmail.com <ar...@gmail.com> #5Jul 4, 2016 02:46PM

Stephan, do you know what that file does and what changed that meant that it has started being bundled as a java resource in the testing support library?

nk...@google.com <nk...@google.com> #6Jul 6, 2016 08:51PM

I see the same issue in a module referencing com.google.android.gms:play-services-location and com.google.android.gms:play-services-maps after integrating com.google.firebase:firebase-core and com.google.firebase:firebase-perf in the app.

I am running `11.6.2` of all libraries as well as Gradle 4.3.1-all and support lib 26.1.0

be...@gmail.com <be...@gmail.com> #8Aug 10, 2016 02:21AM

This will be fixed in the next release of Espresso v3.0.2.

We will be stripping off protobuf.meta file from the release binaries since they are not actually needed at run time.

fr...@gmail.com <fr...@gmail.com> #9Feb 3, 2017 09:50AM

Hi, is there any news when the next release will be available? It's been almost a year since 2.2.2 and half a year since #6.

xa...@gmail.com <xa...@gmail.com> #10Jun 19, 2017 01:10PM

Hi, is there any update on this bug?

vi...@gmail.com <vi...@gmail.com> #11Jun 20, 2017 11:35AM

Hi guys,
please give us an update about the fix for this :)

It's making making using Espresso extremely unpleasant

nk...@google.com <nk...@google.com> #12Jun 26, 2017 07:39PM

Marked as fixed.

Fixed in the next Runner/Rules v1.0 release.

to...@gmail.com <to...@gmail.com> #13Jul 20, 2017 09:27PM

It's been about a month. When will the v1.0 release of Rules/Runner be released?

su...@gmail.com <su...@gmail.com> #14Jul 26, 2017 05:01AM

1.0 is out. Anyone tests that out yet?

https://android-developers.googleblog.com/2017/07/android-testing-support-library-10-is.html?m=1

to...@gmail.com <to...@gmail.com> #15Jul 26, 2017 04:33PM

We just tried to update our runner/rules, but test suite crashed :) I guess we may need to update Espresso to 3.0 as well.

jc...@gmail.com <jc...@gmail.com> #16Apr 17, 2018 06:31AM

Any updates on this bug? Is this fixed?

nk...@google.com <nk...@google.com> #17Apr 17, 2018 07:28PM

Yes, this was fixed for a while now. Please give it a try with the latest version of our libraries.

nk...@google.com <nk...@google.com> #18Apr 17, 2018 07:28PM

Also we just made another public release, see release notes here:

https://developer.android.com/topic/libraries/testing-support-library/release-notes.html

da...@gmail.com <da...@gmail.com> #19May 23, 2018 10:22AM

Any other updates about this bug? I'm still issuing it with:
- androidTestImplementation 'com.android.support.test:runner:1.0.2'
- androidTestImplementation 'com.android.support.test:rules:1.0.2'
- androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
Tests always passes individually.
Running the suite instead, they have an unpredictable behavior: sometimes they passes, sometimes do not.

gu...@gmail.com <gu...@gmail.com> #20Jul 9, 2018 05:31PM

For some reason with databinding and static class I cannot debug code in ViewModel at every 2 test. Individually everything is just fine. espresso 3.0.2

gu...@gmail.com <gu...@gmail.com> #21Jul 10, 2018 11:35AM

This is works like this (everything is without package). In my View which is Activity I call onStart() and onPause() of ViewModel. In xml I have button with enabled set to MutableLiveData<Boolean> property in ViewModel. I write in Kotlin

//This is button enabled field update, each time when this MutableLiveData<Boolean> is called it prints this.toString()
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180
PersonalViewModel@52cb180

//Second test start onStart() of ViewModel
START: PersonalViewModel@43321e0

PersonalViewModel@43321e0 //Calls button MutableLiveData<Boolean> for enabled property only ONCE

//Second test onPause() of ViewModel
PAUSE: PersonalViewModel@43321e0

//Third test start onStart() of ViewModel
START: PersonalViewModel@dc72c50
PersonalViewModel@dc72c50 //Calls button MutableLiveData<Boolean> for enabled property only ONCE
PAUSE: PersonalViewModel@dc72c50

When the second test starts it seems like it calls MutableLiveData<Boolean> property only once for button and then it fails to bind or something, because button is not updating anymore. Individually all tests work.