Test with Coverage is not working with robolectric [319010149]

Assigned

Bug

Status Update

No update yet.

Description

ga...@linecorp.com

created issue #1

Jan 8, 2024 04:22AM

DESCRIBE THE ISSUE IN DETAIL:

I tried to setup unittest as https://robolectric.org/getting-started/
Then Test Coverage Feature of Android Studio is not working for unittests.

Currently, isIncludeAndroidResources is set true to access resources/assets.

After set to false, test was passed without problem.

STEPS TO REPRODUCE:
1.

$ git clone https://github.com/ganadist/VersionCodeDemo -b coverage_test

Open Project with Android Studio
Open ExampleUnitTest.kt
Perform Run ExampleUnitTest with Coverage from gutter

ATTACH SCREENSHOTS/RECORDINGS OF THE ISSUE

ATTACH LOG FILES (Select Help > Show Log in Files, or Show Log in Finder on a Mac)

> Task :app:testDebugUnitTest
---- IntelliJ IDEA coverage runner ---- 
Line coverage ...
include patterns:
exclude annotations patterns:
.*Generated.*

Bad return type
Exception Details:
  Location:
    android/content/res/ResourcesImpl.$$robo$$android_content_res_ResourcesImpl$loadComplexColorForCookie(Landroid/content/res/Resources;Landroid/util/TypedValue;ILandroid/content/res/Resources$Theme;)Landroid/content/res/ComplexColor; @623: areturn
  Reason:
    Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'android/content/res/ComplexColor' (from method signature)
  Current Frame:
    bci: @623
    flags: { }
    locals: { 'android/content/res/ResourcesImpl', 'android/content/res/Resources', 'android/util/TypedValue', integer, 'android/content/res/Resources$Theme', '[I', 'java/lang/String', 'java/lang/Object', 'android/content/res/XmlResourceParser', 'android/util/AttributeSet', integer, 'java/lang/String' }
    stack: { 'java/lang/Object' }
  Bytecode:
    0000000: 1298 c000 993a 0519 0511 019f 5c2e 0460
    0000010: 4f2c b401 bec7 002c 1905 1101 a05c 2e04
    0000020: 604f bb04 8b59 bb01 2a59 b701 2b13 048d
    0000030: b601 312c b402 d8b6 0194 b601 3bb7 048e
    0000040: bf19 0511 01a1 5c2e 0460 4f2c b401 beb9
    0000050: 01bf 0100 3a06 1905 1101 a25c 2e04 604f
    0000060: 013a 0719 0511 01a3 5c2e 0460 4f14 01e3
    0000070: 1906 b801 ec19 0511 01a4 5c2e 0460 4f19
    0000080: 0613 03a7 b603 ab99 0178 1905 1101 a55c
    0000090: 2e04 604f 2a19 061d 2cb4 01bc 1304 90b6
    00000a0: 0416 3a08 1905 1101 a65c 2e04 604f 1908
    00000b0: b804 963a 0919 0511 01a7 5c2e 0460 4f19
    00000c0: 08b9 0499 0100 5936 0a05 9f00 0c15 0a04
    00000d0: 9f00 06a7 ffe2 1905 1101 a85c 2e04 604f
    00000e0: 150a 059f 0018 1905 1101 a95c 2e04 604f
    00000f0: bb04 1259 1304 9bb7 049c bf19 0511 01aa
    0000100: 5c2e 0460 4f19 08b9 049f 0100 3a0b 1905
    0000110: 1101 ab5c 2e04 604f 190b 1304 a1b6 01a7
    0000120: 9900 1c19 0511 01ac 5c2e 0460 4f2b 1908
    0000130: 1909 1904 b804 a73a 07a7 002e 1905 1101
    0000140: ad5c 2e04 604f 190b 1304 a9b6 01a7 9900
    0000150: 1919 0511 01ae 5c2e 0460 4f2b 1908 1909
    0000160: 1904 b804 ac3a 0719 0511 01af 5c2e 0460
    0000170: 4f19 08b9 041d 0100 1905 1101 b05c 2e04
    0000180: 604f a700 d119 0511 01b1 5c2e 0460 4f3a
    0000190: 0819 0511 01b2 5c2e 0460 4f14 01e3 b802
    00001a0: 8319 0511 01b3 5c2e 0460 4fbb 000d 59bb
    00001b0: 012a 59b7 012b 1301 c5b6 0131 1906 b601
    00001c0: 3113 04ae b601 311d 1905 1101 b45c 2e04
    00001d0: 604f b801 37b6 0131 b601 3bb7 013e 3a09
    00001e0: 1905 1101 b55c 2e04 604f 1909 1908 b601
    00001f0: d957 1905 1101 b65c 2e04 604f 1909 bf19
    0000200: 0511 01b7 5c2e 0460 4f14 01e3 b802 8319
    0000210: 0511 01b8 5c2e 0460 4fbb 000d 59bb 012a
    0000220: 59b7 012b 1301 c5b6 0131 1906 b601 3113
    0000230: 03f2 b601 311d 1905 1101 b95c 2e04 604f
    0000240: b801 37b6 0131 1304 7bb6 0131 b601 3bb7
    0000250: 013e bf19 0511 01ba 5c2e 0460 4f14 01e3
    0000260: b802 8319 0511 01bb 5c2e 0460 4f19 07b0
    0000270:                                        
  Exception Handler Table:
    bci [138, 376] => handler: 389
  Stackmap Table:
    append_frame(@65,Object[#153])
    full_frame(@181,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Null,Object[#1052],Object[#1203]},{})
    append_frame(@214,Integer)
    same_frame(@251)
    append_frame(@316,Object[#213])
    full_frame(@359,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Object[#4],Object[#1052],Object[#1203],Integer,Object[#213]},{})
    full_frame(@389,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Object[#4]},{Object[#342]})
    full_frame(@511,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Null},{})
    full_frame(@595,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Object[#4],Object[#1052],Object[#1203],Integer,Object[#213]},{})

java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    android/content/res/ResourcesImpl.$$robo$$android_content_res_ResourcesImpl$loadComplexColorForCookie(Landroid/content/res/Resources;Landroid/util/TypedValue;ILandroid/content/res/Resources$Theme;)Landroid/content/res/ComplexColor; @623: areturn
  Reason:
    Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'android/content/res/ComplexColor' (from method signature)
  Current Frame:
    bci: @623
    flags: { }
    locals: { 'android/content/res/ResourcesImpl', 'android/content/res/Resources', 'android/util/TypedValue', integer, 'android/content/res/Resources$Theme', '[I', 'java/lang/String', 'java/lang/Object', 'android/content/res/XmlResourceParser', 'android/util/AttributeSet', integer, 'java/lang/String' }
    stack: { 'java/lang/Object' }
  Bytecode:
    0000000: 1298 c000 993a 0519 0511 019f 5c2e 0460
    0000010: 4f2c b401 bec7 002c 1905 1101 a05c 2e04
    0000020: 604f bb04 8b59 bb01 2a59 b701 2b13 048d
    0000030: b601 312c b402 d8b6 0194 b601 3bb7 048e
    0000040: bf19 0511 01a1 5c2e 0460 4f2c b401 beb9
    0000050: 01bf 0100 3a06 1905 1101 a25c 2e04 604f
    0000060: 013a 0719 0511 01a3 5c2e 0460 4f14 01e3
    0000070: 1906 b801 ec19 0511 01a4 5c2e 0460 4f19
    0000080: 0613 03a7 b603 ab99 0178 1905 1101 a55c
    0000090: 2e04 604f 2a19 061d 2cb4 01bc 1304 90b6
    00000a0: 0416 3a08 1905 1101 a65c 2e04 604f 1908
    00000b0: b804 963a 0919 0511 01a7 5c2e 0460 4f19
    00000c0: 08b9 0499 0100 5936 0a05 9f00 0c15 0a04
    00000d0: 9f00 06a7 ffe2 1905 1101 a85c 2e04 604f
    00000e0: 150a 059f 0018 1905 1101 a95c 2e04 604f
    00000f0: bb04 1259 1304 9bb7 049c bf19 0511 01aa
    0000100: 5c2e 0460 4f19 08b9 049f 0100 3a0b 1905
    0000110: 1101 ab5c 2e04 604f 190b 1304 a1b6 01a7
    0000120: 9900 1c19 0511 01ac 5c2e 0460 4f2b 1908
    0000130: 1909 1904 b804 a73a 07a7 002e 1905 1101
    0000140: ad5c 2e04 604f 190b 1304 a9b6 01a7 9900
    0000150: 1919 0511 01ae 5c2e 0460 4f2b 1908 1909
    0000160: 1904 b804 ac3a 0719 0511 01af 5c2e 0460
    0000170: 4f19 08b9 041d 0100 1905 1101 b05c 2e04
    0000180: 604f a700 d119 0511 01b1 5c2e 0460 4f3a
    0000190: 0819 0511 01b2 5c2e 0460 4f14 01e3 b802
    00001a0: 8319 0511 01b3 5c2e 0460 4fbb 000d 59bb
    00001b0: 012a 59b7 012b 1301 c5b6 0131 1906 b601
    00001c0: 3113 04ae b601 311d 1905 1101 b45c 2e04
    00001d0: 604f b801 37b6 0131 b601 3bb7 013e 3a09
    00001e0: 1905 1101 b55c 2e04 604f 1909 1908 b601
    00001f0: d957 1905 1101 b65c 2e04 604f 1909 bf19
    0000200: 0511 01b7 5c2e 0460 4f14 01e3 b802 8319
    0000210: 0511 01b8 5c2e 0460 4fbb 000d 59bb 012a
    0000220: 59b7 012b 1301 c5b6 0131 1906 b601 3113
    0000230: 03f2 b601 311d 1905 1101 b95c 2e04 604f
    0000240: b801 37b6 0131 1304 7bb6 0131 b601 3bb7
    0000250: 013e bf19 0511 01ba 5c2e 0460 4f14 01e3
    0000260: b802 8319 0511 01bb 5c2e 0460 4f19 07b0
    0000270:                                        
  Exception Handler Table:
    bci [138, 376] => handler: 389
  Stackmap Table:
    append_frame(@65,Object[#153])
    full_frame(@181,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Null,Object[#1052],Object[#1203]},{})
    append_frame(@214,Integer)
    same_frame(@251)
    append_frame(@316,Object[#213])
    full_frame(@359,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Object[#4],Object[#1052],Object[#1203],Integer,Object[#213]},{})
    full_frame(@389,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Object[#4]},{Object[#342]})
    full_frame(@511,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Null},{})
    full_frame(@595,{Object[#2],Object[#15],Object[#441],Integer,Object[#23],Object[#153],Object[#213],Object[#4],Object[#1052],Object[#1203],Integer,Object[#213]},{})

	at android.content.res.Resources.__constructor__(Resources.java:319)
	at android.content.res.Resources.<init>(Resources.java:318)
	at org.robolectric.shadows.ShadowResources.getSystem(ShadowResources.java:86)
	at android.content.res.Resources.getSystem(Resources.java)
	at org.robolectric.android.internal.RoboMonitoringInstrumentation.onCreate(RoboMonitoringInstrumentation.java:81)
	at org.robolectric.android.internal.AndroidTestEnvironment.createInstrumentation(AndroidTestEnvironment.java:592)
	at org.robolectric.android.internal.AndroidTestEnvironment.setUpApplicationState(AndroidTestEnvironment.java:204)
	at org.robolectric.RobolectricTestRunner.beforeTest(RobolectricTestRunner.java:340)
	at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$2(SandboxTestRunner.java:281)
	at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:101)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)


Class transformation time: 1.189345792s for 4677 classes or 2.5429672696172757E-4s per class
com.example.myapplication.ExampleUnitTest > addition_isCorrect FAILED
    java.lang.VerifyError at Resources.java:319
1 test completed, 1 failed
> Task :app:testDebugUnitTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:testDebugUnitTest'.
> There were failing tests. See the report at: file:///home/ganadist/src/build/AndroidStudioProjects/MyApplication5/app/build/reports/tests/testDebugUnitTest/index.html
* Try:
> Run with --scan to get full insights.
BUILD FAILED in 3s
24 actionable tasks: 3 executed, 21 up-to-date
1:01:20 PM: Execution finished ':app:testDebugUnitTest --tests "com.example.myapplication.ExampleUnitTest"'.

IMPORTANT: Please read https://developer.android.com/studio/report-bugs.html carefully and supply all required information.

Studio Build: 2023.1.1 / 2023.2.1 / 2023.3.1
Version of Gradle Plugin: 8.2.1
Version of Gradle: 8.2
Version of Java: OpenJDK 17
OS: Any

Comments

ga...@linecorp.com <ga...@linecorp.com> #2Jan 8, 2024 04:59AM

I tried adding screenshotTestImplementation(libs.androidx.window) but it didn't fix it.

hm...@google.com <hm...@google.com> Jan 8, 2024 12:48PM

Assigned to an...@google.com.

ga...@linecorp.com <ga...@linecorp.com> #3Jan 8, 2024 06:38PM

This is likely a duplicate of b/388773416. Could you try adding libs.androidx.window dependency to implementation to see if it works?

ra...@google.com <ra...@google.com> Jan 10, 2024 01:35AM

Reassigned to an...@google.com.

hu...@google.com <hu...@google.com> Jan 11, 2024 10:17PM

Reassigned to hu...@google.com.

ho...@google.com <ho...@google.com> #4Jan 11, 2024 10:22PM

I believe the root cause of this issue is in IntelliJ, since the built-in coverage tool is part of IntelliJ.

That being said, executing coverage on Android classes (android., libcore., etc..) is unnecessary and wasteful.

hu...@google.com <hu...@google.com> #5Jan 11, 2024 10:39PM

Reassigned to cm...@google.com.

+1 for excluding Android classes for coverage by default. Chris, do you know if this should be done by testCoverage DSL (by adding a new property) or by simply using jacoco plugin's DSL?

ho...@google.com <ho...@google.com> #6Jan 11, 2024 10:43PM

FYI Android framework classes have the packages dalvik., libcore., android., and com.android.internal.

cm...@google.com <cm...@google.com> #7Jan 15, 2024 02:26PM

Reassigned to hu...@google.com.

The Android Gradle plugin doesn't have DSL for this, but developers can set excludes using the JacocoTaskExtension on the test task

We could add those hardcoded excludes in the Android Gradle plugin here