Status Update
Comments
js...@google.com <js...@google.com>
mk...@google.com <mk...@google.com> #2
ja...@nimbus.co <ja...@nimbus.co> #3
-keepnames class com.mypackage.renderer.Render1 { }
-keepnames class com.mypackage.renderer.Render2 { }
I tested only using Render2 as as well to rule out some implementation issue and that worked just fine.
mk...@google.com <mk...@google.com> #4
public interface Service {
void print();
}
public class ServiceImpl implements Service {
@Override
public void print() {
System.out.println("Hello World!");
}
}
public class ServiceImpl2 implements Service {
@Override
public void print() {
System.out.println("Hello World 2!");
}
}
public class ForRunner {
public static void main(String[] args) {
for (Service service : ServiceLoader.load(Service.class, null)) {
service.print();
}
}
}
With META-INF/services/Service:
ServiceImpl
ServiceImpl2
And the keep-rules:
-keepnames class Service { }
-keepnames class ServiceImpl { }
-keepnames class ServiceImpl2 { }
Compiling with R8 will remove the the ServiceLoader.load call in my case, so it is pretty strange it does not do it in your case.
Can I ask you to try and run with:
-whyareyoukeeping class com.mypackage.Renderer
to see if there is anything else that is pinning the class?
ja...@nimbus.co <ja...@nimbus.co> #5
interface Renderer {
void render();
static void render(String type) {
Loader.instance().get(response.type).render();
}
}
abstract class Loader {
static volatile Loader sInstance;
abstract Renderer get(String type);
static final class Default extends Loader {
final SimpleArrayMap<String, Renderer> renderMap;
Default() {
renderMap = new SimpleArrayMap<>();
for (Renderer.Factory factory :
ServiceLoader.load(Renderer.Factory.class, Renderer.Factory.class.getClassLoader())) {
renderMap.put(factory.type(), factory.get());
}
}
@Override
Renderer get(String type) {
return renderMap.get(type);
}
}
static Loader instance() {
if (sInstance == null) {
try {
sInstance = new Default();
} catch (ServiceConfigurationError e) {
sInstance = new Loader() {
@Override
Renderer get(String type) {
return null;
}
};
}
}
return sInstance;
}
}
One more thing to note, this configuration is split between 3 different Java library modules consumed by a Kotlin application. Renderer is defined in library A, Render1 in library B and Render2 in library C with both having a dependency on A, all consumed by the application module.
mk...@google.com <mk...@google.com> #6
ja...@nimbus.co <ja...@nimbus.co> #7
The way I have worked around this for now is by defining an implementation of the Loader interface in my App module and installing both Renderers when the Loader is constructed but I was under the impression I can load multiple implementations of a service by iterating over the call to ServiceLoader.load
public interface Renderer {
/**
* @return the type of ad this object can render
*/
String type();
@MainThread
void render(@NonNull NimbusAd response, @NonNull ViewGroup container, @NonNull AdRenderedListener listener);
/**
* Install a renderer in the Nimbus SDK. This method will replace an already installed renderer of the same type.
*
* @param renderer renderer to install
*/
static void install(Renderer renderer) {
Loader.sDelegate.install(renderer);
}
/**
* Get the currently installed renderer for the given type
*
* @param type type of renderer. Currently supported types are "video" and "static"
* @return the currently installed renderer or null
*/
static Renderer get(String type) {
return Loader.sDelegate.get(type);
}
interface Manager {
default void loadAd(NimbusAd response, ViewGroup container, AdRenderedListener listener) {
final Renderer renderer = Renderer.get(response.type);
if (renderer == null) {
listener.onError(new NimbusError(PROVIDER_ERROR, response.type, null));
return;
}
renderer.render(response, container, listener);
}
}
abstract class Loader {
private static Loader sDelegate = create();
abstract Renderer get(String type);
abstract void install(Renderer renderer);
static Loader create() {
try {
return ServiceLoader.load(Loader.class, Loader.class.getClassLoader()).iterator().next();
} catch (ServiceConfigurationError e) {
return new Default();
}
}
static class Default extends Loader {
protected final SimpleArrayMap<String, Renderer> renderMap = new SimpleArrayMap<>();
public Default() {
try {
for (Renderer renderer : ServiceLoader.load(Renderer.class, Renderer.class.getClassLoader())) {
install(renderer);
}
} catch (ServiceConfigurationError e) {
Log.e(Loader.class.getSimpleName(), e.getMessage());
}
}
protected void install(Renderer renderer) {
renderMap.put(renderer.type(), renderer);
}
@Override
Renderer get(String type) {
return renderMap.get(type);
}
}
}
}
Here is the chain of what is pinning the Renderer
com.adsbynimbus.render.Renderer
|- is referenced in keep rule:
| void com.adsbynimbus.render.Renderer$Manager$-CC.$default$loadAd(com.adsbynimbus.render.Renderer$Manager,com.adsbynimbus.NimbusAd,android.view.ViewGroup,com.adsbynimbus
.render.AdRenderedListener)
|- is invoked from:
| void com.adsbynimbus.ui.NimbusAdViewFragment.loadAd(com.adsbynimbus.NimbusAd,android.view.ViewGroup,com.adsbynimbus.render.AdRenderedListener)
|- is overriding method:
| void com.adsbynimbus.render.Renderer$Manager.loadAd(com.adsbynimbus.NimbusAd,android.view.ViewGroup,com.adsbynimbus.render.AdRenderedListener)
|- is invoked from:
| void com.adsbynimbus.ui.NimbusAdViewFragment.onStart()
|- is overriding method:
| void androidx.fragment.app.Fragment.onStart()
|- is invoked from:
| void androidx.fragment.app.FragmentManagerImpl.moveToState(androidx.fragment.app.Fragment,int,int,int,boolean)
|- is invoked from:
| android.view.View androidx.fragment.app.FragmentManagerImpl.onCreateView(android.view.View,java.lang.String,android.content.Context,android.util.AttributeSet)
|- is defined in library method overridden by:
| androidx.fragment.app.FragmentManagerImpl
|- is instantiated in:
| void androidx.fragment.app.Fragment.<init>()
|- is referenced in keep rule:
| C:\Users\jason\Work\nimbus-android\app\build\intermediates\aapt_proguard_file\release\aapt_rules.txt:99:1
mk...@google.com <mk...@google.com> #8
mk...@google.com <mk...@google.com> #9
ja...@nimbus.co <ja...@nimbus.co> #10
In reference to
Similar behavior can be observed by creating a new Kotlin project, adding the coroutines library, and then adding another implementation of the MainDispatcherFactory along with the corresponding services file "kotlinx.coroutines.internal.MainDispatcherFactory". The resulting APK has the AndroidDispatcherFactory and the other implementation completely removed.
mk...@google.com <mk...@google.com>
mk...@google.com <mk...@google.com> #11
Hi Jason, thank you for providing an example.
First a disclaimer, I have tested on 'com.android.tools.build:gradle:4.0.0-alpha09' and I believe the sample is 'com.android.tools.build:gradle:4.0.0-alpha07', but I think there is no difference in R8 implementation wise regarding ServiceLoader.
I am not quite certain on what you specify as incorrect behavior and what you would consider correct behavior. For app-working, R8 synthesizes the following code:
.class public final synthetic L$$ServiceLoaderMethods; .super Ljava/lang/Object; .source "ServiceLoader"
# direct methods
.method public static $load0()Ljava/util/Iterator;
.locals 3
const/4 v0, 0x1
:try_start_0
new-array v0, v0, [Lcom/example/servicebase/Service;
const/4 v1, 0x0
new-instance v2, Lcom/example/servicebase/Service1;
invoke-direct {v2}, Lcom/example/servicebase/Service1;-><init>()V
aput-object v2, v0, v1
invoke-static {v0}, Ljava/util/Arrays;->asList([Ljava/lang/Object;)Ljava/util/List;
move-result-object v0
invoke-interface {v0}, Ljava/util/List;->iterator()Ljava/util/Iterator;
move-result-object v0
:try_end_0
.catchall {:try_start_0 .. :try_end_0} :catchall_0
...
.end method
This will load Service1 and return it.
For broken, we generate the following code:
# direct methods
.method public static $load0()Ljava/util/Iterator;
.locals 3
const/4 v0, 0x2
:try_start_0
new-array v0, v0, [Lcom/example/servicebase/Service;
const/4 v1, 0x0
new-instance v2, Lcom/example/service2/Service2;
invoke-direct {v2}, Lcom/example/service2/Service2;-><init>()V
aput-object v2, v0, v1
const/4 v1, 0x1
new-instance v2, Lcom/example/servicebase/Service1;
invoke-direct {v2}, Lcom/example/servicebase/Service1;-><init>()V
aput-object v2, v0, v1
invoke-static {v0}, Ljava/util/Arrays;->asList([Ljava/lang/Object;)Ljava/util/List;
move-result-object v0
invoke-interface {v0}, Ljava/util/List;->iterator()Ljava/util/Iterator;
move-result-object v0
:try_end_0
.catchall {:try_start_0 .. :try_end_0} :catchall_0
...
.end method
For broken the array contains Service2 and Service1. So in both cases we rewrite the call to ServiceLoader.load with the correct call and the correct number of services. Do you think we should not rewrite it - or does it not rewrite in your test-sample?
ja...@nimbus.co <ja...@nimbus.co> #12
Using the sample provided, I get the following crash when I try to run the build brokenDebug:
Process: com.example.myapplication, PID: 16228
java.util.ServiceConfigurationError: com.example.servicebase.Service: Provider com.example.service2.Service2 not found
at java.util.ServiceLoader.fail(ServiceLoader.java:233)
at java.util.ServiceLoader.access$100(ServiceLoader.java:183)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:373)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:416)
at java.util.ServiceLoader$1.next(ServiceLoader.java:494)
at com.example.myapplication.MainActivity.onCreate(:17)
at android.app.Activity.performCreate(Activity.java:7802)
at android.app.Activity.performCreate(Activity.java:7791)
...
Caused by: java.lang.ClassNotFoundException: com.example.service2.Service2
at java.lang.Class.classForName(Native Method)
at java.lang.Class.forName(Class.java:454)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:371)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:416)
at java.util.ServiceLoader$1.next(ServiceLoader.java:494)
at com.example.myapplication.MainActivity.onCreate(:17)
at android.app.Activity.performCreate(Activity.java:7802)
at android.app.Activity.performCreate(Activity.java:7791)
...
Caused by: java.lang.ClassNotFoundException: Didn't find class "com.example.service2.Service2" on path: DexPathList[[zip file "/data/app/com.example.myapplication-Gm-chqMgALmB_kIUoVh8LA==/base.apk"],nativeLibraryDirectories=[/data/app/com.example.myapplication-Gm-chqMgALmB_kIUoVh8LA==/lib/x86, /system/lib, /system/product/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:196)
at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
at java.lang.Class.classForName(Native Method)
at java.lang.Class.forName(Class.java:454)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:371)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:416)
at java.util.ServiceLoader$1.next(ServiceLoader.java:494)
at com.example.myapplication.MainActivity.onCreate(:17)
at android.app.Activity.performCreate(Activity.java:7802)
at android.app.Activity.performCreate(Activity.java:7791)
...
Not only is the call to ServiceLoader.load not being rewritten, the services themselves are being removed as well. I expect the output you got when you ran my sample where the call is rewritten on both variants.
I've attached a zip file of the entire build directory after running it on my machine. This was tested with the following config.
- OS: Windows 10
- Gradle: 6.2.1
- Android Gradle Plugin: 4.0.0-beta01
- org.gradle.jvmargs=-Xmx6g -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8
- OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)
mk...@google.com <mk...@google.com> #13
OK, since the example was working on our infrastructure and failing on yours, it seems reasonable to think it may be because of the operating system - and I was able to reproduce on a local windows machine. Basically, we assumed that the line separator used in the service files where using the platform line-ending. That is not always the case and your example shows that. I have a bug-fix out for review and will roll the fix into Studio. I will also report with how to use the version with the fix when it has passed review.
Thank you for the repro and enough information for us to solve the issue.
ap...@google.com <ap...@google.com> #14
Branch: master
commit a6669638a2c23286a948982d643bbf7f76327f0a
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date: Thu Feb 27 13:04:16 2020
Check for any line separator when reading service files
We cannot depend on the system format.
Bug: 144437165
Change-Id: I12009a88adb720be1ce8de00b804cc6f15863f29
M src/main/java/com/android/tools/r8/graph/AppServices.java
A src/test/java/com/android/tools/r8/rewrite/ServiceLoaderRewritingLineSeparatorTest.java
M src/test/java/com/android/tools/r8/rewrite/ServiceLoaderRewritingTest.java
ap...@google.com <ap...@google.com> #15
Branch: 1.6
commit abcc8c59ccc55d37337affcd69a0cd1db6354a40
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date: Thu Feb 27 22:52:48 2020
Version 1.6.76
Cherry pick: Check for any line separator when reading service files
CL:
Bug: 144437165
Change-Id: If812f0ac5109f4d516a26e5b67c730573f44d3cf
M src/main/java/com/android/tools/r8/Version.java
M src/main/java/com/android/tools/r8/graph/AppServices.java
M src/test/java/com/android/tools/r8/rewrite/ServiceLoaderRewritingTest.java
mk...@google.com <mk...@google.com> #16
Can I ask you to try out version 1.6.76 by adding the following to your top-level build gradle file:
buildscript {
repositories {
maven {
url 'https://storage.googleapis.com/r8-releases/raw'
}
}
dependencies {
classpath 'com.android.tools:r8:1.6.76' // Must be before the Gradle Plugin for Android.
classpath 'com.android.tools.build:gradle:X.Y.Z' // Your current AGP version.
}
}
That should hopefully resolve the issue you have with ServiceLoader and services.
ap...@google.com <ap...@google.com> #17
Branch: 2.0
commit 32aafeda4ec5f16c48e72b2f4ee8a2ac99502305
Author: Morten Krogh-Jespersen <mkroghj@google.com>
Date: Fri Feb 28 13:20:54 2020
Version 2.0.41
Cherry pick: Check for any line separator when reading service files
CL:
Bug: 144437165
Change-Id: Ifc07dbd62f1c94f59f24c6175b705521e6ec2e0a
M src/main/java/com/android/tools/r8/Version.java
M src/main/java/com/android/tools/r8/graph/AppServices.java
A src/test/java/com/android/tools/r8/rewrite/ServiceLoaderRewritingLineSeparatorTest.java
M src/test/java/com/android/tools/r8/rewrite/ServiceLoaderRewritingTest.java
ja...@nimbus.co <ja...@nimbus.co> #18
I can confirm that with version 1.6.76 of R8 that multiple services are being rewritten as expected on my Windows development environment.
Thank you for getting this fix out!
mk...@google.com <mk...@google.com> #19
Thank you for reporting the issue. Will close the issue since it is now resolved.
Description
final SimpleArrayMap<String, Renderer> renderMap;
Default() {
renderMap = new SimpleArrayMap<>();
for (Renderer renderer : ServiceLoader.load(Renderer.class, Renderer.class.getClassLoader())) {
renderMap.put(renderer.type(), renderer);
}
}
/* Works */
META-INF/services/com.mypackage.Renderer
com.mypackage.renderer.Render1
/* Does not work */
META-INF/services/com.mypackage.Renderer
com.mypackage.renderer.Render1
com.mypackage.renderer.Render2
Both classes have no-arg default constructors and the correct proguard rules are in place.
I am using AGP 4.0.0-alpha02 and Gradle 6.0