Status Update
Comments
co...@google.com <co...@google.com>
se...@gmail.com <se...@gmail.com> #2
I can’t record screen using Media Projection API for my android app on Chrome OS (ver. 98.0.4758.51 (Platform version: 14388.27.0)) which is a fixed version.
The recording result is always black image.
Confirmation environment
Chromebook: Lenovo IdeaPad Flex550i Chromebook 82B80018JP
Chrome OS ver. 98.0.4758.51 (Platform version: 14388.27.0) (beta channel)
Android OS ver. 11
What is the expected behavior?
The entire screen of Chromebook can be recorded using the Media Projection API.
What went wrong?
The recording result is always black image.
In what version will it be fixed?
do...@chromium.org <do...@chromium.org> #3
Can you reproduce the black capture with the Media > MediaProjection activity of the ApiDemos APK?
se...@gmail.com <se...@gmail.com> #4
I made a sample using MediaProjection API and confirmed that permission dialog where you can preview / select the screen to share.
I will provide the sample.
This is the project link:
Please check the video to see how it was confirmed.
This is the operation.
1. Launch the ScreenCapture app
2. Allow storage access.
3. Press the START button to display the permissions dialog.
4. Press the SHARE button to start the screen capture and display the captured image.
5. It will continue to be saved as a jpeg file until you press the stop button.
6. Press the stop button to check the saved file.
Storage location
case Android OS ver. 9: Downloads folder
case Android OS ver. 11: Picture folder
◆Expected result
・ChromeOS ver. 97.0.4692.102, Android OS ver. 9
Video:「osver_9704692102_android9.webm」
The above video is the operation on Android 9 chromebook.
Please make it possible to capture the entire screen even on an Android 11 chromebook, as in the video above.
◆The following are both bad cases.
・ ChromeOS ver.97.0.4692.102, Android OS ver. 11
Video:「osver_9704692102_android11.webm」
I can only capture the screen of the Android app.
・ ChromeOS ver.98.0.475867 beta channel, Android OS ver. 11
Video:「osver_980475867_android11.webm」
I can't capture anything.
> Can you reproduce the black capture with the Media > MediaProjection activity of the ApiDemos APK?
I installed the below ApiDemos APK and checked the operation, but there was no corresponding function.
I reviewed the below source code of ApiDemos ’s MediaProjection API.
I think the usage of MediaProjection API is wrong.
Android 10 and above requires the MediaProjection API to be used in the service.
The ApiDemos code wasn't using the service.
Therefore, it cannot be confirmed on the Chromebook(Android11).
Please tell me the correct sample.
Or, check out the sample source code we provided and tell us how to capture the entire screen on your Android 11 Chromebook.
The sample is in mediaprojectionsample.zip.
se...@gmail.com <se...@gmail.com> #5
The Android 11 chrome book is on the market, so this issue is bothering users of our product.
Specifically, there is the problem that the Chromebook screen cannot be mirrored to some display in the education market.
Please resolve this issue as soon as possible.
co...@gmail.com <co...@gmail.com> #6
Model: Acer Spin CP713-2W
Chrome OS version: 103.0.5060.114
do...@google.com <do...@google.com> #7
(Sorry about the delay; lots of competing priorities.)
Thanks for the sample code. It appears our M98 fixes were partial in that they fixed the case where the virtual display is piped to an output surface for scanning out (which is what ApiDemos
does), but not to an ImageReader
or video encoder. The problem is that, on some devices, ARCVM allocates buffers with opaque metadata (DRM format modifier, e.g. tiled or compressed) that cannot be decoded in Android.
IIUC, Gralloc should guarantee linear RGBA_8888
for non-planar HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED
. Tao, does that sound right? More context in
ad...@emplifi.io <ad...@emplifi.io> #8
do...@google.com <do...@google.com>
le...@google.com <le...@google.com> #9
do...@google.com <do...@google.com>
li...@google.com <li...@google.com> #10
zz...@google.com <zz...@google.com> #11
Re #7 & #9: I completely missed this due a weird copybara
email filter added...until being assigned. I'm not familiar with the media pieces involved here. I can share my understanding based on the aosp bits (publicly available) here. Per IMPLEMENTATION_DEFINED
/**
* A format indicating that the choice of format is entirely up to the
* allocator.
*
* The allocator should examine the usage bits passed in when allocating a
* buffer with this format, and it should derive the pixel format from
* those usage flags. This format must never be used with any of the
* BufferUsage::CPU_* usage flags.
*
* Even when the internally chosen format has an alpha component, the
* clients must assume the alpha vlaue to be 1.0.
*
* The interpretation of the component values is defined by the dataspace.
*/
IMPLEMENTATION_DEFINED = 0x22,
So at least ImageReader direct cpu readback for the IMPLEMENTATION_DEFINED
format is invalid usage. There must involve a bliter in between those 2. e.g. camera -> gpu (egl/vulkan) or hw blit (to an explicit format) -> image reader.
Scaning throught this entire thread, I'm not sure how IMPLEMENTATION_DEFINED
gets involved here...maybe I missed something since I'm not familiar with media projection api.
My understanding of the screen capture flow is: layers of the on-screen apps => SurfaceFlinger gl render engine => virtual display fb surface
. For protected content (and camera preview?), recording to black is intended behavior. For others, if something black is shown, it's less likely IMPLEMENTATION_DEFINED
format from the app layer.
Side suggestions: it's better to file an internal issue for investigation to avoid more folks sharing infos on this public issue tracker, and use the newly filed issue as the blocker for related issues.
te...@gmail.com <te...@gmail.com> #12
MadiaProjection API produces black screen on some ChromeOS / Chromebooks when called from inside an Android App.
le...@google.com <le...@google.com> #13
do...@google.com <do...@google.com>
te...@gmail.com <te...@gmail.com> #14
The app "Screen Mirroring for Roku" on Google Play is still broken on Chromebooks with Android 11.
More specific: ImageReader.OnImageAvailableListener->onImageAvailable(ImageReader reader) produces a black image which leads to a black screen displayed on the connected Roku device.
It does not matter if the app is in "phone", "tablet" or "resizable" window mode on the chromebook.
Here is a video. You will need a Roku device to reproduce the issue:
The app is available on Google Play as free or pro version. Both versions are affected:
Free:
Pro:
I am not sure if AnyDesk/ScreenShot X use a different approach to capture the android screen.
Please note that the same app is working without any issues on Chromebooks with Android 9.
md...@google.com <md...@google.com> #15
te...@gmail.com <te...@gmail.com> #16
le...@google.com <le...@google.com> #17
BTW, it sounds like 109.0.5414.125 is a chrome version instead of chrome os version, you can get chrome os version by type "chrome://version" in browser and check the version of "platform"
Also, if the warning about "not connected to WIFI" could be removed in the future release, that would be nice. I connected my chromebook to my home network with ethernet. It seems the app insists asking me to connect to WIFI while actually it's connected to the same local network (just via ehternet)
te...@gmail.com <te...@gmail.com> #18
We have taken the version number 109.0.5414.125 from the Chromebook Settings -> "About ChromeOS" -> "Google ChromeOS" -> "Version xxx.x..." , so we thought this is the actual ChromeOS version and we also thought it is always synced with the Chrome Browser version. I would suggest that this gets changed to avoid confusing developers and users.
We will update here when we have the real ChromeOS version from the device.
Thank you also for the hint about the "not connected to WIFI" popup when connected via ethernet. This is something we missed when we added compatibilty for ChromeOS, as our chromebooks do not have ethernet ports. We will consider this for the next update.
le...@google.com <le...@google.com> #19
te...@gmail.com <te...@gmail.com> #20
le...@google.com <le...@google.com> #21
te...@gmail.com <te...@gmail.com> #22
To clarify, the last test that resulted in black screen was on platform "15236.80.0 (Official Build) stable-channel octopus", so we tested the wrong version.
The workaround HardwareBuffer.USAGE_GPU_COLOR_OUTPUT on "15236.80.0 (Official Build) stable-channel octopus" fixes the black screen but another problem remains: The image produced by imageReader only contains the screen of Android. The surrounding ChromeOS screen is still black.
On the new platform "15350.0.0 (official build) dev-channel octopus" everything works perfectly. The screen is completely mirrored, without black screen.
So we will wait until this version is rolled out. Thank you very much for your support.
Re #18+19:
We have prepared a fix for "Screen Mirroring for Roku" to address the issue with "not connected to WIFI" popup on ethernet connected Chromebooks and Android devices. The fix will be rolled out with the next app update.
Description
Comments from crbug:
unknown user: UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36
Platform: Acer Chromebook 712 [C871]
Steps to reproduce the problem:
I have an Acer hatch board which I can upgraded to ChromeOS - Android 11 based.
After I upgraded, I can not record screen using MediaProjection API for my android app anymore. The recording is always black when the app records the other apps.
I try to downgrade to ChromeOS 86 which is Android 9, everything work fine.
I also tested with other android apps on Play Store (such as Mobizen Screen Recorder,...) and it likely all of them stop working after update ChromeOS to Android 11.
What is the expected behavior?
I can record my screen
What went wrong?
The recording is black
WebStore page:
Did this work before? N/A
Chrome version: 93.0.4577.85 Channel: stable
OS Version: ChromeOS version: 93.0.4577.85
-----------------------------------
unknown user: This is an Android app downloaded from Google Play
-----------------------------------
unknown user: Just adding that this issue has a massive impact on schools that rely on screen recording tools that are integrated into their learning management systems.
All schools that implemented a 3rd party screen recording tool and have devices updated to Android 11 are dead in the water. They must role back all systems and await a resolution.
-----------------------------------
unknown user: Please address to help schools and teachers trying to communicate in a remote learning environment!
-----------------------------------
qjw@google.com: +Enterprise>Android who might be interested in this.
ARC folks, can you take a look at this?
@reporter: If you have an Android 11 device at hand, could you quickly confirm the app works fine on an Android phone?
-----------------------------------
unknown user: I confirm the app can record on Android 11 device normally. The black recording is only on ChromeOS - Android 11 based
-----------------------------------
pastarmovj@google.com: Removing the Enterprise>Android component because it is about Chrome on Android devices instead of Android emulation on Chrome OS.
-----------------------------------
unknown user: Sorry, you misunderstood, I didn't talk about Chrome.
We make the app here
- If I use the app on ChromeOS - Android 9 based (support ARM++), then I can record the system screen without any issue
- If I upgrade the OS to ChromeOS - Android 11 based (hatch board, support ARMvm), when I record the system screen, the result video will be black. This is a big bug from the OS that needs to be resolved.
You can find more about ARMvm here
Thanks for reading
-----------------------------------
changmar@google.com: This seems specific to either Android 11 or video on ARCVM. I'm looping in +posciak to see if he can help deep dive into why this is showing up as a black screen when recording.
Also +arthurhsu in case he's aware of any new changes of Android 11 in general around permissions and onscreen recording, and if this doesn't play well on ChromeOS (particularly since
-----------------------------------
arthurhsu@google.com: Maybe an Android 10 change.
It will need the app developer to update their app (maybe already did? Try to download a new one fits Android 10+)
-----------------------------------
unknown user: I am seeing multiple comments suggesting the app able to record on Android 11.
I have created the following video using USB-c>HDMI into a capture card to show end to end process of the app recording the screen on Android 11, and then playing it back without black video.
Video link:
Ultimately, it appears the issue is as Kiem stated. This appears to be specific to ARCVM being in play.
-----------------------------------
changmar@google.com: +jseelig to help with this video issue for ARCVM
-----------------------------------
changmar@google.com: moving this over to the Platform team's public issue tracker
-----------------------------------
unknown user: Any update here? Schools are pinging us daily on this issue.
-----------------------------------
arthurhsu@google.com: re #13: have you tried suggestions in link #9? Are you already a foreground service?
-----------------------------------
unknown user: In
On Android 9 ChromeOS devices, the app is also able to record without issue. If that same device is updated to Android 11 ChromeOS, the recordings are black.
Does the new ARCvm environment require special considerations to sharing the textures from ChromeOS to the Android app?
I am fairly confident that we are a foreground service. Unless there was some change required there with Android 11 on ChromeOS that does not apply to typical Android 11 devices?
-----------------------------------
changmar@google.com: +akahuang +bhthompson to help get some traction from Platform side as it might be related to video
-----------------------------------
bhthompson@google.com: Some questions:
1. Does any screen capture app ever work on a Chrome OS device running Android 11?
2. Does this ever work on Android 11 on Chrome OS (e.g. is this specific to some device)?
These systems pass CTS fully, so if this is an expected API behavior we should have caught it already. If this is generic to all of Android 11 on Chrome OS I am not sure who the right owner would really be, on the platform side we are more focused on device specific issues in the underlying platform (e.g. dependencies on kernel/drivers/etc.), but not the overall Android stack, it is not clear where this starts to fail.
We could probably dig into this more if this was a video encoding failure (e.g. a video chat or camera app), as we do rely on device specific acceleration hardware for this kind of use case, but I am not clear if this is the same situation for screen capture software.
-----------------------------------
unknown user: Questions:
1. In our testing, all screen capture apps in the android app store, fail on ChromeOS devices that are running Android 11.
2. No, it always fails. Issue is specific to ChromeOS+Android11.
Is this simply a case of the MediaProjection API being incompatible with the changes in ARCvm?
Would a testApp that boils this down to the base level of failure be helpful?
Are there any testing methodologies we could explore on our side, that could provide the information to make this actionable on your side?
We have internal documentation on the architecture of our application. If we can schedule a call with a developer on your side, we could likely provide some additional insight that we otherwise would not be able to share here publicly. A test app with code would likely solve the same goal however.
Please advise how we can assist here. Thanks!
-----------------------------------
bhthompson@google.com: Thanks!
I don't know that a test app is necessary yet, I think we need to do some more digging on our side as to who would be most familiar with the MediaProjection API and where that might have integration issues.
-----------------------------------
bhthompson@google.com: Dom, is there another bug we should dupe this against perhaps?
-----------------------------------
arthurhsu@google.com: Add Matt, this could be DisplayArea related and Aka may need some help from your team to rule out that possibility.
-----------------------------------
dstaessens@google.com: I gave it a quick try on my hatch device (Acer Chromebook 712):
- Using the builtin screen recording tool (Shift + Ctrl + Show windows) seems to work fine as expected, as it shouldn't use any of the new ARCVM video stack.
- I tried to use the Screencast-O-Matic app but it doesn't seem supported on my hatch: "This item is not compatible with your device"
- Using the "Screen Recorder" app from the app store works fine. The HW encoder seems to be properly used to encode the video, and I can play back the video.
- As a final try I opened the Netflix app in the background and used the "Screen Recorder" app to record my screen. Interestingly the screen recorder app window is recorded, but the Netflix app in the background is completely replaced by a black square.
The ARCVM HW encoding path seems to be working fine, so maybe the recorder app doesn't have the correct permissions or something else in ARCVM causes Android app windows to not show up in the recorded video. I'm not familiar with window composition in ARCVM though, so maybe someone else would be able to comment on this?
-----------------------------------
unknown user: I tested on my hatch device (Acer Chromebook 712 C871)
- "Screen Recorder" app: I can records the android apps only. When I record ChromeOS apps, it results in a black video.
- You can download Screencast-O-Matic app here for testing (we don't support ChromeOS + Android 11 due to the bug)
-----------------------------------
changmar@google.com: Re:
I'm going to assign directly to Dom to see if he knows more about the MediaProject API
-----------------------------------
changmar@google.com: + Elijah since I see Dom may be OOO for the next week or so
-----------------------------------
changmar@google.com: I chatted with Elijah offline and it looks like we'll just need to wait until Dom comes back from OOO (tomorrow)
-----------------------------------
domlaskowski@google.com: Yes, this is an ARCVM platform bug. The reason for the partial screen capture (i.e. only Android layers, and black elsewhere) is that we are missing compositor delegation to transmit frames captured by Chrome OS back into Android, without which we fall back to local capture by the Android compositor. While we have done this integration work (tracked internally at
-----------------------------------
unknown user: Any update here?
-----------------------------------
unknown user: Any ETA here? One of our customers with just over 700 chromebooks is asking us if they should pivot away from chromebooks to the new windows devices targeting schools.
We would like to be able to tell them to stick with Chromebook and that it will be resolved soon.
-----------------------------------
domlaskowski@google.com: Sorry, the investigation of the aforementioned bug was blocked on my limited bandwidth.
I'm prioritizing this now, so stay tuned for an update in the next few days.
-----------------------------------
domlaskowski@google.com: I've followed up on the blocking bug (
I'll update this bug as we make progress.
-----------------------------------
unknown user: Thank you @Dom !
-----------------------------------
domlaskowski@google.com: We have a fix (thanks to Lloyd for the assist!) making its way through the CI pipeline.
I will merge the patches to the M98 release branch once I get approval, probably next week as tomorrow is a company-wide day off.
-----------------------------------
unknown user: Any idea on when we would see this in the wild? Thanks for the hard work here!
-----------------------------------
domlaskowski@google.com: Sorry about the delay. There was a bit of extra work to ensure the feature plays well with Android CTS.
I've merged the fixes to M98 (14388.25.0) and M99 (14416.0.0), so they will show up in the next beta:
-----------------------------------