IssueTracker

(Sorry about the delay; lots of competing priorities.)

Thanks for the sample code. It appears our M98 fixes were partial in that they fixed the case where the virtual display is piped to an output surface for scanning out (which is what ApiDemos does), but not to an ImageReader or video encoder. The problem is that, on some devices, ARCVM allocates buffers with opaque metadata (DRM format modifier, e.g. tiled or compressed) that cannot be decoded in Android.

IIUC, Gralloc should guarantee linear RGBA_8888 for non-planar HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED. Tao, does that sound right? More context in b/197364098.

Re #7 & #9: I completely missed this due a weird copybara email filter added...until being assigned. I'm not familiar with the media pieces involved here. I can share my understanding based on the aosp bits (publicly available) here. Per IMPLEMENTATION_DEFINED description:

    /**
     * A format indicating that the choice of format is entirely up to the
     * allocator.
     *
     * The allocator should examine the usage bits passed in when allocating a
     * buffer with this format, and it should derive the pixel format from
     * those usage flags. This format must never be used with any of the
     * BufferUsage::CPU_* usage flags.
     *
     * Even when the internally chosen format has an alpha component, the
     * clients must assume the alpha vlaue to be 1.0.
     *
     * The interpretation of the component values is defined by the dataspace.
     */
    IMPLEMENTATION_DEFINED = 0x22,

So at least ImageReader direct cpu readback for the IMPLEMENTATION_DEFINED format is invalid usage. There must involve a bliter in between those 2. e.g. camera -> gpu (egl/vulkan) or hw blit (to an explicit format) -> image reader.

Scaning throught this entire thread, I'm not sure how IMPLEMENTATION_DEFINED gets involved here...maybe I missed something since I'm not familiar with media projection api.

My understanding of the screen capture flow is: layers of the on-screen apps => SurfaceFlinger gl render engine => virtual display fb surface. For protected content (and camera preview?), recording to black is intended behavior. For others, if something black is shown, it's less likely IMPLEMENTATION_DEFINED format from the app layer.

Side suggestions: it's better to file an internal issue for investigation to avoid more folks sharing infos on this public issue tracker, and use the newly filed issue as the blocker for related issues.