Text's inlineContent is not added to TalkBack in 1.7 [376479686]

Verified

Bug

Status Update

No update yet.

Description

si...@gmail.com

created issue #1

Oct 31, 2024 09:34AM

Jetpack Compose version: 1.7.5

Jetpack Compose component(s) used: foundation

Steps to Reproduce or Code Sample to Reproduce:

BasicText(
    text = buildAnnotatedString {
        append("Outer text")
        appendInlineContent("info")
    },
    inlineContent = mapOf("info" to InlineTextContent(Placeholder(16.sp, 16.sp, Center), {
        Icon(Icons.Default.Warning, contentDescription = "Important warning")
    }))
)

TalkBack TreeDebug logs:

Version 1.6.8 (BOM 2024.08.00)

Window: AccessibilityWindowInfo[title=My Application, displayId=0, id=39, type=TYPE_APPLICATION, layer=0, region=SkRegion((0,0,1080,2400)), bounds=Rect(0, 0 - 1080, 2400), focused=true, active=true, pictureInPicture=false, hasParent=false, isAnchored=false, hasChildren=false]
 (-2147455771)39.FrameLayout:(0, 0 - 1080, 2400):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (33643)39.View:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS)(custom action)
 (33767)39.TextView:(112, 149 - 329, 202):TEXT{Outer text�}:GRANULARITY{31}(action:CLEAR_A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):accessibilityFocused:supportsTextLocation
 (33829)39.ImageView:(284, 153 - 329, 198):CONTENT{Important warning}:GRANULARITY{11}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):screenReaderfocusable

Notice last line - "Important warning" is present

Version 1.7.5 (BOM 2024.10.01)

Window: AccessibilityWindowInfo[title=My Application, displayId=0, id=42, type=TYPE_APPLICATION, layer=0, region=SkRegion((0,0,1080,2400)), bounds=Rect(0, 0 - 1080, 2400), focused=true, active=true, pictureInPicture=false, hasParent=false, isAnchored=false, hasChildren=false]
 (-2147455768)42.FrameLayout:(0, 0 - 1080, 2400):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (33646)42.View:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS)(custom action)
 (33770)42.TextView:(112, 149 - 329, 202):TEXT{Outer text�}:GRANULARITY{31}(action:CLEAR_A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):screenReaderfocusable:accessibilityFocused:supportsTextLocation

There "Important warning" is missing

Version 1.8.0-alpha05 (BOM-alpha 2024.10.01)

Window: AccessibilityWindowInfo[title=My Application, displayId=0, id=44, type=TYPE_APPLICATION, layer=0, region=SkRegion((0,0,1080,2400)), bounds=Rect(0, 0 - 1080, 2400), focused=true, active=true, pictureInPicture=false, hasParent=false, isAnchored=false, hasChildren=false]
 (-2147455766)44.FrameLayout:(0, 0 - 1080, 2400):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (32687)44.ComposeView:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (33648)44.View:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS)(custom action)
 (33772)44.TextView:(112, 149 - 329, 202):TEXT{Outer text�}:GRANULARITY{31}(action:CLEAR_A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):accessibilityFocused:supportsTextLocation
 (33834)44.ImageView:(284, 153 - 329, 198):CONTENT{Important warning}:GRANULARITY{11}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):screenReaderfocusable

There "Important warning" is back again

It seems like this regression was added in Icd0becab6818712ab7fa6f3abe83f441a841af5e and fixed in I531e7bad97cda327974696686357a9e56430a6a6 (see TextAnnotatedStringNode.shouldClearDescendantSemantics). Although it is fixed in 1.8-alpha, version 1.7 is broken, preventing our app from updating.

Comments

mo...@google.com <mo...@google.com> Nov 6, 2024 05:31PM

Assigned to an...@google.com.

gr...@google.com <gr...@google.com> #2Nov 7, 2024 06:24PM

Reassigned to gr...@google.com.

Thanks for the additional information in the report!

Can confirm that this repros on foundation 1.7.5 and is fixed in 1.8.0-alpha05 (API 34 pixel 5). Using talkback, the icon is skipped not skipped in 1.8 but is in 1.7.5.

repro code:

import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.text.BasicText
import androidx.compose.foundation.text.InlineTextContent
import androidx.compose.foundation.text.appendInlineContent
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.semantics.contentDescription
import androidx.compose.ui.semantics.role
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.Placeholder
import androidx.compose.ui.text.PlaceholderVerticalAlign
import androidx.compose.ui.text.buildAnnotatedString
import androidx.compose.ui.unit.sp

@Composable
fun Repro() {
    val id = "inline"
    val placeholder = Placeholder(16.sp, 16.sp, PlaceholderVerticalAlign.Center)
    val inlineContent = InlineTextContent(placeholder) { InlineContent() }
    val inlineContentMap = mapOf(id to inlineContent)
    val text = buildAnnotatedString {
        append("before text - ")
        appendInlineContent(id)
        append(" - after text")
    }
    BasicText(text, inlineContent = inlineContentMap)
}

@Composable
private fun InlineContent() {
    Box(
        modifier = Modifier
            .semantics {
                this.contentDescription = "Red Box"
                this.role = Role.Image
            }
            .background(Color.Red)
            .fillMaxSize()
    )
}

Message last modified on Nov 8, 2024 12:28AM

gr...@google.com <gr...@google.com> #3Nov 8, 2024 09:03PM

1.7 cherry pick cl: https://android-review.git.corp.google.com/c/platform/frameworks/support/+/3341487

Add test to ToT cl: https://android-review.git.corp.google.com/c/platform/frameworks/support/+/3340855

Cherry pick bug: b/378121688

ap...@google.com <ap...@google.com> #4Nov 12, 2024 10:47PM

Project: platform/frameworks/support
Branch: androidx-main
Author: Grant Toepfer <grantapher@google.com>
Link: https://android-review.googlesource.com/3340855

Add test for text inline content semantics

Expand for full commit details

Add test for text inline content semantics 
 
Adds a test for a cherry-pick in 1.7, see aosp/3341487. 
 
Bug: b/376479686 
Test: androidx.compose.foundation.text.BasicTextSemanticsTest.inlineContentSemantics_matchesInMergedSemantics 
Change-Id: I22c5be1bf542a390e7d6d14851a1134787fb2028

Files:

M compose/foundation/foundation/src/androidInstrumentedTest/kotlin/androidx/compose/foundation/text/modifiers/BasicTextSemanticsTest.kt

Hash: 21332df933adc3afa9bdee0fb729d6ed81d83669
Date: Thu Nov 07 17:16:32 2024

gr...@google.com <gr...@google.com> #5Dec 2, 2024 07:24PM

update to cherry pick: https://android-review.git.corp.google.com/c/platform/frameworks/support/+/3376831 then https://android-review.git.corp.google.com/c/platform/frameworks/support/+/3341487 into 1.7. Currently hung up on a cross-module experimental annotation usage.

gr...@google.com <gr...@google.com> #6Dec 4, 2024 12:42AM

Cherry pick complete, https://r.android.com/3376831, https://r.android.com/3387946, and https://r.android.com/3341487 in order are targeting 1.7.6.

Sibling CL https://r.android.com/3387948 is merged into main as cleanup in 1.8.

gr...@google.com <gr...@google.com> #7Dec 4, 2024 12:42AM

Reassigned to bu...@google.com.

Bugjuggler: wait 12d

ap...@google.com <ap...@google.com> #8Dec 4, 2024 12:42AM

Marked as fixed.

Project: platform/frameworks/support
Branch: androidx-main
Author: Grant Toepfer <grantapher@google.com>
Link: https://android-review.googlesource.com/3387948

Stabilize invisibleToUser

Expand for full commit details

Stabilize `invisibleToUser` 
 
A 1.7 cherry-pick (aosp/3341487) to fix b/376479686 calls `invisibleToUser` from the foundation module, but it cannot depend on `invisibleToUser` if it is still experimental, so experimental was removed from `invisibleToUser` in 1.7 (via aosp/3387946). This change removes experimental from `invisibleToUser` in 1.8+ so that it matches the change in 1.7 for binary compatibility. 
 
Leaving relnote blank since `invisibleToUser` is still deprecated in 1.8+ 
 
Relnote: N/A 
 
Bug: b/376479686 
Test: N/A 
Change-Id: I40cdd6a3d335011e890761560a5a3416aa579991

Files:

M compose/ui/ui/api/current.txt
M compose/ui/ui/api/restricted_current.txt
M compose/ui/ui/src/androidMain/kotlin/androidx/compose/ui/platform/SemanticsUtils.android.kt
M compose/ui/ui/src/commonMain/kotlin/androidx/compose/ui/semantics/SemanticsProperties.kt

Hash: 211cbf6550baa068eb56bf80d767c5ef1e882674
Date: Mon Dec 02 16:33:49 2024

gr...@google.com <gr...@google.com> Dec 4, 2024 12:43AM

Status: Assigned (reopened)

bu...@google.com <bu...@google.com> #9Dec 4, 2024 12:43AM

Accepted by bu...@google.com.

Hi. I've received your bug and will wait until 2024-12-15 16:42 -0800 PST and then assign the bug to grantapher@google.com.

gr...@google.com <gr...@google.com> #10Dec 4, 2024 12:43AM

Assigned to bu...@google.com.

Bugjuggler: wait 12d

bu...@google.com <bu...@google.com> #11Dec 4, 2024 12:43AM

Accepted by bu...@google.com.

Hi. I've received your bug and will wait until 2024-12-15 16:43 -0800 PST and then assign the bug to grantapher@google.com.

gr...@google.com <gr...@google.com> #12Dec 11, 2024 10:11PM

Marked as fixed, reassigned to gr...@google.com.

Should be fixed in compose foundation 1.7.6. Reporter, please verify your use case is working.

si...@gmail.com <si...@gmail.com> #13Dec 12, 2024 09:22AM

Can confirm the fix is working in 1.7.6

gr...@google.com <gr...@google.com> #14Dec 12, 2024 05:51PM

Verified by gr...@google.com.

Thank you!