Text's inlineContent is not added to TalkBack in 1.7 [376479686]

Verified

Bug

Status Update

No update yet.

Description

si...@gmail.com

created issue #1

Oct 31, 2024 09:34AM

Jetpack Compose version: 1.7.5

Jetpack Compose component(s) used: foundation

Steps to Reproduce or Code Sample to Reproduce:

BasicText(
    text = buildAnnotatedString {
        append("Outer text")
        appendInlineContent("info")
    },
    inlineContent = mapOf("info" to InlineTextContent(Placeholder(16.sp, 16.sp, Center), {
        Icon(Icons.Default.Warning, contentDescription = "Important warning")
    }))
)

TalkBack TreeDebug logs:

Version 1.6.8 (BOM 2024.08.00)

Window: AccessibilityWindowInfo[title=My Application, displayId=0, id=39, type=TYPE_APPLICATION, layer=0, region=SkRegion((0,0,1080,2400)), bounds=Rect(0, 0 - 1080, 2400), focused=true, active=true, pictureInPicture=false, hasParent=false, isAnchored=false, hasChildren=false]
 (-2147455771)39.FrameLayout:(0, 0 - 1080, 2400):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (33643)39.View:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS)(custom action)
 (33767)39.TextView:(112, 149 - 329, 202):TEXT{Outer text�}:GRANULARITY{31}(action:CLEAR_A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):accessibilityFocused:supportsTextLocation
 (33829)39.ImageView:(284, 153 - 329, 198):CONTENT{Important warning}:GRANULARITY{11}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):screenReaderfocusable

Notice last line - "Important warning" is present

Version 1.7.5 (BOM 2024.10.01)

Window: AccessibilityWindowInfo[title=My Application, displayId=0, id=42, type=TYPE_APPLICATION, layer=0, region=SkRegion((0,0,1080,2400)), bounds=Rect(0, 0 - 1080, 2400), focused=true, active=true, pictureInPicture=false, hasParent=false, isAnchored=false, hasChildren=false]
 (-2147455768)42.FrameLayout:(0, 0 - 1080, 2400):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (33646)42.View:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS)(custom action)
 (33770)42.TextView:(112, 149 - 329, 202):TEXT{Outer text�}:GRANULARITY{31}(action:CLEAR_A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):screenReaderfocusable:accessibilityFocused:supportsTextLocation

There "Important warning" is missing

Version 1.8.0-alpha05 (BOM-alpha 2024.10.01)

Window: AccessibilityWindowInfo[title=My Application, displayId=0, id=44, type=TYPE_APPLICATION, layer=0, region=SkRegion((0,0,1080,2400)), bounds=Rect(0, 0 - 1080, 2400), focused=true, active=true, pictureInPicture=false, hasParent=false, isAnchored=false, hasChildren=false]
 (-2147455766)44.FrameLayout:(0, 0 - 1080, 2400):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (32687)44.ComposeView:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action)
 (33648)44.View:(0, 0 - 441, 309):GRANULARITY{0}(action:A11Y_FOCUS)(custom action)
 (33772)44.TextView:(112, 149 - 329, 202):TEXT{Outer text�}:GRANULARITY{31}(action:CLEAR_A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):accessibilityFocused:supportsTextLocation
 (33834)44.ImageView:(284, 153 - 329, 198):CONTENT{Important warning}:GRANULARITY{11}(action:A11Y_FOCUS/SCROLL_TO_POSITION)(custom action):screenReaderfocusable

There "Important warning" is back again

It seems like this regression was added in Icd0becab6818712ab7fa6f3abe83f441a841af5e and fixed in I531e7bad97cda327974696686357a9e56430a6a6 (see TextAnnotatedStringNode.shouldClearDescendantSemantics). Although it is fixed in 1.8-alpha, version 1.7 is broken, preventing our app from updating.

Comments

mo...@google.com <mo...@google.com> Nov 6, 2024 05:31PM

Assigned to an...@google.com.

gr...@google.com <gr...@google.com> #2Nov 7, 2024 06:24PM

Reassigned to gr...@google.com.

We have some support in androidx.compose.ui.autofill

Leaving this bug open in case Ralston wants to add more info

Message last modified on Nov 8, 2024 12:28AM

gr...@google.com <gr...@google.com> #3Nov 8, 2024 09:03PM

I found an example here. One problem is that it only works with the Google autofill service. When using any other service like Microsoft Authenticator or OneKey, the following error is shown:

D/Autofill Status: Autofill popup isn't shown because autofill is not available.
    
    Did you set up autofill?
    1. Go to Settings > System > Languages&input > Advanced > Autofill Service
    2. Pick a service
    
    Did you add an account?
    1. Go to Settings > System > Languages&input > Advanced
    2. Click on the settings icon next to the Autofill Service
    3. Add your account

Is this a bug on your side or do the app developers of these password managers need to change their implementation?

ap...@google.com <ap...@google.com> #4Nov 12, 2024 10:47PM

Another problem I am facing with autofill is that it won't save my credentials. I tried forcing to finish the context as described here, but it won't do that either although I filled in the username with the one provided by the Google autofill service.

gr...@google.com <gr...@google.com> #5Dec 2, 2024 07:24PM

deleted

gr...@google.com <gr...@google.com> #6Dec 4, 2024 12:42AM

Facing the same issue here, Google autofill service seems to work. Zero documentation on adding support for Autofill framework on jetpack compose.

gr...@google.com <gr...@google.com> #7Dec 4, 2024 12:42AM

Reassigned to bu...@google.com.

Hello, I am trying to implement the same thing - it seems like there's no way for current password managers like 1Password or Bitwarden, to automatically pick up the fields. Is there a possible solution?

ap...@google.com <ap...@google.com> #8Dec 4, 2024 12:42AM

Marked as fixed.

I setup this modifier:

@OptIn(ExperimentalComposeUiApi::class)
fun Modifier.autofill(
    autofillTypes: List<AutofillType>,
    onFill: ((String) -> Unit),
) = composed {
    val autofill = LocalAutofill.current
    val autofillNode = AutofillNode(onFill = onFill, autofillTypes = autofillTypes)
    LocalAutofillTree.current += autofillNode

    this.onGloballyPositioned {
        autofillNode.boundingBox = it.boundsInWindow()
    }.onFocusChanged { focusState ->
        autofill?.run {
            if (focusState.isFocused) {
                requestAutofillForNode(autofillNode)
            } else {
                cancelAutofillForNode(autofillNode)
            }
        }
    }
}

And use it like this:

val emailState = remember { EmailState() }
Email(
    modifier = Modifier.autofill(
        autofillTypes = listOf(
            AutofillType.Username,
            AutofillType.EmailAddress
        ),
        onFill = { emailState.text = it },
    ),
    emailState = emailState,
    onImeAction = { onForgotPasswordSubmitted(emailState.text) }
)

With these steps, autofill works for me.

gr...@google.com <gr...@google.com> Dec 4, 2024 12:43AM

Status: Assigned (reopened)

bu...@google.com <bu...@google.com> #9Dec 4, 2024 12:43AM

Accepted by bu...@google.com.

@lo...@gmail.com comment#8

what version of compose are you using? I'm using latest alpha and the above doesn't work for me. 😭

gr...@google.com <gr...@google.com> #10Dec 4, 2024 12:43AM

Assigned to bu...@google.com.

Works perfectly fine on the latest stable and alpha versions. Did you check in my second comment if you enabled everything?

bu...@google.com <bu...@google.com> #11Dec 4, 2024 12:43AM

Accepted by bu...@google.com.

I believe so?

I copied the modifier and then used

modifier = Modifier.autofill(
        autofillTypes = listOf(
            AutofillType.Username,
            AutofillType.EmailAddress
        ),
        onFill = { emailState.text = it },
    )

on my text field.

gr...@google.com <gr...@google.com> #12Dec 11, 2024 10:11PM

Marked as fixed, reassigned to gr...@google.com.

The auto-filling works for me with the latest alpha version, but I'm never presented with the save dialog though.

si...@gmail.com <si...@gmail.com> #13Dec 12, 2024 09:22AM