AnnotatedString rendered with URLs do not appear in TalkBack 'Links' context menu [231495122]

Fixed

Bug

Status Update

No update yet.

Description

br...@monzo.com

created issue #1

May 5, 2022 10:32AM

I'm currently exploring rendering markdown-style links using ClickableText/AnnotatedStrings, similar to how JetChat works here: https://github.com/android/compose-samples/blob/ee198110d8a7575da281de9bd0f84e91970468ca/Jetchat/app/src/main/java/com/example/compose/jetchat/conversation/Conversation.kt#L481

I'm running into issues with matching the accessibility experience of the old View system. In the old system, URLSpans automatically provided a great accessibility UX:

When reading out any link, there would be a small ding sound to help indicate where the link is.
TalkBack tells the user that this text has links in it.
Upon opening the TalkBack context menu, there would be a "Links" submenu where the user can scroll through all of the links contained within the text.

This doesn't seem to be possible to replicate in Compose. All I can find in the semantics API is "custom actions", which is a similar but worse experience (no ding sound, no specific "links" submenu, no hint to the user that there's links to be clicked, etc).

I think this inconsistency will be confusing for accessibility users. Can we get a new API to help replicate the functionality of the View system? For example, could we add an annotation to indicate to TalkBack where the links are?

Thanks!

Comments

so...@google.com <so...@google.com> May 24, 2022 09:06AM

Reassigned to so...@google.com.

zs...@salesforce.com <zs...@salesforce.com> #2May 26, 2022 10:58PM

Project: platform/frameworks/support
Branch: androidx-main

commit 5b70a786cc5b328f69b2fb9c2dc017998a4e9782
Author: Halil Ozercan <halilibo@google.com>
Date: Fri Apr 21 04:37:07 2023

Stabilize Experimental Text APIs

We have been adding experimental attributes to TextStyle, SpanStyle, and ParagraphStyle to flesh out the Text styling options we provide in Compose. It's now time to stabilize these relatively long standing APIs.

This CL includes stabilization of

- Brush
- DrawStyle
- TextMotion
- DrawScope.drawText
- Paragraph.paint (brush, drawStyle, blendMode)
- MultiParagraph.paint (brush, drawStyle, blendMode)

Alongside these changes, previous constructors and copy functions are being deprecated at HIDDEN level to point developers towards newly stabilized APIs that encompass the previous alternatives.

Relnote: "Stabilized many Text APIs that include `Brush`, `DrawStyle`, `TextMotion`, `DrawScope.drawText`, `Paragraph.paint(Brush)`, `MultiParagraph.paint(Brush)`."
Test: N/A
Fix:

b/261581564

Fix:

b/261581931

Fix:

b/261561245

Change-Id: I2f7409cb64ade53b1ebc3385293456b126314b14

M compose/ui/ui-text/api/current.ignore
M compose/ui/ui-text/api/current.txt
M compose/ui/ui-text/api/public_plus_experimental_current.txt
M compose/ui/ui-text/api/restricted_current.ignore
M compose/ui/ui-text/api/restricted_current.txt
M compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/AndroidParagraph.android.kt
M compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/Paragraph.android.kt
M compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/style/TextMotion.android.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/MultiParagraph.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/Paragraph.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/ParagraphStyle.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/SpanStyle.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/TextPainter.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/TextStyle.kt
M compose/ui/ui-text/src/skikoMain/kotlin/androidx/compose/ui/text/Paragraph.skiko.kt
M compose/ui/ui-text/src/skikoMain/kotlin/androidx/compose/ui/text/style/TextMotion.skiko.kt

https://android-review.googlesource.com/2550856

kl...@google.com <kl...@google.com> #3May 26, 2022 11:17PM

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.compose.ui:ui-text:1.5.0-alpha04

so...@google.com <so...@google.com> #4May 27, 2022 09:13AM

Reassigned to kl...@google.com.

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.compose.ui:ui-text-android:1.5.0-beta01

so...@google.com <so...@google.com> #5May 31, 2022 05:46PM

Reassigned to so...@google.com.

Re-assigning back for now

cl...@google.com <cl...@google.com> #6Jun 7, 2022 06:19PM

is this a P4?

so...@google.com <so...@google.com> #7Jun 7, 2022 06:36PM

Reassigned to kl...@google.com.

To comment#6, generally speaking, we haven't been using priorities due to its specific meaning/ SLO status.

Re-assigning back to Zach. As discussed, we'll proceed with aosp/2108696

kl...@google.com <kl...@google.com> Jun 8, 2022 11:02PM

Accepted by kl...@google.com.

ap...@google.com <ap...@google.com> #8Jun 9, 2022 04:55PM

Marked as fixed.

Project: platform/frameworks/support
Branch: androidx-main

commit 0c11fb9996e4baf9bf46dc57de9df714b7f61214
Author: Zach Klippenstein <klippenstein@google.com>
Date: Thu May 26 17:01:28 2022

Introduce UrlAnnotation to support link a11y.

This change is basically just a copy of `VerbatimTtsAnnotation`.
Includes the new annotation type, methods on `AnnotatedString` and
`Builder`, a new `Saver`, support for converting to `URLSpan`s when creating
`SpannableString`s, a demo, and tests including a missing test for
`VerbatimTtsAnnotation`. All new APIs are marked experimental.

Fixes:

b/231495122

Test: ./gradlew :compose:ui:ui-text:test
Test: Tested TalkBack integration manually
Relnote: "Introduced the `UrlAnnotation` annotation type and associated
methods to support TalkBack link support in `AnnotatedString`s."
Change-Id: I1c754dfa0ce88a36989888b43816333dfc94b0aa

A compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/UrlAnnotation.kt
M compose/ui/ui-text/api/restricted_current.txt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/AnnotatedString.kt
M compose/ui/ui-text/src/androidAndroidTest/kotlin/androidx/compose/ui/text/platform/AndroidAccessibilitySpannableStringTest.kt
M compose/ui/ui-text/src/test/java/androidx/compose/ui/text/AnnotatedStringBuilderTest.kt
M compose/ui/ui-text/src/test/java/androidx/compose/ui/text/SaversTest.kt
M compose/ui/ui-text/src/test/java/androidx/compose/ui/text/input/TextFieldValueTest.kt
M compose/foundation/foundation/src/androidAndroidTest/kotlin/androidx/compose/foundation/textfield/TextFieldTest.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/Savers.kt
M compose/foundation/foundation/integration-tests/foundation-demos/src/main/java/androidx/compose/foundation/demos/text/ComposeTextAccessibility.kt
A compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/platform/extensions/UrlAnnotationExtensions.android.kt
M compose/ui/ui-text/api/public_plus_experimental_current.txt
M compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/platform/AndroidAccessibilitySpannableString.android.kt
M compose/ui/ui-text/api/current.txt

https://android-review.googlesource.com/2108696

[Deleted User] <[Deleted User]> #9Oct 7, 2022 12:24AM

This has a bug in it - it does not call through to the onClick for a ClickableText. See repro example app here:

https://github.com/chanakin/ClickableTextWithUrlExample

Should I open a new issue to track this?

br...@monzo.com <br...@monzo.com> #10Oct 7, 2022 10:46AM

I'm also just trying this for the first time and:

a) can confirm the above issue. When double clicking on a link in the accessibility menu, nothing happens.

b) even if the click handler ran, it would just launch an ACTION_VIEW intent, but this default behaviour isn't good enough for our use case. We want to be able to intercept the link in our Compose code and handle it appropriately.

zl...@google.com <zl...@google.com> #11Oct 7, 2022 10:56PM

Also experiencing the issue in

https://github.com/chanakin/ClickableTextWithUrlExample, in which double-clicking a UrlAnnotation link in the TalkBack menu does not fire the onClick handler set in a ClickableText

[Deleted User] <[Deleted User]> #12Oct 12, 2022 11:20PM

I created a new issue, hopefully that gets some attention. Those of you who are following along and reporting the same issue, please star it!

https://issuetracker.google.com/issues/253292081

Message last modified on Oct 12, 2022 11:27PM

sr...@gmail.com <sr...@gmail.com> #13Jan 22, 2024 01:15AM

Any Update on this issue?