AnnotatedString rendered with URLs do not appear in TalkBack 'Links' context menu [231495122]

Fixed

Bug

Status Update

No update yet.

Description

br...@monzo.com

created issue #1

May 5, 2022 10:32AM

I'm currently exploring rendering markdown-style links using ClickableText/AnnotatedStrings, similar to how JetChat works here: https://github.com/android/compose-samples/blob/ee198110d8a7575da281de9bd0f84e91970468ca/Jetchat/app/src/main/java/com/example/compose/jetchat/conversation/Conversation.kt#L481

I'm running into issues with matching the accessibility experience of the old View system. In the old system, URLSpans automatically provided a great accessibility UX:

When reading out any link, there would be a small ding sound to help indicate where the link is.
TalkBack tells the user that this text has links in it.
Upon opening the TalkBack context menu, there would be a "Links" submenu where the user can scroll through all of the links contained within the text.

This doesn't seem to be possible to replicate in Compose. All I can find in the semantics API is "custom actions", which is a similar but worse experience (no ding sound, no specific "links" submenu, no hint to the user that there's links to be clicked, etc).

I think this inconsistency will be confusing for accessibility users. Can we get a new API to help replicate the functionality of the View system? For example, could we add an annotation to indicate to TalkBack where the links are?

Thanks!

Comments

so...@google.com <so...@google.com> May 24, 2022 09:06AM

Reassigned to so...@google.com.

zs...@salesforce.com <zs...@salesforce.com> #2May 26, 2022 10:58PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit b90079595f33f58fece04026a97faa0d243acdb1
Author: Yuichi Araki <yaraki@google.com>
Date: Wed Sep 18 16:55:49 2019

Change the way to detect mismatch between POJO and query

This fixes cursor mismatch warnings with expandProjection.

Bug: 140759491
Test: QueryMethodProcessorTest
Change-Id: I7659002e5e0d1ef60fc1af2a625c0c36da0664d8

M room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt
M room/compiler/src/test/kotlin/androidx/room/processor/QueryMethodProcessorTest.kt
M room/compiler/src/test/kotlin/androidx/room/testing/TestProcessor.kt

https://android-review.googlesource.com/1123258

https://goto.google.com/android-sha1/b90079595f33f58fece04026a97faa0d243acdb1

kl...@google.com <kl...@google.com> #3May 26, 2022 11:17PM

The following changes were cherrypicked through Coastguard (

https://android-build.googleplex.com/coastguard/)
Release Track: (

https://android-build.googleplex.com/coastguard/dashboard/6093116828680192/#/cherrypickHistory/all)
Created: 2019-10-14 17:23:42.610596+00:00
Changes: aosp/1123258

so...@google.com <so...@google.com> #4May 27, 2022 09:13AM

Reassigned to kl...@google.com.

Project: platform/frameworks/support
Branch: androidx-master-dev

commit bdde5a1a970ddc9007b28de4aa29d60ffa588f08
Author: Yigit Boyar <yboyar@google.com>
Date: Thu Apr 16 16:47:05 2020

Re-factor how errors are dismissed when query is re-written

This CL changes how we handle errors/warnings if query is
re-written.
There was a bug in expandProjection where we would report warnings
for things that Room already fixes automatically (

b/140759491

).
The solution to that problem (I7659002e5e0d1ef60fc1af2a625c0c36da0664d8)
solved it by deferring validating of columns until after re-write
decision is made. Unfortunately, this required changing PojoRowAdapter
to have a dummy mapping until it is validating, make it hard to use
as it does have a non-null mapping which is not useful.

This CL partially reverts that change and instead rely on the log
deferring logic we have in Context. This way, we don't need to break
the stability of PojoRowAdapter while still having the ability to
drop warnings that room fixes. This will also play nicer when we
have different query re-writing options that can use more information
about the query results.

Bug: 153387066
Bug: 140759491
Test: existing tests pass
Change-Id: I2ec967c763d33d7a3ff02c1a13c6953b460d1e5f

M room/compiler/src/main/kotlin/androidx/room/log/RLog.kt
M room/compiler/src/main/kotlin/androidx/room/processor/QueryMethodProcessor.kt
M room/compiler/src/main/kotlin/androidx/room/solver/TypeAdapterStore.kt
M room/compiler/src/main/kotlin/androidx/room/solver/query/result/PojoRowAdapter.kt

https://android-review.googlesource.com/1288456

so...@google.com <so...@google.com> #5May 31, 2022 05:46PM

Reassigned to so...@google.com.

Re-assigning back for now

cl...@google.com <cl...@google.com> #6Jun 7, 2022 06:19PM

is this a P4?

so...@google.com <so...@google.com> #7Jun 7, 2022 06:36PM

Reassigned to kl...@google.com.

To comment#6, generally speaking, we haven't been using priorities due to its specific meaning/ SLO status.

Re-assigning back to Zach. As discussed, we'll proceed with aosp/2108696

kl...@google.com <kl...@google.com> Jun 8, 2022 11:02PM

Accepted by kl...@google.com.

ap...@google.com <ap...@google.com> #8Jun 9, 2022 04:55PM

Marked as fixed.

Project: platform/frameworks/support
Branch: androidx-main

commit 0c11fb9996e4baf9bf46dc57de9df714b7f61214
Author: Zach Klippenstein <klippenstein@google.com>
Date: Thu May 26 17:01:28 2022

Introduce UrlAnnotation to support link a11y.

This change is basically just a copy of `VerbatimTtsAnnotation`.
Includes the new annotation type, methods on `AnnotatedString` and
`Builder`, a new `Saver`, support for converting to `URLSpan`s when creating
`SpannableString`s, a demo, and tests including a missing test for
`VerbatimTtsAnnotation`. All new APIs are marked experimental.

Fixes:

b/231495122

Test: ./gradlew :compose:ui:ui-text:test
Test: Tested TalkBack integration manually
Relnote: "Introduced the `UrlAnnotation` annotation type and associated
methods to support TalkBack link support in `AnnotatedString`s."
Change-Id: I1c754dfa0ce88a36989888b43816333dfc94b0aa

A compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/UrlAnnotation.kt
M compose/ui/ui-text/api/restricted_current.txt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/AnnotatedString.kt
M compose/ui/ui-text/src/androidAndroidTest/kotlin/androidx/compose/ui/text/platform/AndroidAccessibilitySpannableStringTest.kt
M compose/ui/ui-text/src/test/java/androidx/compose/ui/text/AnnotatedStringBuilderTest.kt
M compose/ui/ui-text/src/test/java/androidx/compose/ui/text/SaversTest.kt
M compose/ui/ui-text/src/test/java/androidx/compose/ui/text/input/TextFieldValueTest.kt
M compose/foundation/foundation/src/androidAndroidTest/kotlin/androidx/compose/foundation/textfield/TextFieldTest.kt
M compose/ui/ui-text/src/commonMain/kotlin/androidx/compose/ui/text/Savers.kt
M compose/foundation/foundation/integration-tests/foundation-demos/src/main/java/androidx/compose/foundation/demos/text/ComposeTextAccessibility.kt
A compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/platform/extensions/UrlAnnotationExtensions.android.kt
M compose/ui/ui-text/api/public_plus_experimental_current.txt
M compose/ui/ui-text/src/androidMain/kotlin/androidx/compose/ui/text/platform/AndroidAccessibilitySpannableString.android.kt
M compose/ui/ui-text/api/current.txt

https://android-review.googlesource.com/2108696

[Deleted User] <[Deleted User]> #9Oct 7, 2022 12:24AM

This has a bug in it - it does not call through to the onClick for a ClickableText. See repro example app here:

https://github.com/chanakin/ClickableTextWithUrlExample

Should I open a new issue to track this?

br...@monzo.com <br...@monzo.com> #10Oct 7, 2022 10:46AM

I'm also just trying this for the first time and:

a) can confirm the above issue. When double clicking on a link in the accessibility menu, nothing happens.

b) even if the click handler ran, it would just launch an ACTION_VIEW intent, but this default behaviour isn't good enough for our use case. We want to be able to intercept the link in our Compose code and handle it appropriately.

zl...@google.com <zl...@google.com> #11Oct 7, 2022 10:56PM

Also experiencing the issue in

https://github.com/chanakin/ClickableTextWithUrlExample, in which double-clicking a UrlAnnotation link in the TalkBack menu does not fire the onClick handler set in a ClickableText

[Deleted User] <[Deleted User]> #12Oct 12, 2022 11:20PM

I created a new issue, hopefully that gets some attention. Those of you who are following along and reporting the same issue, please star it!

https://issuetracker.google.com/issues/253292081

Message last modified on Oct 12, 2022 11:27PM

sr...@gmail.com <sr...@gmail.com> #13Jan 22, 2024 01:15AM

Any Update on this issue?