Talkback reading of ExposedDropdownMenuBox does not work like the layout system and is buggy [257209915]

Fixed

Bug

Status Update

No update yet.

Description

da...@well.co

created issue #1

Nov 3, 2022 04:17PM

Jetpack Compose version: 1.3.0 Jetpack Compose component used: ExposedDropdownMenuBox,TextField Android Studio Build: 2022.1.1 Canary 2 Kotlin version: 1.7.10

How the layout system would read a dropdown

Focus on drop down it reads: "[SELECTION] drop down list [Drop Down Label]. Double tap to change."
Click on Drop down it reads: "Popup window [FIRST SELECTION] in list. Double tap to activate"
Scroll to second option in popup window it reads: "[SECOND SELECTION] double tap to activate"
Select second option in drop down it reads: "[SECOND SELECTION] drop down list [Drop Down Label]. Double tap to change."

Which the above is pretty reasonable

Compose for the same situation and configuration reads:

Focus on drop down it reads: "drop down menu double tap to activate."
Click on Drop down it reads: Nothing but if you swipe again it reads "[FIRST SELECTION] EditBox Label. Then if you swipe again it reads the Label "Blah 2" as you can see in the video. Swipe again "Window popup window" then swipe again
Scroll to second option in popup window it reads: "[SECOND SELECTION] double tap to activate"
Select second option in drop down it reads: "2 replaced 1" then moves focus to the text below which is "Blah2"

So there is multiple things wrong above. I am not saying it needs to be the same as layout system but the same or better.

Also if you look at second 0:40-0:50 you will see it reads out the drop down and edit box separately. This is not working like a user would expect.

This is a really bad UX and I think the only reasonable work around is to use the layout system for all drop downs.

Appreciate you looking into this ADA issue.

XRecorder_03112022_110635.mp4

1.7 MB

Download

Comments

da...@well.co <da...@well.co> #2Nov 3, 2022 04:18PM

Project: platform/frameworks/support
Branch: androidx-master-dev

commit c60f33e229e31ab328ef6b59dab63b264954831c
Author: Alexandre Elias <aelias@google.com>
Date: Fri Jul 10 16:23:09 2020

Semantics no-op cleanups

Partly in response to lmr's broad code review, I did a pass of
superficial API/implementation cleanup. The main changes are:

- I changed each Boolean SemanticsProperty where false is equivalent to
not being present to take "Unit" instead. This is conceptually
clearer: it avoids questions like "can I cancel out a semantics from a
merged child by setting it to false?" Because "property = Unit" looks
weird, I also changed the style of these to "property()".

- I moved the Semantics id generator closer to where it's used, in
SemanticsModifierCore. I made it internal and an AtomicInt.
(Note that integer ids are heavily used in the Android
AccessibilityNodeInfo APIs so I can't simply remove them entirely.)

- I deleted dead code. Some examples include SemanticsHintOverrides,
a public API not connected to anything, and SemanticsPropertyKey
merge() open method which is never called. (In both cases I have
a different plan in mind for accessibility.)

Fixes: 145951226
Fixes: 145955412
Test: existing tests
Relnote: "Single-value semantics properties now use a calling style.
For example, 'semantics { hidden = true }' is now written as:
'semantics { hidden() }'."

Change-Id: Ic1afd12ea22c926babc9662f1804d80b33aa0cfc

M ui/integration-tests/benchmark/src/androidTest/java/androidx/ui/benchmark/test/LayoutNodeModifierBenchmark.kt
M ui/ui-core/api/0.1.0-dev15.txt
M ui/ui-core/api/current.txt
M ui/ui-core/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-core/api/public_plus_experimental_current.txt
M ui/ui-core/api/restricted_0.1.0-dev15.txt
M ui/ui-core/api/restricted_current.txt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/graphics/vector/VectorTest.kt
M ui/ui-core/src/androidAndroidTest/kotlin/androidx/ui/semantics/SemanticsTests.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidActuals.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeView.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidComposeViewAccessibilityDelegateCompat.kt
M ui/ui-core/src/androidMain/kotlin/androidx/ui/core/AndroidPopup.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/Expect.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsConfiguration.kt
D ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsHintOverrides.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsModifier.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsNode.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsOwner.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/core/semantics/SemanticsWrapper.kt
M ui/ui-core/src/commonMain/kotlin/androidx/ui/semantics/SemanticsProperties.kt
M ui/ui-foundation/api/0.1.0-dev15.txt
M ui/ui-foundation/api/current.txt
M ui/ui-foundation/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-foundation/api/public_plus_experimental_current.txt
M ui/ui-foundation/api/restricted_0.1.0-dev15.txt
M ui/ui-foundation/api/restricted_current.txt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Clickable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Dialog.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/Scroller.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Selectable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/selection/Toggleable.kt
M ui/ui-foundation/src/main/java/androidx/ui/foundation/semantics/FoundationSemanticsProperties.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ButtonTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CardTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/CheckboxScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/RadioButtonScreenshotTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ScaffoldTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SnackbarTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/SurfaceTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/ripple/RippleIndicationTest.kt
M ui/ui-material/src/androidTest/java/androidx/ui/material/textfield/TextFieldScreenshotTest.kt
M ui/ui-material/src/main/java/androidx/ui/material/AppBar.kt
M ui/ui-material/src/main/java/androidx/ui/material/TextFieldImpl.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/AssertsTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/CallSemanticsActionTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ErrorMessagesTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/FindersTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/PrintToStringTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/ScrollToTest.kt
M ui/ui-test/src/androidTest/java/androidx/ui/test/TextActionsTest.kt
M ui/ui-test/src/main/java/androidx/ui/test/Actions.kt
M ui/ui-test/src/main/java/androidx/ui/test/Filters.kt
M ui/ui-text/api/0.1.0-dev15.txt
M ui/ui-text/api/current.txt
M ui/ui-text/api/public_plus_experimental_0.1.0-dev15.txt
M ui/ui-text/api/public_plus_experimental_current.txt
M ui/ui-text/api/restricted_0.1.0-dev15.txt
M ui/ui-text/api/restricted_current.txt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/CoreTextField.kt
M ui/ui-text/src/commonMain/kotlin/androidx/ui/text/TextSemanticsProperties.kt

https://android-review.googlesource.com/1360099

co...@google.com <co...@google.com> #3Nov 9, 2022 06:34PM

Hi commenter #1,

Thank you for reporting this issue.

It sounds like you are using components from compose.material, have you considered migrating to compose.material3 (please see https://developer.android.com/jetpack/compose/designsystems/material2-material3)? Some of the issues have been fixed in Material 3 but difficult to port back to Material 2 because it requires breaking API.

Based on your info, we identified the following problems, please let us know if there are any missing:

The "dropdown menu box" and the "edit text representing the menu" are different focusable elements

This has been fixed in compose.material3

There is no feedback when opening the menu

This has been fixed in compose.material3

Swiping through talkback elements focuses on an element below the menu before cycling through the menu options.

This exists in compose.material3 as well. We will consult with our a11y advisors for what the proper behavior should be.

da...@well.co <da...@well.co> #4Nov 9, 2022 07:36PM

I am not against saying the resolution to the issues will be to migrate to material 3. I dont think we will do it for a while but as long as it works there I am good with that as a solution.

Summary of issues with Layout vs Material 2

Drop down announces differently when focused. Layout: "[SELECTION] drop down list [Drop Down Label]. Double tap to change." Compose: "drop down menu double tap to activate."
Open drop down it reads nothing and then reads the EditBox and then it finally goes into the menu
When you make a selection it reads differently Layout: "[SECOND SELECTION] drop down list [Drop Down Label]. Double tap to change." Compose: "2 replaced 1" then moves focus to the text below which is "Blah2"
It reads drop down and edit box seperately

What version of material3 you testing on. I tried the below code on the material3 android studio template. I updated all versions to latest. In specific compose is 1.3.1 and material 3 is 1.0.1:

Column {
        Text("Blah")
        ExposedDropdownMenuBox(
            expanded = expanded,
            onExpandedChange = {
                expanded = !expanded
            }
        ) {
            OutlinedTextField(
   modifier = Modifier.menuAnchor(),
                readOnly = true,
                value = selectedOptionText,
                onValueChange = { },
                label = { Text("Label") },
                trailingIcon = {
                    ExposedDropdownMenuDefaults.TrailingIcon(
                        expanded = expanded
                    )
                },
                colors = ExposedDropdownMenuDefaults.textFieldColors()
            )
            ExposedDropdownMenu(
                expanded = expanded,
                onDismissRequest = {
                    expanded = false
                }
            ) {
                options.forEach { selectionOption ->
                    DropdownMenuItem(
                        text = { Text(text = selectionOption) },
                        onClick = {
                            selectedOptionText = selectionOption
                            expanded = false
                        }
                    )
                }
            }
        }
        Text("Blah2")
    }

Material 3 results

Focus on drop down it reads: "[SELECTION] EditBox drop down menu [Drop Down Label]. Double tap and hold to to long press. Actions available use tap with 3 fingers to view."
Click on Drop down it reads: "Expanded" swipe "Blah 2", swipe "[Label] window popup window. Double tap to activate"
Scroll to second option in popup window it reads: "[SECOND SELECTION] double tap to activate"
Select second option in drop down it reads: "2 replaced 1" then moves focus to the text below which is "Blah2"

Would be good to have a full ADA audit of this control as it seems to me to still have many issues. I think issue 4 from my list of issues is resolved but all other issues are still existing. Also with step 1 in the material 3 steps above it says actions available. I use 3 fingers to view and it seems to be presenting options that are related to TextField and text selection. I think it would be good to remove any reference to an EditBox with drop downs as well.

Hopefully this is kind of clear as there is alot above.

ya...@well.co <ya...@well.co> #5Oct 18, 2023 01:36PM

Do we have any update here ? this issue still exists in Material 3 upgrade.

ya...@well.co <ya...@well.co> #6Nov 2, 2023 01:29PM

Even the M3 ExposedDropDownBox does not match ADA of ViewSystem, In ViewSystem ExposedDropDown Style announcing as "[Selected Option] Drop Down List [Label] Double tap to change", compared to M3 ExposedDropDown "[SELECTION] EditBox drop down menu [Drop Down Label]. Double tap and hold to to long press. Actions available use tap with 3 fingers to view."

da...@gmail.com <da...@gmail.com> #7Nov 4, 2023 07:28AM

dependencies { implementation "androidx.compose.material:material:1.5.4" }

android { buildFeatures { compose true }

composeOptions {
    kotlinCompilerExtensionVersion = "1.5.3"
}

kotlinOptions {
    jvmTarget = "1.8"
}

}

ap...@google.com <ap...@google.com> #8Apr 5, 2024 07:26PM

Marked as fixed.

Project: platform/frameworks/support
Branch: androidx-main

commit b01be1f0984e1ae035582524be989edc14b83b42
Author: Max Alfonso-Ying <maxying@google.com>
Date: Fri Apr 05 00:22:27 2024

Improve a11y of editable ExposedDropdownMenus

Updated `Modifier.menuAnchor` to support icons in addition
to text fields. Now takes a parameter for MenuAnchorType
so we can set popup flags accordingly. Also added an enabled
parameter to control if the anchor is enabled.

This means editable EDMs can use 2 menu anchors to
support both flows of typing and direct selection when
a11y services are enabled.

Updated sample to show this usage.

Fixes:

b/257209915

b/308840226

Test: Added unit tests for enabled/disabled menuAnchor. Manual testing with TalkBack
Relnote: "`ExposedDropdownMenuBoxScope` no longer permits subclasses.
Exposed dropdown menus now have a `MenuAnchorType` which
should be passed to `menuAnchor` to support better a11y.
This should be used instead of passing `focusable` to
`ExposedDropdownMenu`, which is now deprecated.
`menuAnchor` has a new parameter to control `enabled` state."
Change-Id: I55ee632daf66ef4df90297350cbff901e26ea446

M compose/material3/benchmark/src/androidTest/java/androidx/compose/material3/benchmark/ExposedDropdownMenuBenchmark.kt
M compose/material3/material3/api/current.txt
M compose/material3/material3/api/restricted_current.txt
M compose/material3/material3/samples/src/main/java/androidx/compose/material3/samples/ExposedDropdownMenuSamples.kt
M compose/material3/material3/src/androidInstrumentedTest/kotlin/androidx/compose/material3/ExposedDropdownMenuTest.kt
M compose/material3/material3/src/androidMain/kotlin/androidx/compose/material3/ExposedDropdownMenu.android.kt
M compose/material3/material3/src/androidMain/kotlin/androidx/compose/material3/internal/Strings.android.kt
M compose/material3/material3/src/androidMain/res/values/strings.xml
M compose/material3/material3/src/commonMain/kotlin/androidx/compose/material3/internal/Strings.kt
M compose/material3/material3/src/desktopMain/kotlin/androidx/compose/material3/internal/Strings.desktop.kt

https://android-review.googlesource.com/3028145

na...@google.com <na...@google.com> #9Apr 17, 2024 05:17PM

The following release(s) address this bug.It is possible this bug has only been partially addressed:

androidx.compose.material3:material3:1.3.0-alpha05
androidx.compose.material3:material3-android:1.3.0-alpha05
androidx.compose.material3:material3-desktop:1.3.0-alpha05