IssueTracker

First of all thanks for this detailed issue.

This issue had been investigated thoroughly when it was first reported internally. The surprising detail in this report is that the issue is not reproducible before 1.7. I will look into this.

The main problem with POBox is the fact that it is deprecated. Since 2021 Sony has been shipping new Xperia devices with Gboard pre-installed. Although we are aware that there is still a considerable amount of users still using POBox, the described behavior is caused by POBox's noncompliant behavior with InputConnection and InputMethodManager documentation. However, this is understandable since TextView implementation was also not respecting the behavior that is expected from Editors.

Ultimately we have decided to enforce the documented behavior with specifically regards to when editors should call InputMethodManager.updateSelection. Also, although unconfirmed, there were traces of possible custom code being included in Sony OEM images that changed how InputMethodManager was notified from TextView. If POBox also depended on something like this, it would be impossible for Compose code to replicate the same unknown behavior.

Just to be sure, it is dangerous to replace Compose TextField with Android View EditText as a workaround for this issue.

Compose 1.7 has a bug that causes ANR when the focus is on EditText. https://issuetracker.google.com/issues/369354336

Another View-related bug in Compose 1.7 is that an Android View is focused by calling FocusManager.clearFocus(). https://issuetracker.google.com/issues/374031296

Perhaps there is a lack of testing of Compose 1.7 in combination with Android View. There is also a possibility that there are other fatal bugs related to View.

In other words, the only options for apps targeting the Japanese market that require POBox support are to continue using Compose 1.6 or to use EditText in combination with various workarounds.

Project: platform/frameworks/support
Branch: androidx-main
Author: Halil Ozercan <halilibo@google.com>
Link:      https://android-review.googlesource.com/3317256

Fix POBox keyboard issue

Fix POBox keyboard issue 
 
Fix: 373743376 
Fix: 329209241 
Test: NullableInputConnectionWrapperTest 
Change-Id: I94e0e598274fb88b255f977f9fbd50dfbbb1ecb1

Many thanks again for this report. Especially for giving us a huge clue in terms of what could be going wrong. The fix is now merged and I will ask for a cherry-pick into a stable release.

Yes, this fix is planned to be included in a future 1.7.x release.

Thanks for the fix. Sorry to follow up on this. is it possible for you to share specific release version/date for the stable version? We are waiting on this to decide on our direction.

Yes, disambiguating based on zIndex sounds like a good idea, I can't think of any reason not to do it that way. It looks like the reason ViewGroup didn't have it is simply because View System didn't have zIndex (it supported translationZ, but structurally zIndex is better suited to calculate overlap order without requiring any matrix math).

"allow navigation between fixed components on screen like the app bar and fab " seems to be a feature which should implemented in talkback.

Agree.

I think it's important to distinguish between properly describing the UI and correct TalkBack behavior.

Agree.

My suggestion was that it should be possible to describe the nature of components like FABs and other app chrome like appbars by being able to mark them as chrome, similar to how it's possible to mark arbitrary views as headings. This needs to be implemented in compose and/or the view system since this is what the app developer has access to.

It adds metadata about these components that accessibility services can then use (which is separate). TalkBack could add it as a navigation type like headings, paragraphs, words etc. which allows the user to jump between layers. TalkBack could use it in the autoscroll case that Phill mentioned too (iterate through app chrome before autoscroll - though it does sound annoying, maybe it could be an opt-out setting). Switch access could incorporate it too, since it has the same issue.

Hey, so we are working on App Widgets, and one of the layout we are working with is this: http://screen/3DhenbTZNKtEqXw (only showing the silhouette, as this is under development).

There are five locations for buttons, and iterating through them should probably be from more important to less important, meaning the center first, and then around in a circle, starting probably from top-left, although this might depend on whether the UI is LTR or RTL and on the exact application.

Blocked by a private issue 😪

Agree with #23. It would be ideal to have the tools and implement them on a case-by-case for some of our more complex components. I don't believe anyone wants to make inconsistent behaviour for end-users but getting to that consistent behaviour should be available to us if the default implementation can't reach it in this case. It's problematic for us when our expected traversal order in design (which may be set by accessibility analysts) doesn't meet our actual implementation and sometimes the design/implementation refactor costs are high.

While not on compose, there's been issues in the past with traversal where even with refactors we couldn't get an expected result leading to difficult decisions, which is not ideal for end-users.

Let us take responsibility for meeting the compliance, consistency, and accessibility of our own implementation.

Thanks for the feedback. What I would find helpful to make progress here is to hear the concrete cases you're running into where the default order deviates from the one your a11y analysts recommend. Feel free to file those as separate bugs under the following component: https://issuetracker.google.com/issues/new?component=1070466

Then, case by case, we could: 1) suggest a technical workaround to make the order conform without redesigning your app, 2) fix our implementation so that the default order conforms to what you want, or 3) acknowledge that it is technically impossible right now and block those bugs on this feature request. If we wind up accumulating a critical mass of case 3), that would justify putting this on our roadmap, as well as making sure that it supports all the motivating use cases.

Here is an example:

I have a list item (see screenshot) where there is Text, Detail Text, and Meta text. I want these to be announced in that same order.

However, talkback announces the order by default like "Text, Meta text, Detail text".

I believe this is because my layout is like this:

Column {
    Row { 
        Text()
        MetaText()
    }
    DetailText()
}

So I think the only way I could workaround this is by creating a custom layout that places the composables in the same order that they should be read out loud in talkback. But this isn't very convenient.

Another use case: https://github.com/google/accompanist/issues/1254

In short, there is a topBar in a Scaffold that is being placed in the same location as the content slot (and the content is given the space that the top bar occupies as padding values via subcomposition)

Since topBar and content are placed at the same location, the logic in SemanticsSort eventually choose the content first, since it is a taller element: https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:compose/ui/ui/src/commonMain/kotlin/androidx/compose/ui/semantics/SemanticsSort.kt;l=130;drc=627183600bfd9920c47bd26bdfe5ec906b54d2a7

That is unexpected, however, since the topBar will visually be the top-most element. There doesn't appear to be any way currently to affect this ordering, other than by changing where the topBar and content are placed.

The same issue is also currently affecting the M3 Scaffold for the same reason.

Indeed, so far, more than half the focus-traversal problem cases I've heard involve a top bar. We'll see what we can do to improve that particular use case.

Here is another example where we think it would be nice to have an API to specify the custom accessibility focus order. Have a look at this repo. We have a LazyRow of tabs and a HorizontalPager. When used with TalkBack, the user has to scroll through all the pages back to the first element of the first page to finally jump out of the Pager and to return to the tabs. The tabs are there to quicken the navigation (the user selects a tab, then the Pager goes straight to its content). So we think it would be nice to allow the user to exit from the current page in the middle of the Pager straight to the active tab - not to the previous page. Maybe, it would also be helpful to automatically focus the page right after the user selects a tab, so it is not required to scroll through all the rest tabs in LazyRow to reach the now-active page.

Hello, anybody can show a use case of:

modifier = modifier
        .semantics {
            requestFocus(label = "label", action = {
                anyAction()
                true
            } ...

For handle talkback focus

Or:

modifier = modifier
            .semantics {
                focused = false
               ...
to lost a focus and recovery again.

Thanks!

Any news on when this fix will actually hit a release? The changes look beautiful!

The new traversalIndex/traversalGroup SemanticsProperties will first be released in 1.5.0-beta01, scheduled for mid-next-week. We think it should be a flexible tool to improve a11y focus order for many of the known use cases, I hope it solves your problem!

If it doesn't behave as you expect, please post a video along with sample code, and we'll investigate. Also, please note that there are still known issues with TalkBack initial focus after navigation which traversalIndex is not yet able to solve (see b/272065229), so we plan to continue to add more features in this area in Compose 1.6.

Greatly appreciated. My use case will definitely put it to the test!

Thank you so much for the travesalIndex/traversalGroup release! It has mostly worked as expected. The one issue I am seeing is when the parent has semantics(mergeDescendants = true) {}. I made an example:

class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            TraveralTestTheme {
                TraversalExample(
                    modifier = Modifier
                        .semantics(mergeDescendants = true) {  }
                        .background(color = Color.Gray)
                        .fillMaxSize()
                )
            }
        }
    }
}

@Composable
fun TraversalExample(
    modifier: Modifier = Modifier,
) {
    Row(
        horizontalArrangement = Arrangement.Center,
        verticalAlignment = Alignment.CenterVertically,
        modifier = modifier.semantics { isTraversalGroup = true },
    ){
        Column {
            Text(
                text = "I should be announced first",
                modifier = Modifier
                    .padding(16.dp)
                    .semantics { traversalIndex = 1f },
            )
            Text(
                text = "I should be announced third",
                modifier = Modifier
                    .padding(16.dp)
                    .semantics { traversalIndex = 3f },
            )
        }
        Text(
            text = "I should be announced second",
            modifier = Modifier
                .padding(16.dp)
                .semantics { traversalIndex = 2f },
        )
    }
}

The announcement order is: 1,3,2.

The announced string is: "I should be announced first, I should be announced third, I should be announced second"

Got it. traversalIndex's current implementation only control the order that TalkBack moves its focus box around: we hadn't yet considered the use case of controlling the announcement order within a single merged element. Feel free to file a separate bug on that.

For now, clearAndSetSemantics { contentDescription = "I should be announced first, I should be announced second, I should be announced third" } could be an alternate way of solving this, although I recognize clearAndSetSemantics has downsides and it's usually better to avoid it.

OK. I will do that! Thank you very much. I have noticed other interesting behavior but haven't been as successful in reproducing it. We think some of the behavior is related to this:

Also remove the idea of 'structurally significant' to only talk about traversalGroups by adding isTraversalGroup = true to Foundation classes (Modifier.scroll semantics and LazyLayoutSemantics). This way, all scrollables are considered traversalGroups in preparation of aosp/2491418 and go/traversal-index-changes.

as we have solved some of our issues by using Modifier.semantics { isTraversalGroup = false } with scrollable components. The basic description of the issue "the scrollable item seems to be moved deeper into the traversal tree than its peers as it is skipped until the next layer is announced. However, it is properly announced if isTraversalGroup is set to false".

OK, that sounds like another thing worth filing a separate bug for. We can use your example either to improve our default behavior, or to provide guidance on when to specify isTraversalGroup = false. Please include two videos of what you are seeing (with isTraversalGroup = false on the scrollable, and without).

Of course. Thank you!

I made the issue for that first bug! https://issuetracker.google.com/issues/285043826

Thanks for this long waiting feature. During integration we faced runtime crash when we switch compose screens: java.lang.IllegalStateException: LayoutNode should be attached to an owner

Similar issue: https://issuetracker.google.com/issues/285195974

Marking this bug fixed because the new traversalIndex & traversalGroup SemanticsProperties provide the desired manual control of focus order and they are scheduled for release soon in Compose 1.5.

If anyone still can't find a way to solve their use case after experimenting with them, please file a new bug and post a video along with sample code.