Add ability to inject keyboard input [229831515]

Fixed

Feature Request

Status Update

No update yet.

Description

iz...@google.com

created issue #1

Apr 20, 2022 10:59AM

Currently, we have SemanticsNodeInteraction.performXyzInput {/* XyzInjectionScope functions */} for touch, mouse, rotary scroll, and multi modal input types.

This pattern should be extended to allow for keyboard input injection. Specifically, it should be able to inject the following types of keyboard event:

Keycode down/up
Meta keys down/up
Possibly also Attaching/detaching keyboard

Comments

iz...@google.com <iz...@google.com> Apr 20, 2022 10:59AM

Accepted by iz...@google.com.

ap...@google.com <ap...@google.com> #2May 23, 2022 03:22PM

Project: platform/frameworks/support
Branch: androidx-main

commit 2de6989fa4b50214b35e26073b66973ff70dc30e
Author: Izer Onadim <izeronadim@google.com>
Date: Mon May 16 15:49:00 2022

Add key injection API for Compose UI tests

Adds a `KeyInjectionScope` similar to the existing Touch, Mouse and
Rotary injection scopes. Adds support for keyboard input injection
in `MultiModalInjectionScope` and `InputDispatcher` as well as a
platform-specific implementation of `KeyEvent` injection in
`AndroidInputDispatcher`.

The API in `KeyInjectionScope` is currently experimental and has
been annotated with `@ExperimentalTestApi` since it makes use of the
experimental `androidx.compose.ui.input.key` API for representing
keyboard keys in a platform independent way.

Bug: 229831515
Test: ./gradlew compose:ui:ui-test:test
&& ./gradlew compose:ui:ui-test:cC
Relnote: "Added experimental support for injecting key events. Use
`performKeyInput` to send key events, or send them through the `key`
property of `MultiModalInjectionScope` during a multi modal input
gesture with `performMultiModalInput`. See `KeyInjectionScope` for
documentation of the API."

Change-Id: Ic5000922484274309323ee4d13c9941e0d737673

M compose/ui/ui-test/src/test/kotlin/androidx/compose/ui/test/inputdispatcher/MouseEventsTest.kt
M compose/ui/ui-test/api/current.txt
M compose/ui/ui-test/api/1.2.0-beta03.txt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/Common.kt
A compose/ui/ui-test/src/test/kotlin/androidx/compose/ui/test/inputdispatcher/KeyEventsTest.kt
M compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/MultiModalInjectionScope.kt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyUpTest.kt
A compose/ui/ui-test/src/androidCommonTest/kotlin/androidx/compose/ui/test/util/InputEventRecorder.kt
M compose/ui/ui-test/api/restricted_current.txt
M compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/Actions.kt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyDownTest.kt
A compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/KeyInjectionScope.kt
M compose/ui/ui-test/src/test/kotlin/androidx/compose/ui/test/inputdispatcher/TouchEventsTest.kt
M compose/ui/ui-test/api/restricted_1.2.0-beta03.txt
M development/build_log_simplifier/messages.ignore
M compose/ui/ui-test/src/androidMain/kotlin/androidx/compose/ui/test/AndroidInputDispatcher.android.kt
D compose/ui/ui-test/src/androidCommonTest/kotlin/androidx/compose/ui/test/util/MotionEventRecorder.kt
M compose/ui/ui-test/api/public_plus_experimental_current.txt
M compose/ui/ui-test/src/test/kotlin/androidx/compose/ui/test/inputdispatcher/InputDispatcherTest.kt
M compose/ui/ui-test/src/androidCommonTest/kotlin/androidx/compose/ui/test/util/Verifications.kt
M compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/InputDispatcher.kt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/util/TestTextField.kt
M compose/ui/ui-test/api/public_plus_experimental_1.2.0-beta03.txt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyPressTest.kt
M compose/ui/ui-test/src/desktopMain/kotlin/androidx/compose/ui/test/DesktopInputDispatcher.desktop.kt

https://android-review.googlesource.com/2098982

ap...@google.com <ap...@google.com> #3May 25, 2022 11:39AM

Project: platform/frameworks/support
Branch: androidx-main

commit c89f385724e8b3177b2ace26dd35b59ed1fb6652
Author: Izer Onadim <izeronadim@google.com>
Date: Mon May 23 17:50:43 2022

Add lock key state getters to key injection API

Adds getter methods for determining the states of lock keys to
`KeyInjectionScope`. Also adds a function that enables injecting
the same key repeatedly a given number of times.

Additionally, adds code to pass the meta and lock key state of the
keyboard to the mouse injection functions, to facilitate multi-modal
input such as ctrl+click.

Bug: 229831515
Test: ./gradlew compose:ui:ui-test:test
&& ./gradlew compose:ui:ui-test:cC
Relnote: "Added `pressKeyTimes` as well as `isCapsLockOn` and friends
to `KeyInjectionScope`. Additionally, the API now supports mouse and
keyboard combined injection patterns such as clicking a mouse button
with a meta key held down."

Change-Id: I4c8dae112eeb7526cbc32b9bce90d7e3bb5ce73a

M compose/ui/ui-test/api/restricted_1.2.0-beta03.txt
M development/build_log_simplifier/messages.ignore
M compose/ui/ui-test/src/androidMain/kotlin/androidx/compose/ui/test/AndroidInputDispatcher.android.kt
M compose/ui/ui-test/api/current.txt
M compose/ui/ui-test/api/public_plus_experimental_current.txt
M compose/ui/ui-test/api/1.2.0-beta03.txt
M compose/ui/ui-test/src/test/kotlin/androidx/compose/ui/test/inputdispatcher/KeyEventsTest.kt
A compose/ui/ui-test/src/test/kotlin/androidx/compose/ui/test/inputdispatcher/multimodal/KeyAndMouseEventsTest.kt
M compose/ui/ui-test/src/androidCommonTest/kotlin/androidx/compose/ui/test/util/Verifications.kt
M compose/ui/ui-test/api/restricted_current.txt
M compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/InputDispatcher.kt
M compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/KeyInjectionScope.kt
M compose/ui/ui-test/api/public_plus_experimental_1.2.0-beta03.txt
M compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyPressTest.kt

https://android-review.googlesource.com/2102714

ap...@google.com <ap...@google.com> #4May 27, 2022 01:10PM

Project: platform/frameworks/support
Branch: androidx-main

commit 4461a3ad26e0321740aefa15ff495553185f7cd1
Author: Izer Onadim <izeronadim@google.com>
Date: Fri May 27 10:56:55 2022

Add higher-order functions to key injection API

Adds higher-order functions to `KeyInjectionScope` designed to
provide more readable and maintainable ways of injecting certain
key presses while other keys are held down or toggled on.

Also adds properties that allow the checking of a meta key's state,
such as `isCtrlDown`. This verifies whether either one of the
control keys is down.

Bug: 229831515
Test: ./gradlew compose:ui:ui-test:test
&& ./gradlew compose:ui:ui-test:cC
Relnote: "Added higher-order functions to `KeyInjectionScope` for
injecting key presses while other keys are held down or toggled on.
These functions include `withKeysDown`, `withKeysToggled` etc. Also
added properties for checking if a particular meta key is down, for
example, `isCtrlDown` to check if either control key is depressed.
Refer to `KeyInjectionScope` for documentation on each function."

Change-Id: I9f6cdab059c04f4af089a28f99dd267747c723de

M compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyDownTest.kt
M compose/ui/ui-test/src/commonMain/kotlin/androidx/compose/ui/test/KeyInjectionScope.kt
M compose/ui/ui-test/api/public_plus_experimental_current.txt
M compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/Common.kt
M compose/ui/ui-test/api/public_plus_experimental_1.2.0-beta04.txt
M compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyPressTest.kt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/LockKeysTest.kt
A compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/MetaKeysTest.kt
M compose/ui/ui-test/src/androidAndroidTest/kotlin/androidx/compose/ui/test/injectionscope/key/KeyUpTest.kt

https://android-review.googlesource.com/2104528

iz...@google.com <iz...@google.com> #5Aug 22, 2022 09:02AM

Marked as fixed.

Tentatively marking this as fixed since all core functionality is now in place. Related work items that should be picked up in order to tie off loose ends include:

b/233186704 (bug fix)
b/233199964 (desktop implementation)

Issue 229831515