[GameActivity] Y coordinate for MotionEvent is not correct when the app is in split screen mode [370676393]

Assigned

Bug

Status Update

No update yet.

Description

bo...@unity3d.com

created issue #1

Oct 1, 2024 11:22AM

Component used: androidx.games:games-activity
Version used: 3.0.5
Devices/Android versions reproduced on: Google Pixel 6/Android 15, OnePlus 8T/Android 14

- Get agdktunnel sample application (

https://github.com/android/games-samples/tree/main/agdk/agdktunnel)
- Modify AndroidManifest.xml by adding android:resizeableActivity="true" to activity
- Add logging to MotionEvent processing method (

https://github.com/android/games-samples/blob/f78c6fcfb175d92ca1450daf2640c97d02ed0f9f/agdk/common/base_game_framework/src/android/platform_events_android.cpp#L147), log should print GameActivityPointerAxes_getY(&event.pointers[0]) and event.pointers[0].rawY values.
- Build and run application
- Make it run in split screen mode with any other application, so the agdktunnel application is in the lower half of the screen.
- Touch agdktunnel view, closer to the upper border
- Check log, logged RawY and Y values will be different, RawY greater than Y by about 1200
- Make agdktunnel run in split screen mode with any other application, so the agdktunnel application is in the upper half of the screen.
- Touch agdktunnel view, closer to the upper border
- Check log, logged RawY and Y values will be equal, both greater than 128 - this is buggy behavior, expected RawY value greater than Y because of the system bar.

Notes:
- The same values can be logged if add onTouchEvent method with logging event.getRawY() and event.getY() values to

https://github.com/android/games-samples/blob/main/agdk/agdktunnel/app/src/main/java/com/google/sample/agdktunnel/AGDKTunnelActivity.java
- Bug is not reproducible if use Activity

Comments

ar...@google.com <ar...@google.com> Oct 2, 2024 11:24AM

Reassigned to ar...@google.com.

be...@google.com <be...@google.com> #2Oct 23, 2024 10:00AM

there are some image for help

ar...@google.com <ar...@google.com> Oct 24, 2024 10:16AM

Accepted by ar...@google.com.

ar...@google.com <ar...@google.com> #3Oct 24, 2024 02:00PM

hello?

bo...@unity3d.com <bo...@unity3d.com> #4Oct 24, 2024 02:13PM

Other Activity where bug is not reproducible is the android.app.Activity (

https://developer.android.com/reference/android/app/Activity), or to be more correct our class which is inherited directly from android.app.Activity (UnityPlayerActivity).
Application is obviously not a fullscreen, there is the system bar above the app.

ar...@google.com <ar...@google.com> #5Oct 24, 2024 03:22PM

I created a new app with empty Activity, added logging in onTouchEvent, and the behavior is the same.

Code:

package com.example.fullscreen

import android.os.Bundle
import android.util.Log
import android.view.MotionEvent
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.tooling.preview.Preview
import com.example.fullscreen.ui.theme.FullscreenTheme

class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        enableEdgeToEdge()
        setContent {
            FullscreenTheme {
                Scaffold(modifier = Modifier.fillMaxSize()) { innerPadding ->
                    Greeting(
                        name = "Android",
                        modifier = Modifier.padding(innerPadding)
                    )
                }
            }
        }
    }

    override fun onTouchEvent(event: MotionEvent?): Boolean {
        Log.d("fs_test_app", String.format("y=%f rawY=%f", event!!.y, event.rawY))
        return super.onTouchEvent(event)
    }
}

@Composable
fun Greeting(name: String, modifier: Modifier = Modifier) {
    Text(
        text = "Hello $name!",
        modifier = modifier
    )
}

@Preview(showBackground = true)
@Composable
fun GreetingPreview() {
    FullscreenTheme {
        Greeting("Android")
    }
}

output:

2024-10-24 16:17:16.461  1479-1479  fs_test_app             com.example.fullscreen               D  y=225.000000 rawY=225.000000
2024-10-24 16:17:17.569  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.580  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.591  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.602  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.613  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.624  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.635  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.646  1479-1479  fs_test_app             com.example.fullscreen               D  y=211.000000 rawY=211.000000
2024-10-24 16:17:17.938  1479-1479  fs_test_app             com.example.fullscreen               D  y=204.000000 rawY=204.000000
2024-10-24 16:17:17.949  1479-1479  fs_test_app             com.example.fullscreen               D  y=204.000000 rawY=204.000000
2024-10-24 16:17:17.960  1479-1479  fs_test_app             com.example.fullscreen               D  y=204.000000 rawY=204.000000
2024-10-24 16:17:17.971  1479-1479  fs_test_app             com.example.fullscreen               D  y=204.000000 rawY=204.000000
2024-10-24 16:17:17.983  1479-1479  fs_test_app             com.example.fullscreen               D  y=204.000000 rawY=204.000000
2024-10-24 16:17:17.987  1479-1479  fs_test_app             com.example.fullscreen               D  y=204.000000 rawY=204.000000
2024-10-24 16:17:33.150  1479-1490  mple.fullscreen         com.example.fullscreen               I  Background concurrent mark compact GC freed 2053KB AllocSpace bytes, 0(0B) LOS objects, 87% free, 3619KB/27MB, paused 264us,5.786ms total 83.692ms
2024-10-24 16:17:46.488  1479-1489  mple.fullscreen         com.example.fullscreen               W  Missing inline cache for long androidx.compose.ui.node.NodeCoordinator.localPositionOf-R5De75A(androidx.compose.ui.layout.LayoutCoordinates, long)
2024-10-24 16:17:46.489  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.505  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.523  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.540  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.551  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.562  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.573  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000
2024-10-24 16:17:46.578  1479-1479  fs_test_app             com.example.fullscreen               D  y=1218.000000 rawY=1218.000000

Screenshot is attached. App is not fullscreen, but Y coordinates are the same.

Screenshot_20241024-161940.png

24 KB

View

Download

bo...@unity3d.com <bo...@unity3d.com> #6Oct 24, 2024 03:33PM

Have you tried to run your app in split screen mode?
I'm going to prepare simple Unity project to demonstrate the difference between Activity and GameActivity, export grade project from it and attach to this bug report.

ar...@google.com <ar...@google.com> #7Oct 24, 2024 03:38PM

Same in fullscreen, coordinates are the same:

 y=688.000000 rawY=688.000000

Unity project is too big, and there is a lot happening behind the curtains. Can you repro the problem with the Empty Activity app?

Screenshot_20241024-163557.png

66 KB

View

Download

bo...@unity3d.com <bo...@unity3d.com> #8Oct 24, 2024 03:53PM

ok, will try with Empty Activity.
but anyway, this can't be expected behaviour, according to

https://developer.android.com/reference/android/view/MotionEvent#getY(int) "The coordinate is in the coordinate space of the view that received this motion event." Otherwise there is simply no way to get touch coordinates in the view space the way it works in all cases.

Message last modified on Oct 25, 2024 09:14AM

bo...@unity3d.com <bo...@unity3d.com> #9Oct 25, 2024 09:14AM

Today I spent some time for experiments with empty Activity application (similar to yours) and with agdksample. I can confirm that empty Activity application has the same problem - RawY is the same as Y and I still think that it's not the expected behavior.
However Unity is using FULLSCREEN flag for the Window, we have this call:
getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
If add this call to the empty Activity onCreate method, everything starts working as expected, RawY is always greater than Y, when app is running alone, when app is in split screen mode in the upper or in the lower half.
I've tested this on Pixel 6 / Android 15, Samsung S10+ / Android 12, OnePlus 8T / Android 14.
As for the agdlksample, adding the same call to the AGDKTunnelActivity.onCreate method doesn't change anything, RawY is the same as Y when app is running alone (but in this case there is no status bar, so it's expected), or when app is in split screen mode in the upper half (which is not expected).
Also I've found one device which behaves differently - Samsung ZFold2 /Android 11, when running apps on the external (portrait) screen.
On this device agdksample works correctly, RawY is the same as Y when app is running alone (it's expected as there's no status bar), RawY greater than Y when app is in split screen mode in the upper or in the lower half (no need to set fullscreen flags).
Empty Activity application without setting fullscreen flags on this device works correctly in split screen mode, but still RawY is the same as Y when running alone. With setting fullscreen flags, everything always works correctly, just like on other devices.

ar...@google.com <ar...@google.com> #10Oct 28, 2024 12:01PM

Assigned to so...@google.com.

The same values can be logged if add onTouchEvent method with logging event.getRawY() and event.getY() values to https://github.com/android/games-samples/blob/main/agdk/agdktunnel/app/src/main/java/com/google/sample/agdktunnel/AGDKTunnelActivity.java

Since you mentioned that input data at Java side is already wrong, this is not an AGDK bug. AGDK is a set of native libraries that serves as an bridge to use Java interfaces in C++ projects like games. Unfortunately I can't help here anymore. I'd ask our product managers to find the team which might be able to address this issue.

bo...@unity3d.com <bo...@unity3d.com> #11Oct 28, 2024 12:15PM

Can this be a bug in GameActivity Java class which is a part of AGDK package (or at least can some workaround implemented there)?

Message last modified on Oct 28, 2024 01:22PM

ar...@google.com <ar...@google.com> #12Oct 28, 2024 03:03PM

No, this can't be a GameActivity bug because inputs to onTouchEvent are already wrong.

Unfortunately, I don't understand what workaround could be made here. First of all, we want to understand if it's indeed a bug or an expected behavior.

so...@google.com <so...@google.com> #13Oct 28, 2024 03:45PM

Reassigned to da...@google.com.

@Daniel, do you know the right large screens team to route this to?