PORTS: ANDROID: Mouse Cursor Snaps to Top Left Corner on Android 8.1.0 Device

[Zeimyth]

I have a Contixo V8 Android tablet running Android 8.1.0. When I launch ScummVM, the application's mouse cursor repeatedly snaps back to the top left corner of the screen. This happens both in touchpad and direct touch mode. I verified the issue happens on the latest nightly build (I can't run the version command, but the in-app menu says "ScummVM 2.3.0git16082-gd60abf0cc2 (May 10 2021 04:09:38)"). (The displayed version text has any descender segments of the font glyphs cut off; hopefully those "g"s are really "g"s.)

The full experience looks something like this:

• Launch the ScummVM app, arrive at main menu
• Touch a location on the menu. Mouse cursor behaves appropriately as long as the touch continues
• Lift finger from screen. Mouse cursor returns to top left corner of the screen

I have also verified that this issue continues throughout the entire main menu (even while navigating through the settings dialog) and while in games as well.

Curiously, the issue goes away temporarily whenever I open my notification bar by dragging down from the top of the screen. As long as the notification bar is visible, the mouse cursor will not snap to the top left. The moment the system hides the notification bar, though, the cursor returns to the top left if I am not in the middle of a touch action.

I compared the behavior with a second Android device (Pixel 3 running Android 11) and the issue does not occur. It seems to be something strange with my Contixo tablet specifically.

[Zeimyth]

I know this has the potential to be an annoying issue to debug, especially if it's somewhat specific to my device. I'm happy to toy around with the code myself to find a solution if someone can point me in the right direction.

[antoniou79]

I haven't had this issue on my devices or on the Android emulation.
I'll make a note to look again into the code for touch gestures and see if cursor co-ordinates are not set/maintained properly.

A cursor "jumping" issue could occur for multi-touch gestures because the co-ordinates of the first finger were kept -- this is a known issue that I thought I fixed, but it can still occur in certain edge cases. Not snapping to top left corner though.

Could it be that the issue is a side-effect from another third party app on your phone or perhaps a touch screen (faulty) hardware issue?

Or a result from rogue dust/dirt on the screen?

There are options way to enable a visual indication of touch gestures in the Android Developer settings (For my phone those are: Show taps (show visual feedback for taps), Show Pointer location (show touch co-ordinates)).
​https://medium.theuxblog.com/enabling-show-touches-in-android-screen-recordings-for-user-research-cc968563fcb9

There are also apps on the Play Store that could possibly provide some info on the issue -- if it's unrelated to ScummVM. Examples:

Touch Screen Test: ​https://play.google.com/store/apps/details?id=jp.rallwell.siriuth.touchscreentest&hl=en_US&gl=US
Touchscreen Calibration: ​https://play.google.com/store/apps/details?id=redpi.apps.touchscreencalibration

Could you test with the Developer option setting for visual feedback and/or the Android apps to see if we can get a better idea of what is happening?

[Zeimyth]

This doesn't appear to be a device touch issue.

I enabled the visual touch and pointer coordinates settings on my device, and things appear to be working as expected. In particular, when I stop touching the device, the pointer coordinates don't suddenly snap to the top left. Also, when I am touching the device, it only reports a single pointer (or two if I am using two fingers), which seems to suggest that the device is not tracking a persistent phantom touch in the top left. I recorded a screen capture if you want to see what my testing looked like, though just a warning that the quality is rather low: ​https://drive.google.com/file/d/1oce1Vdks0UC39JZso3YyRFGFIVxZTjo6/view?usp=sharing

I tried the Touchscreen Calibration app as well and it didn't reveal any issues.

This was a good line of inquiry. I should have thought to test my device's touch debugging information before opening the bug for thoroughness. Unfortunately, it doesn't appear to be what is causing the issue. Another thing I could test would be a different app with a dedicated cursor like ScummVM, but I'm not aware of any like that.

[antoniou79]

Hmm, then I'll look more into our code when I get the time.

Searching with Google, it seems that 8.1.0 specifically had issues with touch interface --although they seems to be with multitouch gestures -- on many devices.
Apparently a fix for that was implemented only in the next Android OS version (9 Pie)

Some mention that they had issues with single touch too. I don't know if it's relevant, but here's a few links with comments or info on that issue:
​https://forum.xda-developers.com/t/8-1-magisk-module-multi-touch-fix-for-8-1-devices.3773150/
​https://android-review.googlesource.com/c/platform/frameworks/native/+/640606
​https://issuetracker.google.com/issues/70344455#comment464
​https://www.reddit.com/r/GooglePixel/comments/87o4s9/pixel_touchscreen_issue_multitouch_in_android_810/

Maybe the Android emulator for a device specifically running 8.1.0 could provide the same effect (?)

[Zeimyth]

I recently obtained a second newer tablet from the same brand. This one is a Contixo V9 running Android 10. One of the first things I tried with the new tablet was installing ScummVM through the Google Play Store. Unfortunately, this new tablet has an identical issue to the Android 8 one. I've only tested it with the Google Play version of the app (2.2.1pre), not the nightly build, but all I wanted to do was see if the newer version of the tablet would make a difference.

I am in the process of trying to get the Android build working on my machine so I can investigate things.

[Zeimyth]

So I've done a little bit of digging.

I found some commented out touch event debugging code ( ​https://github.com/scummvm/scummvm/blob/master/backends/platform/android/org/scummvm/scummvm/ScummVMEventsBase.java#L428-L441 ), which I have uncommented in a local build. I also added my own debug line at the top of MouseHelper.onMouseEvent ( ​https://github.com/scummvm/scummvm/blob/master/backends/platform/android/org/scummvm/scummvm/MouseHelper.java#L169 ). I then performed a single swiping motion across my screen on both the Contixo V9 and my Pixel 3. Curiously, the Pixel did not trigger the MouseHelper debug statements, but the Contixo did. I've attached an output sample from both. It looks like my Contixo device is reporting extra mouse hover events at the start and end of my touch action. According to the Android documentation for the MotionEvent enumeration, 7 is ACTION_HOVER_MOVE, 9 is ACTION_HOVER_ENTER, and 10 is ACTION_HOVER_EXIT.

For thoroughness, the line I added to MouseHelper.java is:

android.util.Log.d("MouseHandler.java onMouseEvent", e.getAction() + ": (" + e.getX() + ", " + e.getY() + ")");

[Zeimyth]

It appears the entry point for the mouse events is MouseHelper.onHover ( ​https://github.com/scummvm/scummvm/blob/master/backends/platform/android/org/scummvm/scummvm/MouseHelper.java#L59 ). Uncommenting L60 and L177 in that file gives output like:

05-22 12:26:51.122 433 433 D ScummVM : onHover mouseEvent 05-22 12:26:51.123 433 433 D MouseHandler.java onMouseEvent: 9: (0.0, 0.0) 05-22 12:26:51.123 433 433 D ScummVM : onMouseEvent buttonState = 0 05-22 12:26:51.125 433 433 D ScummVM : onHover mouseEvent 05-22 12:26:51.126 433 433 D MouseHandler.java onMouseEvent: 10: (0.0, 0.0) 05-22 12:26:51.126 433 433 D ScummVM : onMouseEvent buttonState = 0

I'm uncertain what a "hover" really means in the world of touch inputs. If I hold my finger extremely close to the screen and move it around, I can create a stream of hover debug statements without ever triggering a touch event. I don't know why these are all being reported at coordinates (0, 0), but my device does seem to be picking up something legitimate. I also don't know why I would be getting hover inputs on the Contixo device but not the Pixel device.

[antoniou79]

Interesting. Thank you for these findings.

I am also not so sure about the hover events on a touch interface. It's been a while since I've revised the code. I mean, it wasn't that much long ago, but I suppressed most of the work about it -- it's *that* frustrating working on Android issues sometimes).

After Googling for a bit, people seem to suggest that hover events are mostly for interfacing with a mouse, pen or stylus. I wonder if some devices do extra stuff with it, to detect a finger "hovering" above the surface without touching it. Like perhaps Samsung's "Air View" (​https://www.samsung.com/global/galaxy/what-is/air-view/). And that confuses our implementation.

[Zeimyth]

I created a patched version of the ScummVM Android app for personal use which removes the hover event processing code entirely, and this has fixed the problem well enough that the app is now usable on my Contixo devices, but I recognize that this is not a solution that is suitable for merging into the main codebase. With my hack of a fix, this issue has a much lower priority for me now. However, I am still happy to help however I can with digging into this, particularly in an effort to come up with a code solution for it (rather than disabling some hardware/OS feature (like "Air View", which these tablets definitely do not have)). I'm willing to run any potential fix or investigation apps on one of my devices for verification or debugging purposes.

[antoniou79]

Replying to Zeimyth:

I created a patched version of the ScummVM Android app for personal use which removes the hover event processing code entirely, and this has fixed the problem well enough that the app is now usable on my Contixo devices, but I recognize that this is not a solution that is suitable for merging into the main codebase. With my hack of a fix, this issue has a much lower priority for me now. However, I am still happy to help however I can with digging into this, particularly in an effort to come up with a code solution for it (rather than disabling some hardware/OS feature (like "Air View", which these tablets definitely do not have)). I'm willing to run any potential fix or investigation apps on one of my devices for verification or debugging purposes.

Could you please share your exact patch?
I haven't really put time to look further into this, but if it's not something like an obvious bug in the code (that is triggered on only some devices), then we could potentially add a backend specific checkbox to disable the hover code as a workaround.

[Zeimyth]

I'm not sure what the best way is for me to share the patch, but I think you're right that it would be a good candidate for a checkbox workaround.

I pushed to a commit in my own fork of the repo: ​https://github.com/Zeimyth/scummvm/commit/f235fad2b80cb81a376449023ed996ef7ea08801 Hopefully this is sufficient. :)

[antoniou79]

A quick question; do you think that a user with this issue would be able to navigate properly to the setting, click on the checkbox and Apply the workaround or would the mouse snapping to the top corner make this difficult or even impossible?

[Zeimyth]

It is doable but tricky. Because the default mode for ScummVM is touchpad mode rather than absolute touch mode, you have to touch the screen far enough in the top left corner to drag the cursor all the way to the options menu in one attempt. Then, without lifting your finger (since that would reset the cursor), you have to perform a click to select the menu button. You have to repeat the process to navigate the options menu (usually easier because the cursor doesn't have to travel as far). It's tedious and I only have moderate success doing it.

[lephilousophe]

First, is it possible to check again with latest version?

Then, is it possible to try something else?
Before the line return onMouseEvent(motionEvent, true);
Add:

if (!isMouse(motionEvent)) {
    return false;
}

That would have the effect to ignore hover events from touchscreens.
I am not sure we want them anyway.