iOS Pencil Support

[artrelated]

With the addition of the hover feature to the apple pencil it is now possible to use it to move the cursor around without the need to tap first to for example inspect things. Would be a great addition to any point and click type game. Also the pencil supports double tapping on the pencil itself, so it can be used for example for right clicking.

Would it possible to add support for this in a future release?

[rib3ye]

+1

[larsamannen]

I've started to look into adding support for the Apple Pencil.
Can you recommend an app or a game that make use of the pencil in a nice way that I can use as inspiration?

[rib3ye]

Replying to larsamannen:

I've started to look into adding support for the Apple Pencil.
Can you recommend an app or a game that make use of the pencil in a nice way that I can use as inspiration?

Awesome! My daughter and I are super excited for this to happen!

Currently Procreate is the best use of the Pencil that we've seen. I know it's a drawing app and not a game, but they handle a lot of different modes with the pencil.

[artrelated]

Funny enough, I just played Return to Monkey Island with the pencil (you can get it for free, which allows you to play in the first location). There is no special implementation for the pencil, but you can see how the interface was adapted for mobile devices.
For me, the pencil would help most with point and click games, where it's difficult to use fingers to precisely tap where needed. Also, in most cases your finger would cover up the tooltip that describes the item. This can be alleviated by using the pencil, as a more precise tool, which also won't obscure the tooltip. In most games you use left and right mouse click for interact and examine, respectively. Not sure how to go about this with the pencil though. There is the possibility to tap the pencil, also the iPad would sense the pencil hovering over an area without the need to actually touch the screen. Maybe this "sensing" could be used to move the cursor? Though maybe not so easy to always keep the pencil at a certain distance.

Please, share your progress, would be glad to help where possible.

Replying to larsamannen:

I've started to look into adding support for the Apple Pencil.
Can you recommend an app or a game that make use of the pencil in a nice way that I can use as inspiration?

[andshrew]

Does the current version on the App Store have any Apple Pencil support right now?

I’m sure when I manually built it for iPad when adding full screen support for the Pro a few years ago it worked out of the box essentially as a finger replacement, but that doesn’t seem to be the case now.

Just wondering if something is messed up my end, or if something did change and the Pencil needs specific support now.

[artrelated]

that may very well be the case. i have noticed much older apps that work with the pencil without any special adjustments. for example the original idos app, which, i think, predates the pencil by years, works without issues.
a custom implementation could be a nice addition but if it's an easy fix, having it as a simple finger replacement would be a great thing to have.

last version that i tested, the release from a couple of weeks ago doesn't work at all with the pencil. this is the case for over a year now.

Replying to andshrew:

Does the current version on the App Store have any Apple Pencil support right now?

I’m sure when I manually built it for iPad when adding full screen support for the Pro a few years ago it worked out of the box essentially as a finger replacement, but that doesn’t seem to be the case now.

Just wondering if something is messed up my end, or if something did change and the Pencil needs specific support now.

[andshrew]

Thanks for confirming that.

I've gone back and rebuilt my version from a couple of years ago (v2.2.0), and yes the Pencil input does "just work" in that version.

Looking at the current version, input handling has become a lot more robust since then. There's some stuff going on here where the type of every input is being overridden:

​https://github.com/scummvm/scummvm/blob/6c8c73a2452f1384a0428633a9043738f58e94a5/backends/platform/ios7/ios7_touch_controller.mm#L45

If I remove that from the code then Pencil movement is recognised again, at least for moving the cursor around screen. But actual presses aren't, and it looks specific events need to be added to handle Pencil taps on the screen, which is a bit beyond what I can figure out.

So in response to larsamannen comment, it would be nice if basic "finger replacement" input could be restored as a starting point, so pressing or dragging the Pencil moves the cursor and tapping is like a left mouse button click.

Replying to artrelated:

that may very well be the case. i have noticed much older apps that work with the pencil without any special adjustments. for example the original idos app, which, i think, predates the pencil by years, works without issues.
a custom implementation could be a nice addition but if it's an easy fix, having it as a simple finger replacement would be a great thing to have.

last version that i tested, the release from a couple of weeks ago doesn't work at all with the pencil. this is the case for over a year now.

Replying to andshrew:

Does the current version on the App Store have any Apple Pencil support right now?

I’m sure when I manually built it for iPad when adding full screen support for the Pro a few years ago it worked out of the box essentially as a finger replacement, but that doesn’t seem to be the case now.

Just wondering if something is messed up my end, or if something did change and the Pencil needs specific support now.

[andshrew]

I've submitted a PR to add basic support for the Pencil that would have these controls:

Gesture	Description
Touch	Left mouse click
Touch and hold for 0.5 seconds	Left mouse hold and drag (eg. for COMI object interaction)
3 quick taps on screen	Right mouse click

​https://github.com/scummvm/scummvm/pull/6162