Web touch event

[ryo33]

• [x] Tested on all platforms changed
• [x] Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
• [x] Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
• [x] Created or updated an example program if it would help users understand this functionality
• [x] Updated feature matrix, if new features were added or implemented

Closes #1673

This PR is inherited from inactive PR #1945.

The differences are as follows:

• Correctly calculates the physical position by treating with scale factor and flipping the Y-axis.
• Support touch pressure.
• add touch-action: none to a canvas to prevent the browser from canceling pointer events.
• Working demo (pressure will be reflected in the cursor size).

Source code of the demo is here

[ryo33]

I resolved the conflicts.

[ryo33]

Also, I update the demo with the latest code

[ryo33]

Thanks for reviewing!

[ryo33]

@ryanisaacg Could you review this and suggest another reviewer for this?

[ryanisaacg]

I haven't been involved in any winit development for a few years at least; I'm not a good reviewer any longer. (Besides, it does look like you have an approving review?)

[ryo33]

I referred https://github.com/rust-windowing/winit/wiki/Testers-and-Contributors This list seems to be not maintained. I need one more review as written in CONTRIBUTING.md. Can someone review and merge it for me?

[maroider]

This list seems to be not maintained.

It is maintained. Ryan is marked as E for Expert, which means he is (or at least was) willing to answer questions regarding the platform, but isn't volunteering to review any code or do any other maintainer and/or contributor tasks.

I need one more review as written in CONTRIBUTING.md.

The rule, as written, is two reviews from maintainers. The rule isn't really enforced all that strictly, in the interest of getting things done with our limited manpower.

Can someone review and merge it for me?

I can have a look at this once I've cleared out a couple of other things from my to-do list.

[ryo33]

Very helpful. Thank you so much!

[johanhelsing]

I tested this out again on my game (using winit through bevy_winit). I get the expected touch events, however I now also get mouse events at the same time, which does not match the behavior on linux at least. Also, looking at the docs for WindowEvent::MouseInput I couldn't fint a way to tell whether a mouse button event originated from touch or mouse.

Maybe the best solution would be to add checks to the pointer event handlers and ignore events if PointerEvent::pointer_type == "touch"? At least this seemed to do the trick for my game.

like this https://github.com/johanhelsing/winit/commit/736168e29c92bd2ba507cf9a084c3156a0416c74 ?

Thanks for picking this up, btw. Looking forward to using it!

[ryo33]

Thank you for testing this and giving me a feedback. I’ll add the checks to the pointer event handlers like your commit when I have a time.

[ryo33]

I merged @johanhelsing's branch. thanks!

[Zageron]

Any intent to merge and close this out in the near future? :)

[johanhelsing]

Any intent to merge and close this out in the near future? :)

As far as I understood, what remains to be done is just remove the workaround for the scale factor.

And also fix the conflicts, I guess.

[ryo33]

I'll work within this week.

[oscrim]

Hello, is this PR still worked on? I could very much use this in my project :D

[ryo33]

I am taking some time off for mental reasons. It's a simple fix and I'd like to merge it as soon as someone sends a pull request to my branch.

[oscrim]

Alright, Ill give it a go

[ryo33]

Thanks! @oscrim I’ve merged it.

[ryo33]

I've restored flipping and updated the demo (it works well).

[ryo33]

I've resolved conflicts and failed checks

[ryo33]

• [x] have one callback per event
• [x] correct use of prevent_default
• [x] change the changelog expression with "breaking change"
• [x] update the documentation for the Touch structure / the WindowEvent::Touch to indicate which platforms this is supported on.
• [x] update the DEMO

[gilescope]

Tried it. Touch events are coming through for me on wasm in firefox responsive mode and on firefox on iphone. I don't seem to get TouchpadMagnify gestures coming through - that would be nice to have but not essential (as you can probably compute it from the touch events).

[gilescope]

" - Only available on macOS." Ok. so that explains that one. If someone writes a nice gesture detection lib then let me know as I really can't see why we can't build something generic above the touch events that works on all OSes.

[gilescope]

One weird thing, the y axis seems inverted - needs to be height - y. Top left should be 0,0. egui touch handling and my brain expects this, but it seems that bottom left is currently 0,0. I'm assuming that oddness is coming from this PR.

[ryo33]

Oh, I may misunderstand the treatment of the y-axis in winit.

https://docs.rs/winit/latest/winit/event/enum.WindowEvent.html#variant.CursorMoved

(x,y) coords in pixels relative to the top-left corner of the window. Because the range of this data is limited by the display area and it may have been transformed by the OS to implement effects such as cursor acceleration, it should not be used to implement non-cursor-like interactions such as 3D camera control.

So we don't need y-axis flipping?

[gilescope]

Personally I don’t think so. Have you got a second flip that offsets it in the web demo? Just wondering why that one works.

On Sun, 13 Nov 2022 at 06:36, Ryo Hirayama @.***> wrote:

Oh, I may misunderstand the treatment of the y-axis in winit.

https://docs.rs/winit/latest/winit/event/enum.WindowEvent.html#variant.CursorMoved

(x,y) coords in pixels relative to the top-left corner of the window. Because the range of this data is limited by the display area and it may have been transformed by the OS to implement effects such as cursor acceleration, it should not be used to implement non-cursor-like interactions such as 3D camera control.

So we don't need y-axis flipping?

— Reply to this email directly, view it on GitHub https://github.com/rust-windowing/winit/pull/2188#issuecomment-1312651136, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGEJCEFF7OIY435XLVMONTWICD5RANCNFSM5NZDJQ4A . You are receiving this because you commented.Message ID: @.***>

[mockersf]

So we don't need y-axis flipping?

I'm trying this PR in Bevy, and I had to flip back the Y axis in Bevy for it to work. ~~So I would say that it isn't needed.~~ Nevermind I think it's expected in Bevy to flip it for now.

[donmai-me]

I've tried the demo on my iPad and it works great! However, there are two problems I've encountered. First is that touch events using Apple Pencil don't register. Second is that canceled touch events, such as when switching windows when using four-finger swipes, aren't removed.

The last one may be because the demo's source code doesn't handle canceled touch events.

[ryo33]

Apple Pencil emits pointer events with pointer_type = "pen". Does anyone know how winit handles Apple Pencil on iOS target? If it handles it as touches, we need to handle the "pen" pointer type.

[ryo33]

I'm finally confident that we don't need the y-flip, and it's ready to merge. I don't think we should support pen events for now because other platforms do not support it yet, as #99.

Demo improvement (now it's really simple, comprehensive, and easy to understand)

See demo and its source.

Changes are:

• Use UI instead of Sprite because bevy's UI and winit share the same coordinate system (so no complex position transformation).
• Handle the touch cancel event (thanks to @donmai-me for reporting).