Async mouse input on windows

[Susko3]

• Related: Investigate receiving input without using window event flow #6141
• Should fix: Input thread blocking causes mouse handling stutters on windows #6181

Depends on:

• Upgrade to SDL3 #6234

This PR simply handles relative mouse and keyboard events in the SDL_EventFilter i.e. HandleEventFromFilter.

This works because SDL queries raw mouse and keyboard inputs on a separate thread. Our SDL_EventFilter will intercept those events before they are pushed to the internal SDL queue, handle them, and then request SDL to drop them.

Testing

Apply the diff and toggle relative mouse mode in TestSceneInputManger.
This isn't a perfect simulation of SDL_PumpEvents() lag, but it shows the async input in action.

diff --git a/osu.Framework/Platform/SDL3Window.cs b/osu.Framework/Platform/SDL3Window.cs
index aac3baf44..737547203 100644
--- a/osu.Framework/Platform/SDL3Window.cs
+++ b/osu.Framework/Platform/SDL3Window.cs
@@ -460,6 +460,7 @@ private void setSDLIcon(Image<Rgba32> image)
         private void pollSDLEvents()
         {
             SDL3.SDL_PumpEvents();
+            Thread.Sleep(100);
 
             int eventsRead;

[peppy]

This seems to break input on my VM setup. Yes I have a weird setup but it did work, so something has potentially changed or broken. I haven't yet investigated why, but can if required (if the issue isn't obvious).

master:

2024-04-15.15.35.29.mp4

this PR:

2024-04-15.15.34.02.mp4

[peppy]

As commented

[Susko3]

Hmm, it's weird that the relative mouse movements align with the cursor in one scenario but not the other. But I wouldn't call it broken, the framework cursor is still moving proportionally to your mouse movements.

The weirder thing is that the mouse cursor isn't hidden when in relative mode.

I have no idea what could cause any of this. But if you'll be looking at this, don't forget to enable event logging with

SDL3.SDL_SetHint(SDL3.SDL_HINT_EVENT_LOGGING, "2"u8);

[peppy]

the framework cursor is still moving proportionally to your mouse movements

I tried to show this, but it's actually not proportionally correct either. I'll see what I can find.

[peppy]

@Susko3 can you please check back on this? a lot of upstream changes meant that I basically had to reapply your changes here.

I'm also curious when we should not be using the filter method for handling things. I guess only when we want thread synchronisation on the input thread?

[Susko3]

I'm also curious when we should not be using the filter method for handling things. I guess only when we want thread synchronisation on the input thread?

I'm not sure. Maybe it's safe to handle all input events in the watch. Especially if we're not relying on SDL state and are just consuming fields from the SDL_Event structs. https://wiki.libsdl.org/SDL3/SDL_SetEventFilter mentions that the filter will "process all events before they change internal state", but this is not 100% true. Some internal state is updated before the event is pushed and the filter is called.

See the following review for some thread unsafety, and methods that query SDL state:

[Susko3]

This calls SDL_GetWindowRelativeMouseMode() which could be considered as reading SDL state in a event filter, but it's currently just reading a window flag.

[Susko3]

These don't directly depend on SDL state, but they do depend on SDL_EVENT_GAMEPAD_ADDED, SDL_EVENT_GAMEPAD_REMOVED, etc. for correct handling. By handling some events in the filter, we may reverse the order these events are understood by o!f. controllers is also just a regular dictionary, so it's not thread safe.

[Susko3]

I'm fine with moving input handling to the filter, but we need to ensure thread safety (in our code, and minimising SDL function calls). Some testing is needed.

I think it's best to get the high-priority changes in (mouse & keyboard) and leave the rest for later.

Mouse events seem fine to handle in the filter, mousePressedButtons is be modified in a thread-unsafe way, but that's easy to fix. Calling SDL_GetWindowRelativeMouseMode() seems fine.

Hmm, I'm not too sure about keyboard input. It may mess up our textbox handling because the order of events might be changed.

[peppy]

@Susko3 sounds good. i think i'd like to make moving keyboard input there a goal (because otherwise it will run into the same issues as mouse). did you want to make any follow up changes?

[Susko3]

@Susko3 sounds good. i think i'd like to make moving keyboard input there a goal (because otherwise it will run into the same issues as mouse).

Yep, we can keep the original issues open. Having only mouse input async will make the issue less obvious, as the mouse is smooth, but the keyboard is delayed. Keyboard input needs additional consideration because SDL_HINT_WINDOWS_RAW_KEYBOARD is off by default.

did you want to make any follow up changes?

I wanted to make handleMouseButtonEvent() / mousePressedButtons thread-safe, but it will conflict with #6488. Changes in that PR also make it easier to implement thread safety, so it'll be easier if that one goes in first.

I will try to use System.Threading.Interlocked as it's a hot path and it's just setting integer flags.