Entering recorded data regardless of system layout

[EugeneRymarev]

Missing feature

https://github.com/mooltipass/minible/issues/413

Justification

This update will help make the device easier to use for those who have more than one layout on their system.

This will also allow you to enter credentials with mixed layouts.

Workarounds

Manual layout switching

[My1]

The idea is cool, not sure if possible tho for moolticute to grab the current keyboard layout (and let's not get me started with IME states of eg a japanese setup)

Don't forget the system sees the mp device primarily as a keyboard and a keyboard just sends scancodes, which the computer then interprets against the layout table)

[limpkin]

for that use case we'd need moolticute to simulate key presses in the currently opened program.

[My1]

or maybe even better if we have MC anyway, couldnt MC just populate the input field directly (with the added bonus that you wouldnt need to care about the current keymap at all)? otherwise if MC can just read out the current key map and transmit that to the MP the MP would stay doing its keyboarding but with the layout awareness (although obviously no idea if a program can fetch the current keymap)

also an added annoyance of MC-based key input is that elevated windows wouldnt work unless MC itself is also elevated (not sure if that's a good idea in general), if MC can grasp the kb layout and relay that to the MP that wouldnt need to interact with elevated processes

[nekromant]

I looked a little bit into this issue. Qt has a KeyboardLayoutChanged Event that MC the daemon can catch as well as it can query the system for the keyboard layout. This isn't something you can do via pure HID class.

So as it seems, the logic would be: Send a request to MC to notify we're about to type something, so that it can switch layouts via simulated keypress or directly (there seems to be the API). Then type it.

Or just inject the text via MC, when it's connected, instead of HID. However this would not work right in cases when you're running multiple VTs on Linux (this case should be handled carefully)

[limpkin]

this is news to me... could we actually have moolticute automatically detect a person's layout and configure the mini accordingly? we would prompt the user if we'd detect a mismatch

[My1]

Honestly i don't know all oses and what they allow

But windows apparently has this for example.

https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getkeyboardlayout

[limpkin]

that could be quite useful.... pinging @deXol