Bypass events and get the physical keyboard state directly

[Bryan Baldwin]

Hello all,

I've been writing game code, and have gone adrift with a big problem
with keyboard input.

The properties I'm getting through SDL2 contain key up/down,
pressed/released, and repeat. I've read either that the pressed/released
properties are only set with the key actually physically down, or that a
keyboard event will at least have the repeat property set if the key is
a repeat. I can't confirm that any of this is true in code.

No matter what I do, if I hold down a key I get the first key down +
pressed event, a subsequent event flagged as a repeat at the delay
threshold (500ms), and then a deluge of *unflagged* key down+pressed /
key up+released event pairs for the rate interval that continue
repeating well after the key is actually released. I tried using SDL2's
GetKeyboardState instead, but this produces identical results. Its the
same data as events with a different interface.

The documentation I've read for keyboard input with X11 also follows a
very similar event-based model, but what I want is a way to check the
keyboard and literally only see exactly what the state of the keys at
the instant that that call is made, and by-pass any and all other
processing entirely, including auto-repeat.

Can anyone point me in the right direction?

[Bryan Baldwin]

Okay, so further investigation has lead me to test code against evdev
directly. Nevermind ;)

Post by Bryan Baldwin
Hello all,
I've been writing game code, and have gone adrift with a big problem
with keyboard input.
The properties I'm getting through SDL2 contain key up/down,
pressed/released, and repeat. I've read either that the pressed/released
properties are only set with the key actually physically down, or that a
keyboard event will at least have the repeat property set if the key is
a repeat. I can't confirm that any of this is true in code.
No matter what I do, if I hold down a key I get the first key down +
pressed event, a subsequent event flagged as a repeat at the delay
threshold (500ms), and then a deluge of *unflagged* key down+pressed /
key up+released event pairs for the rate interval that continue
repeating well after the key is actually released. I tried using SDL2's
GetKeyboardState instead, but this produces identical results. Its the
same data as events with a different interface.
The documentation I've read for keyboard input with X11 also follows a
very similar event-based model, but what I want is a way to check the
keyboard and literally only see exactly what the state of the keys at
the instant that that call is made, and by-pass any and all other
processing entirely, including auto-repeat.
Can anyone point me in the right direction?
_______________________________________________
xdg mailing list
https://lists.freedesktop.org/mailman/listinfo/xdg

[Pekka Paalanen]

On Mon, 20 Feb 2017 12:37:36 +1300

Post by Bryan Baldwin
Okay, so further investigation has lead me to test code against evdev
directly. Nevermind ;)

But if you run under any kind of display server (windowing system),
that won't usually work at all, or works wrong.

That is also why any kind of "bypass the display server" will not
generally work. There is a myriad of reasons for that, including
permission and security issues, even starting from how to even pick the
right devices.

You really are expected to keep the keyboard state tracked in your app,
based on the events. Even evdev works like that. If you were writing
for Wayland, there would be libxkbcommon to do that, and I believe SDL2
already uses libxkbcommon anyway (on Wayland).

OTOH, if you were not running under any display server, then you'd be
fine with that approach.

Your question would be better directed at SDL or the specific window
system fora (e.g. mailing lists).


Thanks,
pq

[Bryan Baldwin]

_______________________________________________
xdg mailing list
***@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/xdg

[Pekka Paalanen]

(I think you just sent an encrypted email to a mailing list. I assume
this was an accident, since nothing indicates otherwise.)


On Mon, 20 Feb 2017 21:56:03 +1300

I don't know why you mentioned Wayland. As a reference point? I'm not
presently developing with a Wayland target.

Yes, as a reference point.

I cannot track the state of the keyboard with the events I received,
because, as described, they are garbage. Something between evdev and
my code screws the input data. X11, GNOME, or SDL2. Out of that
stack, X11 would have been my goto to look for a way to view input
more directly, which is why I asked about your interfaces here, but
even that was completely wrong.

There is probably a reason why something somewhere converts repeats
into up/down pairs, if that is the only problem you have. You could try
finding out which component is responsible for it, and ask on the
appropriate mailing list why it is so and how to work around it.

I would also hazard a guess that these up/down pairs come really close
to each other, so if you actually drained the event queue before
looking at the tracked keyboard state, maybe it would be what you need.
But that is just speculation from me.

FWIW, Wayland does not have repeat events itself. A toolkit may
manufacture those any way it wants based on keyboard state (the exact
state you seem to be wanting in the first place).

Initial code tests I've made directly against evdev prove that the
input seen is accurate and properly reported. Grabbing all the input
devices is easy. Releasing and reacquiring all of them based on
window focus is easy. Identifying what each device is and which to
use is easy. I have no idea what you are talking about with Re: to
permissions and security. I'm not sure where in the original software
stack the input code is being ruined, but to whomever code that
belongs, if you cannot deliver accurate input from the kernel with
100% confidence, you cannot be trusted to decide permissions or
security, either.

I'm not convinced, but I won't argue about the easyness.

The security/permissions problem is this: if you are allowed to open
the input devices, you can also trivially implement an invisible
keylogger, just "forget" to close the devices. Obviously people don't
like that idea, hence in a usual system the input device permissions
are restricted, probably as far as to the root user. Hence your game
needs to run as root, or ask the user to bypass the set permissions
e.g. by adding himself to the 'input' group (which now opens the door
for keyloggers). Display servers often use logind DBus API to open input
devices to avoid running as root, but I believe logind would refuse
your game if a display server was also active at the same time.


Thanks,
pq

Post by Pekka Paalanen
On Mon, 20 Feb 2017 12:37:36 +1300

Post by Bryan Baldwin
Okay, so further investigation has lead me to test code against
evdev directly. Nevermind ;)

But if you run under any kind of display server (windowing system),
that won't usually work at all, or works wrong.
That is also why any kind of "bypass the display server" will not
generally work. There is a myriad of reasons for that, including
permission and security issues, even starting from how to even pick
the right devices.
You really are expected to keep the keyboard state tracked in your
app, based on the events. Even evdev works like that. If you were
writing for Wayland, there would be libxkbcommon to do that, and I
believe SDL2 already uses libxkbcommon anyway (on Wayland).
OTOH, if you were not running under any display server, then you'd
be fine with that approach.
Your question would be better directed at SDL or the specific window
system fora (e.g. mailing lists).
Thanks,
pq

[Bryan Baldwin]

Yup, the encryption was an automagickal client mistake.

And I understand all your points, and don't disagree with the security model. I just don't think that the people and the software that are assuming responsibility for system security should be. I'm not afraid to go and look, then tear it up and write my own thing if its a good use of time. I think its going to be a very good use of time these days ;)

For posterity, the input problem I had is a bug in SDL2. Pressing and holding down a key was not producing events with the repeat flag set. It produced pairs of keydown & pressed - keyup & released events without the repeat flag set. These events continued to be send long after the key was physically released. It is an untenable expectation for the application to track key state with garbage input.

This is a known SDL2 bug affecting at least verisons 2.0.4
https://bugzilla.libsdl.org/show_bug.cgi?id=3472 and 2.0.5.

There is a patch I've tested locally on my own system, and it works.
https://bugzilla-attachments.libsdl.org/attachment.cgi?id=2594

Post by Pekka Paalanen
(I think you just sent an encrypted email to a mailing list. I assume
this was an accident, since nothing indicates otherwise.)
On Mon, 20 Feb 2017 21:56:03 +1300

I don't know why you mentioned Wayland. As a reference point? I'm not
presently developing with a Wayland target.

Yes, as a reference point.

I cannot track the state of the keyboard with the events I received,
because, as described, they are garbage. Something between evdev and
my code screws the input data. X11, GNOME, or SDL2. Out of that
stack, X11 would have been my goto to look for a way to view input
more directly, which is why I asked about your interfaces here, but
even that was completely wrong.

There is probably a reason why something somewhere converts repeats
into up/down pairs, if that is the only problem you have. You could try
finding out which component is responsible for it, and ask on the
appropriate mailing list why it is so and how to work around it.
I would also hazard a guess that these up/down pairs come really close
to each other, so if you actually drained the event queue before
looking at the tracked keyboard state, maybe it would be what you need.
But that is just speculation from me.
FWIW, Wayland does not have repeat events itself. A toolkit may
manufacture those any way it wants based on keyboard state (the exact
state you seem to be wanting in the first place).

Initial code tests I've made directly against evdev prove that the
input seen is accurate and properly reported. Grabbing all the input
devices is easy. Releasing and reacquiring all of them based on
window focus is easy. Identifying what each device is and which to
use is easy. I have no idea what you are talking about with Re: to
permissions and security. I'm not sure where in the original software
stack the input code is being ruined, but to whomever code that
belongs, if you cannot deliver accurate input from the kernel with
100% confidence, you cannot be trusted to decide permissions or
security, either.

I'm not convinced, but I won't argue about the easyness.
The security/permissions problem is this: if you are allowed to open
the input devices, you can also trivially implement an invisible
keylogger, just "forget" to close the devices. Obviously people don't
like that idea, hence in a usual system the input device permissions
are restricted, probably as far as to the root user. Hence your game
needs to run as root, or ask the user to bypass the set permissions
e.g. by adding himself to the 'input' group (which now opens the door
for keyloggers). Display servers often use logind DBus API to open input
devices to avoid running as root, but I believe logind would refuse
your game if a display server was also active at the same time.
Thanks,
pq

Post by Pekka Paalanen
On Mon, 20 Feb 2017 12:37:36 +1300

Post by Bryan Baldwin
Okay, so further investigation has lead me to test code against
evdev directly. Nevermind ;)

But if you run under any kind of display server (windowing system),
that won't usually work at all, or works wrong.
That is also why any kind of "bypass the display server" will not
generally work. There is a myriad of reasons for that, including
permission and security issues, even starting from how to even pick
the right devices.
You really are expected to keep the keyboard state tracked in your
app, based on the events. Even evdev works like that. If you were
writing for Wayland, there would be libxkbcommon to do that, and I
believe SDL2 already uses libxkbcommon anyway (on Wayland).
OTOH, if you were not running under any display server, then you'd
be fine with that approach.
Your question would be better directed at SDL or the specific window
system fora (e.g. mailing lists).
Thanks,
pq

--

[Pekka Paalanen]

On Tue, 21 Feb 2017 12:19:27 +1300

Post by Bryan Baldwin
Yup, the encryption was an automagickal client mistake.
And I understand all your points, and don't disagree with the
security model. I just don't think that the people and the software
that are assuming responsibility for system security should be. I'm
not afraid to go and look, then tear it up and write my own thing if
its a good use of time. I think its going to be a very good use of
time these days ;)
For posterity, the input problem I had is a bug in SDL2. Pressing and
holding down a key was not producing events with the repeat flag set.
It produced pairs of keydown & pressed - keyup & released events
without the repeat flag set. These events continued to be send long
after the key was physically released. It is an untenable expectation
for the application to track key state with garbage input.
This is a known SDL2 bug affecting at least verisons 2.0.4
https://bugzilla.libsdl.org/show_bug.cgi?id=3472 and 2.0.5.
There is a patch I've tested locally on my own system, and it works.
https://bugzilla-attachments.libsdl.org/attachment.cgi?id=2594

Hi Bryan,

I am glad you found the culprit and told us what it is. I would have
never guessed it to be an SDL bug caused by such subtle interactions.

Happy hacking!


Thanks,
pq

[Bryan Baldwin]

Post by Pekka Paalanen
I am glad you found the culprit and told us what it is. I would have
never guessed it to be an SDL bug caused by such subtle interactions.

Its weirder than that. The input bug hasn't any obvious effect on the video animation until after you add timed sections to your game loop. Specifically, sleeping the game process after processing the current frame until the beginning of the next frame.

Once I started return CPU cycles back to the system, I got slammed with ghosted events that would continue moving the screen around long after my hands had left the keyboard. I'm also doing Win32 platform code with a similar game loop design in another layer, and that was working fine with identical game code.

The entire time, I couldn't believe that was the quality of keyboard input on Linux. Sheesh.

https://bugs.gentoo.org/show_bug.cgi?id=610326

--