WINDOWS HOOKS ARE WEIRD

Source: Dev.to

Hook program

LRESULT CALLBACK KeyboardProc(int nCode, WPARAM wParam, LPARAM lParam)
{
    if (nCode == HC_ACTION && wParam == WM_KEYDOWN)
    {
        KBDLLHOOKSTRUCT *kb = (KBDLLHOOKSTRUCT*)lParam;

        unsigned char keyboard_state[256];
        wchar_t      unicodeData[5] = {0};

        GetKeyboardState(keyboard_state);

        int unicodeKey = ToUnicodeEx(kb->vkCode,
                                    kb->scanCode,
                                    keyboard_state,
                                    unicodeData,
                                    sizeof(unicodeData) / sizeof(wchar_t),
                                    0,
                                    NULL);

        if (unicodeKey > 0)
        {
            printf("%ls", unicodeData);
        }

        if (kb->vkCode == VK_RETURN)
        {
            printf("\n");
        }
    }
    return CallNextHookEx(hook, nCode, wParam, lParam);
}

int main()
{
    hook = SetWindowsHookEx(WH_KEYBOARD_LL, KeyboardProc, NULL, 0);
    MSG msg;
    while (GetMessage(&msg, NULL, 0, 0))
    {
        /* message loop intentionally empty */
    }
}

Not too shabby, I’d say.

---

The Problem

While having a bit of fun, I found that Caps Lock or Shift didn’t register in the printed key logs. I was baffled and did what any self‑respectable programmer would do – Google. I came across this Stack Overflow question:

A comment suggested the following fix:

short caps_held  = GetKeyState(VK_CAPITAL) & 0x0001;
short shift_held = GetKeyState(VK_SHIFT)   & 0x8000;

if (caps_held)  keyboard_state[VK_CAPITAL] |= 0x01;
else           keyboard_state[VK_CAPITAL] &= ~0x01;

if (shift_held) keyboard_state[VK_SHIFT]   |= 0x80;
else           keyboard_state[VK_SHIFT]   &= ~0x80;

Note: I know I was supposed to use GetAsyncKeyState, but being the rebel I am, I used GetKeyState because it worked, so why not? Keep this in mind for later.

The caps and shift keys were fixed, but a new problem appeared: Ctrl.
When I pressed Ctrl + any key, I got weird characters. With Ctrl + C in particular I got ♥. This didn’t happen before, so I investigated.

Ctrl + C in keyboard_state gave a value of 3 in unicodeData[0], which is a control character that gets rendered as ♥. I tried clearing the high‑order bit:

keyboard_state[VK_CONTROL] &= ~0x80;

Problem solved… or is it?

---

Why does GetKeyboardState work at all?

GetKeyboardState() doesn’t work in an app that doesn’t pump its message loop, so what on earth is going on here? The answer? GetKeyState.

For whatever reason, if you call GetKeyState inside the hook callback, it returns the previous state of the key – i.e., the state before your function is called. In doing so it does something that causes GetKeyboardState to actually update.

I found that simply calling GetKeyState(0) removed the need for the caps‑and‑shift conditionals shown above. GetAsyncKeyState(0) doesn’t have this side‑effect, however. Is this intentional? I wondered.

---

Raymond Chen on GetAsyncKeyState / GetKeyState

While reading, I came across an article by Raymond Chen (the legend himself) from 2004:

Excerpt

[excerpt omitted for brevity]

However, I have some issues with this explanation.

1. When he says the function returns keys “based on the messages you have retrieved from your input queue”, what input queue exactly?

2. The MSDN documentation says:

   “The key status returned from this function changes as a thread reads key messages from its message queue.”

I therefore assume the input queue and the message queue are the same, but this doesn’t make sense because GetKeyboardState gets values based on the message‑queue state, and in my low‑level hook this message queue isn’t pumped. How can this be? If the queues are the same, why didn’t GetKeyboardState work from the get‑go? Are these queues different or the same? The documentation is unclear.

My speculation: GetKeyboardState and GetKeyState read from different queues or states, and GetKeyState somehow synchronises whatever state GetKeyboardState reads from.

---

Simple test program

I decided to test GetKeyState myself:

int main()
{
    while (true)
    {
        short ctrlPressed = GetKeyState('A');
        printf("%d\n", ctrlPressed & 0x8000);
    }
}

When I press A, it prints 32768 even though this program doesn’t appear to interact with a message queue at all. If we take Chen’s and MSDN’s words literally, this implies that every Windows app has a message queue (which could be true), but where is GetKeyState reading from?

The MSDN entry for GetKeyState says:

“The key status returned from this function changes as a thread reads key messages from its message queue.”

My simple program never reads from its message queue, nor does my hook program (as far as I’m aware), so something’s not right.

---

GetKeyboardState documentation (MSDN)

“An application can call this function to retrieve the current status of all the virtual keys. The status changes as a thread removes keyboard messages from its message queue. The status does not change as keyboard messages are posted to the thread’s message queue, nor does it change as keyboard messages are posted to or retrieved from message queues of other threads.”

If we take MSDN’s words at face value, then logically this means that GetKeyState removes keyboard messages and processes a queue before getting the key, which would explain why my simple program worked. This is a loaded assumption, and I don’t believe it’s the case.

---

Summary of open questions

1. What queue does GetKeyState read from?
   • Is it the thread’s own message queue, a system‑wide queue, or something else?
2. Why does calling GetKeyState inside a low‑level hook appear to “prime” GetKeyboardState?
   • Is there an undocumented side‑effect that forces the keyboard state to be refreshed?
3. Are GetKeyState and GetKeyboardState looking at different internal representations of the keyboard?
   • If so, how are they kept in sync?
4. What does Raymond Chen’s “input queue” refer to, and how does it relate to the per‑thread message queue?

These are the points I’m still trying to reconcile. Any clarification (or references to deeper documentation) would be greatly appreciated.

---

The only way I can explain this inconsistency is that either Chen’s and MSDN’s posts are oversimplified, outdated, or this is undocumented functionality. Considering Chen’s post was written 20+ years ago, it could be all three, but I honestly don’t know.

After the rabbit hole I fell into, I can say for sure that GetKeyState(0) refreshes whatever GetKeyboardState is reading from, although the why is uncertain. This worked on the version of Windows 10 and 11 I have, but I haven’t tested it on Windows 7. I have no idea if this will ever change later, so BEWARE.

Of course, you could also try to create the state of the key with the hook itself:

else if (nCode == HC_ACTION && wParam == WM_KEYUP) {
    KBDLLHOOKSTRUCT *kb = (KBDLLHOOKSTRUCT*)lParam;

    if (kb->vkCode == VK_LSHIFT || kb->vkCode == VK_RSHIFT) {
        shiftHeld = true;
        printf("%s\n", "Shift released");
    }
}

This is valid too and is more fine‑tuned.

I hope this article helps anyone who uses these hooks. If I’m wrong about anything, or if you understand why this happens, please leave a comment or reach out to me. I’m always available and I do want to understand what’s going on.

Still, next time…