How to check if screen keyboard is showed?

[adamfierek]

I need to get my app know if screen keyboard is showed on screen. I tried to handle events from InputPane using InputPaneInterop.GetForWindow() class but Showing event not fires (only Hiding fires). I also tried IFrameworkInputPane.Location but it returns me 0x0 even when keyboard is opened. I think it not works in Windows 11.

How to properly check it?

[DarranRowe]

How did you activate InputPane?
Using:

auto ip_af = winrt::get_activation_factory<winrt::Windows::UI::ViewManagement::InputPane>();
auto ip_af_i = ip_af.as<IInputPaneInterop>();
winrt::check_hresult(ip_af_i->GetForWindow(get_handle(), winrt::guid_of<winrt::Windows::UI::ViewManagement::IInputPane>(), winrt::put_abi(m_inputpane)));

and then hooking up the Showing and Hiding events:

m_inputpane.Showing([parent = get_handle()](const winrt::Windows::UI::ViewManagement::InputPane &, const winrt::Windows::UI::ViewManagement::InputPaneVisibilityEventArgs &) {MessageBoxW(parent, L"Input Pane Showing", L"Showing", MB_OK); });
m_inputpane.Hiding([parent = get_handle()](const winrt::Windows::UI::ViewManagement::InputPane &, const winrt::Windows::UI::ViewManagement::InputPaneVisibilityEventArgs &) {MessageBoxW(parent, L"Input Pane Hiding", L"Hiding", MB_OK); });

works perfectly fine.

While it is true that this is an unpackaged Windows API application, WinUI 3 is basically just a UI framework/library that runs in a desktop environment. I have no issues double checking this in a WinUI 3 application though.

[DarranRowe]

That is strange.

2023-09-17.14-57-44.mp4

[adamfierek]

Thanks for providing video. Your case is a bit different. You manually invoking keyboard using tray button. This way it works for me too, I checked. But I have a touch screen device and keyboard is showing when textbox gets focus. In this case, I have only "Hiding" event fired. Video attached
keyboard.webm

[DarranRowe]

That code works on a touchscreen device too. If I add a text box and select it using the touchscreen, then the touch keyboard shows and Showing event triggers without issue. The only issue is that I don't have any screen capture software on that touchscreen device right now.

However, there are two potential differences here.

1. My only touchscreen device is an ARM64 device. This means that it is easier to set up remote debugging and use the install of Visual Studio on my development system. My development system is much more powerful. What this means is that I don't have a debugger running on the same system. The fact that it is an ARM64 device shouldn't matter here.
2. As you can see from my videos, I do not have the Xaml toolbar enabled. It can cause problems with WinUI 3 from the Windows App SDK 1.4. Because of that, I just have it disabled.

So one question to potentially cover both of these. Have you tried running your application directly. You could also use Start Without Debugging, which should be on Ctrl+F5 in Visual Studio. I'm wondering if something is momentarily causing focus issues in your application.

[adamfierek]

Yes, I tried to run my app without Visual Studio. Both packaged and unpackaged. Same results - keyboard after textbox focus - no Showing event. Keyboard for tray - both events firing

[DarranRowe]

Well, I can't really say much more on this matter. I currently believe that this is more than likely a driver related issue based upon everything written here. However, that is based upon a sample size of 2. All I can say for sure is that a Surface Pro X doesn't have this issue.

[castorix]

This works for me in WinUI 3 on Windows 10 22H2 (I have put 2 different bips for show/hide) :

    private IntPtr hWndMain = IntPtr.Zero;
    private uint m_nCookie = 0;

hWndMain = WinRT.Interop.WindowNative.GetWindowHandle(this);
var fiph = new FrameworkInputPaneHandler();
var fip = (IFrameworkInputPane)new FrameworkInputPane();
if (fip != null)
{
    HRESULT hr = fip.AdviseWithHWND(hWndMain, fiph, out m_nCookie);
}

with :

   public enum HRESULT : int
   {
       S_OK = 0,
       S_FALSE = 1,
       E_NOINTERFACE = unchecked((int)0x80004002),
       E_NOTIMPL = unchecked((int)0x80004001),
       E_FAIL = unchecked((int)0x80004005)
   }

   [StructLayout(LayoutKind.Sequential)]
   public struct RECT
   {
       public int left;
       public int top;
       public int right;
       public int bottom;
       public RECT(int Left, int Top, int Right, int Bottom)
       {
           left = Left;
           top = Top;
           right = Right;
           bottom = Bottom;
       }
   }

   [ComImport]
   [Guid("226C537B-1E76-4D9E-A760-33DB29922F18")]
   [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
   public interface IFrameworkInputPaneHandler
   {
       [PreserveSig]
       HRESULT Showing(ref RECT prcInputPaneScreenLocation, bool fEnsureFocusedElementInView);
       [PreserveSig]
       HRESULT Hiding(bool fEnsureFocusedElementInView);
   }

   [ComImport]
   [Guid("5752238B-24F0-495A-82F1-2FD593056796")]
   [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
   public interface IFrameworkInputPane
   {
       [PreserveSig]
       HRESULT Advise(IntPtr pWindow, IFrameworkInputPaneHandler pHandler, out uint pdwCookie);
       [PreserveSig]
       HRESULT AdviseWithHWND(IntPtr hwnd, IFrameworkInputPaneHandler pHandler, out uint pdwCookie);
       [PreserveSig]
       HRESULT Unadvise(uint dwCookie);
       [PreserveSig]
       HRESULT Location(out RECT prcInputPaneScreenLocation);
   }

   [ComImport()]
   [Guid("D5120AA3-46BA-44C5-822D-CA8092C1FC72")]
   public partial class FrameworkInputPane
   {
   }

   public partial class FrameworkInputPaneHandler : IFrameworkInputPaneHandler
   {
       HRESULT IFrameworkInputPaneHandler.Showing(ref RECT prcInputPaneScreenLocation, bool fEnsureFocusedElementInView)
       {
           Console.Beep(1000, 10);
           return HRESULT.S_OK;
       }

       HRESULT IFrameworkInputPaneHandler.Hiding(bool fEnsureFocusedElementInView)
       {
           Console.Beep(5000, 10);
           return HRESULT.S_OK;
       }           
   }

[adamfierek]

I have the same result as with InputPaneInterop.GetForWindow() Hiding handler firing, but Showing not.

[castorix]

I have the same result as with InputPaneInterop.GetForWindow() Hiding handler firing, but Showing not.

I cannot reproduce, x86 or x64 (or it does not work at all if I test as Admin...)
On my first try, it did not work because I forgot the [PreserveSig]
I did other tests and it worked too with IImmersiveShellBroker (but in a thread as I did not find events)
or, simpler, with SetWinEventHook + OBJID_WINDOW and EVENT_OBJECT_CLOAKED/EVENT_OBJECT_UNCLOAKED events

[adamfierek]

It's the same situation as with @DarranRowe solutuon. I attached video to that thread. If I manually invoke keyboard using tray icon, it works, but if it's invoked automatically when textbox gets focus, I can only receive Hiding event.

[castorix]

It's the same situation as with @DarranRowe solutuon. I attached video to that thread. If I manually invoke keyboard using tray icon, it works, but if it's invoked automatically when textbox gets focus, I can only receive Hiding event.

Did you test with SetWinEventHook ?
Cloaked events should work in your case, like for any window

[adamfierek]

Hi @castorix , I have tested SetWinEventHook and it works. Thanks!