Again, it would help to know which Duo integration you’re using. You haven’t stated explicitly but I assume it’s Windows Logon from your description of the Duo authentication prompt. Is that correct?
U2F authentication is only supported for Duo authentication in a browser session. You cannot use U2F authentication with Windows Logon. If you purchased Yubikey 4 or Yubikey NEO U2F authenticator tokens they could also be used as OTP tokens for Windows Logon. If the U2F tokens you purchased do not also have OTP capabilities then, unfortunately, you cannot use them with Windows Logon.
You mention needing “to cancel the login prompt to enable the button”. If you are referring to the Windows Logon prompt, you can disable autopush by unchecking the “Use auto push to authenticate if available” option in the installer, or after installation with a registry edit described here. This would eliminate the extra step of cancelling a push authentication before clicking the “Passcode” button.
I apologize if our online documentation did not clearly indicate to you that Duo’s U2F support is limited to the Chrome browser for BOTH enrollment and authentication, but this is indeed the case.
U2F authentication in thick applications is not widely supported yet. Even with online services that support U2F the official browser support is limited to Chrome (there is a third-party Mozilla plugin). The only thick client I’ve heard of with native U2F support is the Dashlane app. Are you aware of others? We’d love to hear about them! It doesn’t appear that AuthLite supports U2F either, as they list support for the Yubikey token types that include OTP, and specifically say that the Yubikey FIDO U2F-only token isn’t supported.
I hope this additional information helps you find a solution that meets your use case.