Duo auth proxy not working for user unless 'bypass' is configured

I have a couple Duo Auth proxies setup to proxy to our our on-prem OpenLDAP servers. Our on-prem servers talk to an off-site AD host for the password authentication (via SASL). This setup allows us to host POSIX attributes for all of our Linux logins as well as centralize all of the security groups and user accounts and supply LDAP authentication for all of our web applications as well.

The auth proxies have been working well for about a year or so.

Over the weekend, I thought I’d try using the auth proxies as the source for SSH authentication. This was simple enough: change to PasswordAuthentication yes in sshd_config and then replace the on-prem ldap servers in /etc/ldap.conf with the IPs of the Duo auth proxies.

In testing, this worked fine for me. My Duo token is on my phone. Another user I was working with today is using a tablet for the token. They are not able to login unless we bypass them in the Duo admin panel. I was able to send them a test push from the admin panel and it worked correctly. Anyone know what’s happening here or where to look for more info? The only clues I have so far are in the Linux auth.log (below)

# duo enabled (password fails with some weird ldap timeout)
Apr 13 15:46:01 myhost sshd[577]: Connection from 45.xxx.xxx.xxx port 52455 on 10.xxx.xxx.xxx port 60022
Apr 13 15:46:02 myhost sshd[577]: Failed publickey for user888 from 45.xxx.xxx.xxx port 52455 ssh2: RSA SHA256:QERZKwp39j/VO+WBqVfLhn1G9wrAxSHHShbO3fzzhsc
Apr 13 15:46:08 myhost sshd[577]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=45.xxx.xxx.xxx  user=user888
Apr 13 15:46:19 myhost sshd[577]: pam_ldap: ldap_result Timed out
Apr 13 15:46:34 myhost sshd[577]: pam_ldap: ldap_result Timed out
Apr 13 15:46:36 myhost sshd[577]: Failed password for user888 from 45.xxx.xxx.xxx port 52455 ssh2


# bypass duo auth and password succeeds
Apr 13 15:58:52 myhost sshd[1749]: Connection from 45.xxx.xxx.xxx port 52862 on 10.xxx.xxx.xxx port 60022
Apr 13 15:58:54 myhost sshd[1749]: Failed publickey for user888 from 45.xxx.xxx.xxx port 52862 ssh2: RSA SHA256:QERZKwp39j/VO+WBqVfLhn1G9wrAxSHHShbO3fzzhsc
Apr 13 15:59:00 myhost sshd[1749]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=45.xxx.xxx.xxx  user=user888
Apr 13 15:59:00 myhost sshd[1749]: pam_ldap: ldap_result Can't contact LDAP server
Apr 13 15:59:00 myhost sshd[1749]: pam_ldap: reconnecting to LDAP server...
Apr 13 15:59:01 myhost sshd[1749]: Accepted password for user888 from 45.xxx.xxx.xxx port 52862 ssh2
Apr 13 15:59:01 myhost sshd[1749]: pam_unix(sshd:session): session opened for user user888 by (uid=0)
Apr 13 15:59:01 myhost sshd[1749]: User child is on pid 1766

Is there any relevant information in the proxy debug log? Is the proxy returning allow or not?

Thanks - This is related to message: Duo Auth Proxy appears to be failing all logins after 10 seconds. Why?

When I found out there was a 10 second timeout after entering the password and the authproxy was logging Received extraneous LDAP PDU while resolving a BindRequest I re-posted with a different question.