Fatal: Internal error: PAM auth succeeded when it should have failed

Hi there,

I am running a new server with Ubuntu 20.04 LTS
I installed duo security via apt-get:

cat /etc/apt/sources.list.d/duosecurity.list

deb [arch=amd64] https://pkg.duosecurity.com/Ubuntu focal main

I followed these instructions from here: Duo Unix - 2FA for SSH with PAM Support (pam_duo) | Duo Security

Normal users can log in fine. I have a bypass use that uses an RSA key and I am getting the following error in /var/log/auth.log: fatal: Internal error: PAM auth succeeded when it should have failed

What do I need to check to understand what is going on?

Cheers,

Noah