Your fastest route for troubleshooting this is to work with the Duo Support team, so I strongly recommend you contact them. However, I’ll try to help you if I can! I am not familiar with using this plugin myself, so I am just going based off of our KB and docs here.
Do you have a multisite WordPress installation? This article on why Duo unexpectedly prompts some user roles to authenticate to the primary WordPress site sounds like it could be the same issue you’re describing.
When you configured the plugin, for which roles did you enable two-factor authentication?
You mention this happens sometimes, but not all. Are there any particular circumstances you’ve noticed under which you do or don’t see the Duo Prompt?
You might also find the Duo for WordPress FAQ helpful for more info and troubleshooting tips.