Double prompt on RDS Server


so… i have a test rds server with web and gateway duo installed works perfectly…
but i also installed the windows agent as well… just to protect any “local” login attempts…

but as you probably guessed, when a user logs in via the rdp client, they get prompted then when they hit the rds server… they get hit again with duo authentication.

am i being too paranoid?, do i need the windows agent, or… I haven’t tried yet… but if i remove the windows agent and try to login to the console directly with one of the protected accounts that has access to the rds server will they get prompted even though they didn’t arrive at the server via the remote desktop app?