Convert Password authentication to SSH key authentication

I have an SFTP server running on Unbuntu 26.04. I originally built it using password authentication for the individual users. I had all the users in a group that had access to the SFTP directory only, and a few users that also had sudo access. This was working great until it was decided to convert to key authentication instead of password. I generated private keys for the users and put the public keys on the server. I have not disabled the passwords yet. This seems to be working for SFTP but when I try to login through PuTTY from my remote machine (running on Windows 11) it returns an error that states the server allows sftp access only and won't even give me access to my own account. How do it make it so users that have sudo permissions can have full access?