To be sure the "passphrase agent" (gpg-agent) is enabled, check your ~/.gnupg/gpg.conf and eventually uncomment the line containing "use-agent".
In KMail'sSettings -> Configure KMail -> Security tab -> Crypto Backends, select OpenGPG, then Configure. On the GPG Agent page, give a path to which logs can be written, in case of problems. You can check the log in KMail'sTools -> GnuPG Log Viewer. It's probably a good idea, too, to increase the cache time - I use 3600.