Restoring a backup with Duplicity

See the last post for how I back up my important files to a remote server. To restore them I do this:

duplicity --file-to-restore path/to/file.txt
    scp://user@example.com//home/example/bkp/home/andy/Maildir file.txt

and enter the passphrase when prompted.

Note that I can restore this backup even if I lose my entire home hard drive because it doesn't rely on any key file - just my passphrase. This is less secure than using a private key file, but where do you back up your key file to? You know it will be lost when you need it ... safer just to write down your passphrase somewhere, I reckon.