Every time I install Arch Linux I am surprised when XFCE insists on running gpg-agent when I log on. When it further doesn't work properly with ssh-keys (admittedly, probably because I don't know how to use it properly) and invokes pinentry to input another passphrase so Gnome Keyring can protect the key the agent is supposed to protect... well... that's when I start to get irritated. Irritation leads to anger, and anger to research, and research to a solution.
Luckily the solution is easy to find every time, but this time I'm making it easier to find. This makes XFCE use ssh-agent even if gpg-agent is installed.
$ xfconf-query -c xfce4-session -p /startup/ssh-agent/type -n -t string -s ssh-agent
The solution was (shockingly) located in the Arch Linux wiki.