I tried a few of these answers, but ssh -v kept showing my public keys getting pulled out of my home directory. However, specifying a bogus identity file did the trick for me:
ssh -i /dev/null host
I have to do this permanently (to work around the broken SSH server in an APC rack-mounted PDU — stay far away from these things if you care about security) so I ended up putting the option into my config file:
Host apc1 apc2
KexAlgorithms +diffie-hellman-group1-sha1
IdentityFile /dev/null