SERVFAIL is quite clear: instead of telling your client "no sorry, no such record", the DNS resolver asked replies with "sorry, I'm broken".
So, you need to fix that DNS resolver, not the client you're using.
The --ipv4 flag helping suggests it might be your name server's is broken when asked via IPv6, for whatever reason. Maybe check that.