If you use JSON, people won't be able to comment out bits of configuration to try different things. For me, that's a deal breaker.
It also means you can't provide a nicely commented sample configuration file for users to customize.
XML is standard and if you can provide a schema, your users will thank you.