I've found that the -delete action does work nicely with the -path test. For instance, the following ought to work on the original posters problem:
find . -path '*/.svn*' -delete
Note that in addition to deleting '.svn' directories (and their contents), this will also delete any files or directories whose names start with '.svn'. For example, if you used -path '*/.git*' it would also delete '.gitignore' and '.gitattribute' in addition to '.git/'. To avoid that, and just delete directories with that exact name use:
find . -path '*/.svn/*' -or -name '.svn' -delete
Note the slash after .svn. This will first find, and delete, all the files under '.svn', then delete the .svn directory itself.