This seems to fit your requirements
rsync -avR user@remote:'/backups/*/latest/' /some/local/path/
The -R flag effectively "copies" the entire source path into the destination and the resulting backups will land as /some/local/path/backups/{whatever}/latest.
If you find you don't want the entire source path you can include /./ to indicate the point from which the path should be copied across to the destination. For example, /backups/./foo/latest/ would result in /some/local/path/foo/latest/ (i.e. the /backups component has been skipped). There's lots more detail in the documentation (man rsync).