The behavior of --remove-source-files is exactly that specified by man rsync:
--remove-source-files
This tells rsync to remove from the sending side the files (meaning non-directories) that are a part of the transfer and have been successfully duplicated on the receiving side.
So they are removed after all files are transferred successfully. If the transfer is interrupted or another error occurs, the source files are not removed, even if some of them transferred OK.