Unused memory is wasted, so any spare memory is considered by the kernel to be available as buffer cache. It can be discarded as soon as necessary should an application need the memory.

If your entire folder is cached then that should mean that comparison between the source files and the destination files is very efficient.

What would help is to see how you're invoking rsync, in case there are unnecessary flags - or indeed flags that should be included but aren't.

Unused memory is wasted, so any spare memory is considered by the kernel to be available as buffer cache. It can be discarded as soon as necessary should an application need the memory.

If your entire folder is cached then that should mean that comparison between the source files and the destination files is very efficient.

The rsync command itself looks fine. You're not using -H (which can be very memory hungry), and you are usefully retaining timestamps (-t implied by -a). The only other part to be aware of is that as far as rsync is concerned you are copying from one part of the local filesystem to another, so files that just need to be updated will still be copied in their entirety.