If RAM usage is higher than the level written in «swapiness», kernel will use SWAP. Ok. But how does the kernel «order» programs? Is it «first come, first served» for RAM? Or biggest programs in SWAP? Most used programs in RAM?
I would like to know how the kernel decide between RAM or SWAP in the case if RAM is full or almost.
The kernel memory management doesn't use the concept of "program" to organize the memory, but pages. The kernel decides based on a 'least recently used' (this may have changed, but the latest references I could find say so), in which when the kernel under memory pressure (by swappiness, it swaps out those pages which are rarely accessed to give space to more frequently accessed or new pages (or why not, disk cache).
So, none of your assumptions are correct. The kernel doesn't order programs to the swap, it statistically select which parts of the memory will commit to swap. The method is LRU. The size of anonymous or private memory committed by the program doesn't affect the way the kernel will select which pages will be committed.
You can see how many KiloBytes (or pages) have the kernel committed to swap using smem -s swap -t -k -n.
