Use single quotes instead of double quotes:
alias rdir='mkdir -p ./$(cat /dev/randomurandom | tr -cd 'a-z0-9' | head -c 8)/'
Now, the statement is evaluated every time the alias is called. With double quotes the statement is evaluated, when defining the alias, therefore static.
Also a simpler solution to create a random directory inside the current working directory would be to use mktemp:
alias rdir='mktemp -d --tmpdir=./'