By default, watch runs your command with /bin/sh -c '...' so the output you see is how /bin/sh interprets the time command. Your /bin/sh apparently doesn't have a builtin time.
To run the command with a different shell, use the -x option to get rid of the default, then add your own explicit invocation of the shell whose builtin you want.
watch -x bash -c 'time ls'
watch -x zsh -c 'time ls'
No matter how you run watch, the command you're watching is not a child of the shell that ran the watch command, so that shell's settings aren't directly relevant.