This is actually a function of the terminal emulator you are using (xterm, gnome-terminal, konsole, screen). It's an alternate screen, or altscreen, that gets launched when programs such as less or vim are invoked. This altscreen has no history buffer and exits immediately when you quit the program, switching back to the original screen which retains the previous window history.
You can tell less not to launch in an altscreen by passing the following the environment variable "-X". So if you are using bash, place this in ~.bashrc:
export LESS="-X"
However, this disbles the termcap initialization and deinitialization, so other views may look off.
Another option would be to use screen and set the option altscreen off in your ~.screenrc. less will not clear the screen and should preserve color formatting. Presumably tmux will have the same option.
This blog entry describes the problem and offers some different solutions specific to gnome-terminal with varying success.