Might as well go to the source. In this 1982 film from AT&T that features the UNIX inventors, The UNIX Operating System, Brian Kernighan pronounces /usr as "user", around the 13:41 mark. He is showing his home directory, /usr/bwk.
Until computers running UNIX got large user bases and multiple disk drives, almost all users' home directories were directly under /usr.