Here is what the bash documentation says:
PS1    The  value  of  this parameter is expanded (see PROMPTING below)
       and used as the primary prompt string.   The  default  value  is
       ``\s-\v\$ ''.
PS2    The  value of this parameter is expanded as with PS1 and used as
       the secondary prompt string.  The default is ``> ''.
PS3    The value of this parameter is used as the prompt for the select
       command (see SHELL GRAMMAR above).
PS4    The  value  of  this  parameter  is expanded as with PS1 and the
       value is printed before each command  bash  displays  during  an
       execution  trace.  The first character of PS4 is replicated mul‐
       tiple times, as necessary, to indicate multiple levels of  indi‐
       rection.  The default is ``+ ''.
So, PS1 is your normal "waiting for a command" prompt, PS2 is the
continuation prompt that you saw after typing an incomplete command,
PS3 is shown when the select command is waiting for input, and
PS4 is the debugging trace line prefix.
The documentation I quoted doesn't say so, but the default for
PS3 in bash is #?:
$ select x in foo bar baz; do echo $x; done
1) foo
2) bar
3) baz
#? 3
baz
#? 2
bar
#? ^C