To get the zsh shell to perform command substitution on the value of PS1, set the PROMPT_SUBST shell option:
set -o PROMPT_SUBST
PS1='$(pwd) > '
Note that there is no need to export the PS1 variable.
The reason why the PROMPT_SUBST shell option is not set by default is likely that the zsh shell already has rich syntax for prompt expansions.
Your example could, for instance, be replaced by
PS1='%/ > '
For more information about this, see the "EXPANSION OF PROMPT SEQUENCES" section in the zshmisc manual on your system.
When you use
PS1="$(pwd) > "
the value given to the PS1 variable is expanded before the assignment takes place. This is why it does not change when you change directories.