Use the awesome xfce-panel-profiles (https://docs.xfce.org/apps/xfce4-panel-profiles/start), previously named xfce-panel-switch.
For Ubuntu install
sudo apt install -yqq xfce4-panel-profiles`profiles
Click on Settings -> Panel Profiles. On that screen save double monitor, disable second panel, and then backup again a single monitor setup. When connecting the second monitor apply "double monitor" again.