Yes, you can if you have OpenSSH For Windows Server 2019 and Windows 10 installed.
From OpenSSH Server Configuration for Windows 10 1809 and Server 2019:
Configuring the default ssh shell is done in the Windows registry by adding the full path to the shell executable to Computer\HKEY_LOCAL_MACHINE\SOFTWARE\OpenSSH in the string value DefaultShell.
As an example, the following Powershell command sets the default shell to be PowerShell.exe:
New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -PropertyType String -Force
To set Git Bash as default shell, run this snippet as administrator in the PowerShell:
New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\Program Files\Git\bin\sh.exe" -PropertyType String -Force
To get a login shell, you could create a batch file sh.bat in C:\Program Files\Git\bin with this content:
@echo off
"C:\Program Files\Git\bin\sh.exe" --login
and replace the DefaultShell value with C:\Program Files\Git\bin\sh.bat.
The DefaultShell registry key is a new key, you can delete it to restore the default command prompt.