UPDATE:
Yesterday I was thinking about this question for some reason, and came across a different approach which might work.
The idea basically consists in ditching the desktop/login managers instead in authenticating from them, and starting X by yourself remotely.
Say your SSH server service is enabled in your desktop. From a remote machine (your phone), stop the desktop manager (SDDM, for example):
# systemctl stop sddm
Now let us suppose you have some other X initialization package installed, like xinit, where a command like startx would suffice. If you were running from a console,
$ startx
would start X according to your configuration files without any problem. I don't know about every Unix-like system, but, from my experience, startx will look after $HOME/.xinitrc or, if the file is nonexistent, /etc/X11/xinitrc, and the X server will start as expected.
Unfortunately, we're not in console, and when trying to run from another shell like the terminal emulator where we're sending commands via SSH, simply sending
$ startx
retrieves
/usr/lib/Xorg.wrap: Only console users are allowed to run the X server
On the other hand, root can initialize the X server remotely from secure shell via a terminal emulator. For instance,
# startx
initializes the X server as expected, following the configuration files from root's home (or /etc/X11/xinitrc, as aforementioned). This makes us think it is about permissions, and not inability to do so.
After a couple minutes of searching, I've stumbled across Error when trying to use Xorg: Only console users are allowed to run the X server?, SSH login shows Only console users are allowed to run the X server, both of which boil down to the same path, edit (or add and edit, if nonexistent) /etc/X11/Xwrapper.config to include the following lines
allowed_users=anybody
needs_root_rights=yes
which indeed allows the startx command to be run by a regular user from the terminal emulator.
A note:
Please verify if your Xresources file is loading accordingly. Remember to correctly add to xinitrc your desktop environment, window manager, xrdb command, and so on.
Relevant:
I am also considering another approach inspired in How to remotely log in with full graphical desktop over X11 but need to think about it a little more. It may serve as inspiration for further solutions and possibly a better or more suitable option.