You could run a super-light window manager based on wayland. For example I would recommend rootston (from the wlroots project). For example rootston -E vlc, you will get nothing else than vlc.
rootston has no menu, tray, clock, desktop, etc. There is nothing at all. However it is more of a proof of concept, and regularly crashes. It should be OK for watching a movie, though.
Also note that for running X11 applications, rootston would launchis not based on X so it answers your question.
However for running application which are not compatible with wayland, Xwayland which is like an invisible mini-X-server running in the backgroundwill be launched as a layer of compatibility and may contain X code.
If it's only the "GUI stuff" which bothers you, you could look into Openbox.