Futzing around with XDM

A buddy of mine asked me a weird question the other day: how do I get a remote GUI on a Linux server? As it happens, I knew at least the starting point to this query. It involves using a bunch of stuff that’s built into X Window that’s been there pretty much forever.

Some background

Most people who are familiar with your average Linux distribution are familiar with at least one of the desktop environments that they come with: GNOME, KDE, Xfce, Cinnamon, etc. These provide the UI and a bunch of utilities and other miscellaneous things to allow you to use your machine. Different distributions use different desktop environments, and some distros offer some choice in the desktop enviroment you end up with. (For example, Manjaro Linux has three - you can download a version that uses KDE Plasma, one that uses GNOME, or one that uses Xfce.) But, the desktop environment is only half the equation. All the desktop environments run on top of a separate graphical layer - generally the X Window System, or just X. (Xorg, specifically, which is the current implementation of it.)

X itself is really a protocol - your computer runs an X server, which talks to your keyboard, mouse, and screen; and the programs you run are X clients and talk to the server to display output and capture input. Most of the time, this is all done on your local system, but there’s no requirement for that. X has supported, from its beginnings in the 80s, the ability for the server and clients to be on separate computers and talk over a network. This gives you a lot of flexibility. X in addition has a concept called the Display Manager, which allows you to request a session and log into a machine via a greeter program. This is actually how you log into your average Linux system - there’s one of the various DMs running and it displays your login prompt and gives you some options, and then actually logs you in and starts your session. Display managers also work over the network, so you can have one (or several) large machines that do all the compute and a bunch of smaller machines that the users actually use.

Back in the day, there were actually hardware devices known as X terminals that contained your standard input/output devices and enough compute and network capability to handle incoming X connections. Sun Microsystems made a couple, and there were some NCD machines as well. Nobody really does this with X anymore, though, oddly enough, this general idea and setup persists in the Windows world.

So, in conclusion, X - the actual thing that makes your GUI work on a Linux box - can push all your pixels over the network to a separate machine. It’s been built into the system since it came out, essentially, and literally everything supports it. (To some extent - some things go around the X server and more-or-less write directly to the display hardware. That stuff’s not using X the protocol, though.)

So, the premise

The ask was to be able to bring a more-or-less normal Linux desktop over the network from the server to a local system. Basically, Windows’ Remote Desktop functionality. It turns out this is relatively not terrible to set up, though it is somewhat.. Unixy to use. As I mentioned, the Display Manager handles setting up logging in graphically and setting up the session, and that additionally can work over the network and supports multiple sessions, so that’s where you start. Then, you need to be able to get something to request the session over the network. Finally, you have to figure out how to do at least a bit of security for all this. (Remember how I mentioned that X did this in the 1980s? That wasn’t a joke - X originally came out in 1984 and this functionality was in place then - but the world was very very different at that point, and so it does a lot of things over unencrypted communication channels, which is just really super not good today.)

My thought was to do a couple things:

1. Set up the DM on the server machine to accept “remote” connections. (They won’t really be remote connections but by default DMs expect to see just one set of physical input devices attached directly to the machine.)
2. Set up a nesting X server on the server machine that’ll request a new session from the DM, and, through the magic of just basically being a proxy for X, display that over the network.
3. Get an X server running on the local machine that can be used. (Surprise! WSL 2 has this built in.)
4. Finally, use SSH to log into the remote system, run the nested X server, and behold! a graphical desktop enviromment from the remote system on your computer.

Setting up the DM is really pretty easy. Most modern Linux systems use either lightdm or gdm or sddm so you really just have to configure one of those. For lightdm and gdm, it’s just a matter of a few configuration lines added to a file and restarting the service. I found this page in the Arch Linux wiki that has some cut and pasteables. (This is nice because the documentation for lightdm specifically is “read the sample config file”, which is terrible. Do better, folks.) Note that you choose which DM you’re using in step 1 and then do the steps for that one - you don’t have to configure all three options provided, and you shouldn’t because you can only run one DM at a time. I chose to stick with the lightdm one for my initial testing.

Setting up the nesting X server is also pretty easy. There’s two: Xnest and Xephyr. You just find those (search for them in your package manager, probably using their names all in lowercase) and install it. No further configuration. What these do is provide an X server that outputs to another X server - any X clients you point at it basically just get thrown in a buffer that gets displayed elsewhere. Because they’re real X servers, they support display managers and can request a session from one. In my experience, Xephyr works better, especially over a slower connection.

You’ll need something for the nesting X server to talk to, and that is an X server on your local machine. On macOS, you use Xquartz for this. Windows is a bit different. There are some X servers for native Windows, but WSL 2 gained support for running graphical Linux apps in August 2022 - that’s just X so it’s got an X server built in, and you don’t have to do anything. (And, let’s face it, you’re about knee-deep in Unixy stuff at this point so why not just go whole hog and use WSL if you’re using Windows at this point?)

Finally, you need to be able to access the remote system. You just use SSH for this. You may need to enable X11Forwarding on the server side, but you should be able to just use the -X flag when you connect to your machine to automatically forward X11 stuff to your machine. You can test this by SSHing into your remote machine with the -X flag and running something simple (like xeyes, which will put a set of eyes on your desktop that will follow your mouse cursor.)

Fun fact: you can run any X program like this! So, if you don’t want to screw around with the whole desktop environment thing, you can literally just ssh -X into something and run whatever and it should appear alongside your other windows. Like this:

xeyes running on my Ryzen 2700, but eyeballing me from the MacBook Air

Actually running things

To actually get a desktop, I ran:

Xephyr -query 127.0.0.1 -br -ac -noreset -screen 1280x720 :1

This runs the Xephyr server, tells it to ask for a session from localhost, and set the screen size to 1280x720 (and make this display :1 - it’s important that you don’t reuse these, in that it won’t let you). Once I ran that, it popped a window on my desktop and gave me the normal login screen. Neat!

Other options

It turns out too that lightdm (and probably others, not sure) will just go ahead and set up VNC for you. There’s some instructions here - if this is set up, you can use a standard VNC server to connect. (macOS has one built in since that’s how its own screen sharing stuff works, and this isn’t hard to come by everywhere else.) The second time I set this up tonight - yes, I did it twice, it really didn’t take long - I ran into some issues with the XDMCP method but for some reason turning on the VNC server worked just fine. Go figure.

So, here’s a neat thing you can do with some crusty old Unix stuff that is amazingly still very well supported today.