Tips, tricks, and downright hacks to make working as a remote developer just a little nicer
Given everything that's happening right now (c. 2020-07-23), many software developers have been fortunate to be able to work from home. However, there are some challenges I encountered with my office's particular brand of development workflow. Over the last couple months, I've fleshed out a pretty good stack of tools for working around them. In this writeup, I want to go over some of them.
"Remote" vs. Remote
There are really two different paradigms for coding remotely. The first, and most obvious, is to simply clone the code repositories to your local machine at home and work from there.
However, part of the reason the transition to a remote workflow was so challenging for me, anyway, was because my office uses a centralized server for development. Our applications have a complex architecture, job queues, and a large database containing a full-restore of test data from production. We host our development workspaces on a central few servers, and NFS/SMB mount those directories to our local machines. Then, we edit files in those mounts in our workstations.
This is much harder to do remotely, because we can't simply clone the code repos and be up and running. We need all the supporting development tools, software dependencies, NGINX/PHP configuration, and, lest we forget, the database.
This meant that, in the early days, my "remote" development setup looked like this:
That, my friends, is Sublime Text 3 running through a compressed Chrome Remote Desktop session. Yep.
As you can imagine, that was a less than desirable experience.
Trick 0: VPN
This is basically requisite for most corporate networks these days, but for me to have any hope at all of getting a good experience, I needed to ditch the remote relays like Chrome Remote Desktop/TeamViewer. All of the remote tools I'm about to talk about rely on good-ole-fashioned SSH and remote network access.
So, in order to do that securely, you need VPN access to the office. This is something I helped stand-up in the weeks after this all started, so we're good to go there. If your shop doesn't have an existing solution, I've had good experience with Pritunl. Pritunl is a VPN-server-manager that provides a web UI for managing users and VPN servers. It uses OpenVPN under the hood, so it's compatible with basically everything.
This project has the side requirement that we want to avoid joining our main development server to the VPN directly for security/control reasons, so our tools will need to work around that. However, I do have access to the Linux workstation at my desk.
Trick 1: the IDE
So, I'm normally a JetBrains guy. Given the choice, I use WebStorm/PHPStorm for my projects. However, for work lately I've been using Visual Studio Code. Why? One reason: remote workspaces.
VS Code is built using web technologies, which means that it's really good at running over a network connection. In fact, several online IDEs are build using VS Code for this reason. This feature also enabled another tool to exist: sshcode. sshcode is a wonderful little CLI tool that automates the process of SSH-ing into a remote computer and installing/starting VS Code's web server.
Then, it tunnels the VS Code server port over SSH and launches Chromium in app-mode on your local machine. That entire process results in what looks like native VS Code running on my local machine:
The benefits of this over remote desktop are many, but here are a few:
- Because the front-end of the editor is running in Chromium on your local machine, there is zero extra latency when actually coding in the files. The only things that are slightly slower are file-operations like searching/saving/opening files.
- Extensions even work! If you use the
--bflag on the sshcode command, it will 2-way sync the extensions from your local machine, so any extensions you install will be saved and restored. - You can have multiple windows of this open across multiple displays, unlike remote desktop which is usually limited to a single display.
- Because the VS Code server is actually running on my workstation at the office, I can open the NFS shares that I normally work on, without the extra overhead of having to mount the NFS shares to my local machine over the VPN.
Probably the biggest quality-of-life improvement for me, though, was the latency editing files. As someone who relies on quick keyboard shortcuts and tricks to navigate/edit code quickly, editing over remote desktop was a nightmare. It may not sound like much, but the extra 100-200ms of latency adds up quick.
Trick 2: SSH
Okay, okay, this isn't particularly ground-breaking, but I felt I should include it for the sake of completeness. When I'm working, I use Guake as an overhead shell, and I have no fewer than 6 tabs open SSH-ed into my remote machine. This means that, at any time, if I need to jump into the shell for something, I don't have to wait for SSH to start up. This goes a long way to making it more immersive.
Plus, having a lot of shells at the ready allows me to do long-running commands like sshcode, or the Angular development server, or HTOP, &c. without having to worry about running out of space.
And, because I'm using VS Code to edit the files that exist on my remote development workspace at the office, all the CLI tool's we have in-house, or that I've added personally still work.
Trick 3: the Database
Because our applications are complex, we have rather large databases with lots of data that we work with and modify throughout the development workflow. So, having a database browser that can keep up with it is a must.
While, for simpler projects, just using MySQL/MariaDB/PostgreSQL from the CLI when you need to check things in the table is probably fine, when you get to the realm of hundreds of tables and millions of records, this becomes impractical.
On my office machine I use DBeaver for this. DBeaver is an awesome cross-platform database tool, and I've gotten pretty quick navigating its interface.
I had been using DBeaver over remote desktop, but again the latency was driving me nuts. Luckily, DBeaver (and most database browsers) have a feature that allows you to tunnel the database connection through SSH:
This means that I can configure the connection as though I'm sitting at my office PC, then configure DBeaver to tunnel the connection using the "Agent" authentication method, and it will connect as if I'm on my work PC.
This is awesome because it means that I can access and work with the databases using the software I'm familiar with, but running on my local machine.
This has many of the same benefits as VS Code server. The interface is dramatically less latent, however there is a bit of a longer delay when connecting to/fetching data from the databases, because it has to tunnel through SSH. However, I've found the slight loss in loading times to be well worth the improvements in UI latency.
Side note:
For personal projects, and because I'm a JetBrains shill, I prefer DataGrip as my DB browser, which also supports similar SSH tunneling, and will even store the schema introspection information locally to make searching through and jumping between databases/tables faster.
Trick 4: Web Development
In case it wasn't clear, I work on a lot of web applications. This means that they need to be tested through a web browser. Because our development is hosted on central servers in the office, the way we do this is to navigate to our user-specific domain on the server, and NGINX connects us with our development workspace.
Something like http://my-username.my-project.some-server/. In the office, this works really well, because it removes the load of running the code and databases from our local workstations to the much more powerful server. However, because we wanted to avoid joining the main development server to the VPN for security/control reasons, this means that I can't directly navigate to my project URL on the server from my local machine.
So, for a long time I was (you guessed it!) doing it through remote desktop. Again, again, latency was terrible, and having to cram the browser's devtools onto the same screen as everything else made debugging difficult. (I usually spread them across 2 screens, but remote desktop doesn't support that.)
After a few weeks of this, I got fed up enough that I spent a couple hours of searching for a solution. I finally landed on something pretty nice: Squid.
For background, I usually use Firefox Developer Edition at work. This is nice because 1) I'm a Firefox person anyway, and 2) it lets me keep my work profile separate from my personal profile in normal Firefox.
Firefox has an interesting feature that I haven't been able to replicate in Chromium that proved useful: browser-level proxy support. See, Chromium will use a proxy, but only if it's configured at the system level. However, Firefox will happily let you configure a proxy in its settings, and it will use it to browse the web. This is important, because I don't want to run my entire local machine through a proxy.
So, I set up a very basic Squid proxy on my office workstation. This is, perhaps, the oldest trick of the bunch. If you've never heard of Squid, it's an open-source web proxy that's been around forever whose main goal is caching resources to make browsing more efficient. However, it has the powerful ability to act as a general HTTP/S proxy.
After setting this up on my work machine, I was able to configure Firefox Developer Edition on my local machine to use the Squid proxy through the VPN. This essentially allows me to browse the web as though I were using my office workstation, but locally:
Helpfully, Firefox lets you whitelist domains that don't need to run through the proxy. So, things like DuckDuckGo, Google, and DevDocs can use my local Internet connection, making them a bit snappier.
Setting up the proxy allows me to navigate to my development URL at work, and Firefox will proxy all the requests through my work PC, so the development server doesn't need to be on the VPN at all, and I can still use my devtools natively.
HTTP/S proxies are very common in large corporate environments that need to provide remote users with access to internal services, like Intranets. This is a much smaller scale version of that. In fact, this setup allows me to access other internal tools like our Git repositories, Hound instance, and deployment tools, which is a nice bonus.
Trick 5: Remote Desktop
I know, I know. I've been railing against Remote Desktop and the latency involved for like 1600 works now. While the above tools mean that all the stuff I do on a regular basis can be done directly from my local machine, there are still one-off things that I need to remote in to my workstation to do.
For example, our time clock in the office only works if you access it from w/in the local network. So, to clock in/out, I need to remote into my workstation. Tasks like this aren't so frequent that they would actually benefit from a more over-engineered solution.
However, tools like Chrome Remote Desktop and TeamViewer have issues. For one, they open your corporate network up to the control of a 3rd party. Even if the company like Google or TeamViewer has a pretty good security track record, it's still a third party service that you're relying on to exist for the forseeable future.
The other issue is that tools like TeamViewer can be exorbitantly expensive. Like $50 a seat, expensive. For a large company, this might be worth it. But, for a smaller shop, the extra cost per developer per month is non-trivial.
But, there are a ton of really good remote desktop tools for LANs, and we just so happen to have a VPN set up that gives us virtual network access to our machine. So, there are two possible solutions here:
Windows
If you use Windows, or are willing to slum it, Windows Remote Desktop is a pretty good tool, and it will work over the VPN. This will give you secure access to your remote machine.
Linux
VNC! I use TightVNC Server. VNC is a great tool, because the desktop session runs on a different display screen than the physical session. So, you don't have to worry about running your physical monitors at work. I've set up a profile in Remmina on my local machine so any time I need to access my work PC, I can just click an icon in my GNOME tray:
Conclusion
I hope someone else might find these tools useful in their transition. The more complex parts, like the VPN and HTTP/S proxy, might seem difficult to set up, but with some patience, you can get them up and running. The nice thing about the VPN/proxy setup is it can be shared by several developers.
In fact, if you have multiple people working remotely, you can (and should) set up the VPN and proxy on a shared server, so that it only has to be set up once. Then, everyone can use those tools.
For any job, the transition to remote work can be difficult. Particularly as developers, we sometimes rely on network resources and specialized software to make our workflows possible. However, through a process of trial and error, these are the tools I use to make my remote work much nicer.
In fact, between the HTTP/S proxy, remote VS Code, DBeaver's SSH tunnel, and plenty of SSH tabs, I rarely have to remote into my work machine directly. While there are still some drawbacks, the experience is not much different than if I were sitting at my office machine developing directly, and that's pretty great.