Ditching Docker for Local Development
Ditching Docker for Local Development
Earlier this month I mentioned on Mastodon that I was replacing a Docker-based local development environment at my day job with a Nix-based one, orchestrated with overmind and a justfile. There was quite a lot of interest in particular in how overmind and just could be used to replace a container / ...
Some folks on the internet were interested in how I had managed to ditch Docker for local development. This is a slightly overdue write up on how I typically do things now with Nix, Overmind and Just.
You can have my Docker development environment when you pry it from my cold dead hands!
And keep it that way! We need people on both sides to further spur progress. Plus I'm jealous cause I still don't have a firm grasp on docker.
Fair!
Python, and its need for virtual environments, is what really drove me to master Docker.
I use Kubernetes at work and a simpler docker-compose driven setup at home. I find that it's a tidier way to build infra, but it does have its limits. Abstracting from bare-metal has its ups and downs, but I find the positives outweigh the negatives.
I wish he had written why he's so anti-container/docker. That's a pretty unusual stance I haven't been exposed to yet.
Hi!
First I'd like to clarify that I'm not "anti-container/Docker". 😅
There is a lot of discussion on this article (with my comments!) going on over at Tildes. I don't wanna copy-paste everything from there, but I'll share the first main response I gave to someone who had very similar feedback to kick-start some discussion on those points here as well:
Some high level points on the "why":
-
Reproducibility: Docker builds are not reproducible, and especially in a company with more than a handful of developers, it's nice not to have to worry about a
docker buildcommand in the on-boarding docs failing inexplicably (from the POV of the regular joe developer) from one day to the next -
Cost: Docker licenses for most companies now cost $9/user/month (minimum of 5 seats required) - this is very steep for something that doesn't guarantee reproducibility and has poor performance to boot (see below)
-
Performance: Docker performance on macOS (and Windows), especially storage mount performance remains poor; this is even more acutely felt when working with languages like Node where the dependencies are file-count heavy. Sure, you could just issue everyone Linux laptops, but these days hiring is hard enough without shooting yourself in the foot by not providing a recent MBP to new devs by default
I think it's also worth drawing a line between containers as a local development tool and containers as a deployment artifact, as the above points don't really apply to the latter.
If your dev documentation includes your devs running
docker build, you're doing docker wrong.The whole point is that you can build a working container image and then ship it to a registry (including private registries) so that your other developers/users/etc don't have to build them and can just run the existing image.
Then for development, you simply use a bind mount to ensure your local copy of the code is available in the container instead of the copy the container was built with.
That doesn't solve the performance issues on Windows and Mac, but it does prevent the "my environment is broke" issues that docker is designed to solve
Cost: Docker licenses for most companies now cost $9/user/month
Are you talking about Docker Desktop and/or Docker Hub? Because plain old docker is free and open source, unless I missed something bug. Personally I've never had much use for Docker Desktop and I use GitLab so I have no reason to use Docker Hub.
Appreciate the in-depth response! I've always been interested in Nix but I'm scared of change lol. And I'm a single systems administrator on a team of mostly non-technicals so large changes like that are .. less necessary. Plus you know, mostly dealing with enterprise software on windows unfortunately. One of these days.
Docker performance on macOS (and Windows), especially storage mount performance remains poor
I remember when I first got a work Macbook and was confused why I had to install some 'Docker Desktop' crap.
I also learnt how much Docker images care about the silicon they're built on... Fucking M1 chip can be a pain...
-
Docker is like, my favorite utility tool, for both deployment AND development (my replacement for Python virtual environments). I wanted to hear more of why I shouldn’t use it also.
@astral_avocado @LGUG2Z That definitely would’ve been helpful for readers new to the Nix scene, but I don’t think that’s the purpose of this article. It’s written as more of an example of a way to move to Nix, rather than an opinion piece on why you should move away from Docker.
I won’t try to argue why you should switch. However, I would recommend you look into the subject more, Docker is a great tool, but Nix is on a diffeeent level 🙃
I'm anti everything that requires daily use of arcane command line bullshit. I thought we were on the way to being over that when Windows 3.1 came out.
If it needs to be done more than once, make it a button on a little program. I've rolled my own for any of them that can be triggered from the windows command line. But Docker and others that require their own unique command line I can't do that. I wouldn't be as annoyed by Docker if Docker Desktop just did all the crap it should instead of requiring command line bullshit every damn day.
I mean.. all those buttons are essentially just calling a command line in the end. And coding that button takes more work so command line is always going to be more likely to be your only option. If you find commands arcane then that's probably an argument that the help docs should be clearer or the commands themselves should be clearer.
I use a similar approach, but I went further by creating a system that compose like docker-compose would. The trick was to write my own nix function
mergeShells.https://her.esy.fun/posts/0024-replace-docker-compose-with-nix-shell/index.html
For now, I am pretty happy with it. Also, I put the init script inside nix-shell and not in external files and use exit signal to cleanup the state.
Thanks for sharing this! Added to my weekend inspiration/reading pile. 🙏
Kind of cool if your production infrastructure can match. But for most companies (ie, Fortune 500 and some medium companies) implementing this would need a force majeure.
Decades of software rot, change in management, change in architecture, waxing and waning of software and hardware trends, half assed implementations, and good ole bottom tier software consultation/contractors brought into the mix make such things impossible to implement at scale.
Once worked at a company where their onprem infra was a mix of mainframe, ibm / dell proprietary crap, Oracle vendor locked, and some rhel/centos servers. Of course some servers were on different versions of the OS. So it was impossible to setup a development environment to replicate issues.
For the most part, that’s why I still use docker for most jobs. Much easier to pull in the right image, configure app deployment declaratively, and reproduce the bug(s). I would say 90% of the time it was reproducible. Before docker/containerization it was much less than that and we had to reproduce in some non production environment that was shared amongst team.
Related, this article talks about combining nix and direnv: https://determinate.systems/posts/nix-direnv
Using these tools you are able to load a reproducible environment (defined in a nix flake) by simply cding into a directory.
I’m surprised no one has mentioned either of the following solutions as alternatives to this explanation (and docker) that still uses Nix:
fleek anybody?, https://getfleek.dev/
and if you want setup per project, try: https://devenv.sh/, both are nix based
I know you won't believe this, but you don't need any of these GTOS (giant towers of shit) to write & ship code. "Replace one GTOS with another" is a horizontal move to still using a GTOS.
You can just install the dev tools you need, write code & libraries yourself, or maybe download one. If you don't go crazy with the libraries, you can even tell a team "here's the 2 or 3 things you need" and everyone does it themselves. I know Make is scary, with the mandatory tabs, but you can also just compile with a shell script.
Deployment is packing it up in a zip and unzipping it on your server.
Lot's of (incorrect) assumptions here and generally a very poorly worded post that doesn't make any attempt to engage in good faith. These are the reasons for what I believe is my very first down-vote of a comment on Lemmy.
What?
Sometimes you need complex tools for complex problems. We just have a homegrown GTOS at my work instead, I wish we had something that made as much sense as Nix!