What do package managers do? Install packages, obviously! But that is not everything. In my opinion, package managers do enough to be characterized as general automation frameworks. For example:
- manage configurations and configuration files
- manage custom compilation options and flags
- provide isolation or containerization
- make sure a specific file is present in a specific place given specific conditions
- change installation files or configuration based on architecture or other conditions
Not all package managers do all of the above, but you get the idea.
Nix even manages your entire setup with a single configuration file.
It occurred to me that package management could theoretically be managed by an automation framework.
What do I mean by automation framework? Ansible, chef, puppet, or Sparrow.
Now imagine if you were to use one of those package managers as an automation framework. For most of them, it would suck. (nix is a notable exception). So maybe common package managers are just bad automation frameworks?
What if we used an automation framework as a package manager? Well currently, it might also suck, but only because it lacks the package definitions. Maybe it is not a bad experiment to have a distribution managed by a modern automation framework like Sparrow.
What are the benefits?
- usable on other distributions
- more easily create your own packages on the fly
- Greater customization and configurability
- use a known programming language that is easy to read to define packages and other functions, instead of a DSL
- your package manager can easily automate just about any task using the same syntax and framework
Well you’re really feeding my Nix confirmation bias here. I used to use Ansible with my dot files to configure my personal computers to make it easy to get set up on a new machine or server shell account. But it wasn’t great because I would have to remember to update my Ansible config whenever I installed stuff with my OS package manager (and usually I did not remember). Then along came Nix and Home Manager which combined package management and configuration management in exactly the way I wanted. Now my config stays in sync because editing it is how I install stuff.
Nix with either Home Manager or NixOps checks all of the benefits you listed, except arguably using a “known” programming language. What are you waiting for?
I am very interested in Nix. The only thing making me hesitate is that it is a bit opinionated. There’s a “Nix way of doing things” rather than a general automation framework than can do anything. Am I wrong in thinking this as an outside observer?
I haven’t worked with the deployment tools which is what I think would make the most direct comparison to other automation frameworks so I don’t know how comparatively opinionated they are. I suspect it varies between those tools.
Using NixOS or Home Manager there is a certain way of doing things. But these are intended to be opinionated. Packages and modules come from either nixpkgs, or from third-party flakes which package for Nix. Services are usually orchestrated through systemd units which come from Nix packages.
You can do anything with Nix. The tools and frameworks encourage certain ways of doing things. But that depends on the framework. You can always build a new framework that works differently. Since they are all based on common concepts of Nix expressions, derivations, and in many cases flakes you get a certain baseline interoperability between frameworks.
I think a big part of what you’re getting at is that we (as humanity) deserve a pull-model, human readable, platform independent, outcome descriptive / desired state / idempotent computer configuration standard (RFC).
Quite a few package managers and orchestration tools almost get us there. Ironically, since package managers have been around longer, on average, many of them scratch that itch as well or better than many of the orchestration platforms.
I’ve done it both ways, many times, and I currently prefer a poor-fit-to-task orchestration tool, over an excellent fit-to-task installer package. And I’ll take a well written Makefile over both of the others combined. That may just be my “old man yells at cloud” energy, though.
Funny you brought up make files.
We’ve been churning though Java technical debt for the past year and a huge pain point is that a lot of configuration gets lost within intelliJ.
Most of this is env vars and jvm args. These could be wonderfully documented using an .env.example and a well written makefile.
As a middle career technology professional just before reading this comment and thread I had the thought:
“Make files really are the only correct way to distribute software”.
Even with OCIs and soon Wasm Components, a makefile can still cover the constant changes in development trends. They can also wire together bash scripts used for tasks and maintenance. Bash and make really are some of the best swiss army knives we have.
We’ve been churning though Java technical debt for the past year and a huge pain point is that a lot of configuration gets lost within intelliJ.
You’ve lost Single Source of Truth.
“Make files really are the only correct way to distribute software”.
Nope. They’re the best way to build your immutable artifacts. Building packages, though, should not be done inside your makefile but by the packaging layer that should sit outside the makefile: horse before the cart and never after. I say this knowing that older/crippled packaging formats and processes do this wrong. We have 3 decades of knowledge to leverage and we still get drek like IP5, but it’s a revelation to understand you need to keep the layers distinct.
We use Puppet to manage Linux and Windows servers + Linux, Mac and Windows client (drives, registry/defaults/dconf, …). A package manager can’t handle this properly ;)
It occurred to me that package management could theoretically be managed by an automation framework.
This is how I know you don’t understand proper package management. The validation cornerstone seems to have been forgotten here.
But you’re close to something. The state of proper package management had us managing systems by updating packages - using files, triggers and templates - and had more responsive code (they call it ‘self healing’ now) than ansible, about 8 years before they even invented the term ‘devops’.
It’s amazing how forgetting the past lets us repeat it. That Ansible is such a dog, almost 25 years after
makeandsedwas most of our engine, shows how much potential Mike squandered.I did not intend to make an exhaustive list of everything package managers do, but validation is a good thing to point out. It further proves that we are dealing with generic automation frameworks.
And just to clarify, I did not mean that something like ansible in its current form is suitable.
Using a programming language to express configuration outcomes, while convenient, has caused me plenty of regret.
I now prefer a domain specific language, so long as it expresses the necessary desired end state, rather than the steps to get there, whenever possible.
Of course, I’'ll trade a domain specific language for an RFC defined shared standard in a New York Minute, given the chance. But of course now there are 16 competing standards applies to our progress towards this ideal.
Why has it caused you regret?
Nothing wrong with a DSL inherently, it is just harder to get right. Maybe Nix does (judging by its popularity. I haven’t used it enough to judge), but in general, package manager DSLs never end up better except for a subset of cases.
What are those 16 standards? I haven’t heard of this before but it sounds interesting!
Although this XKCD is funny, I do not agree with its premise. The presence of many competing standards is usually due to some underlying issue or due to a good reason.
For example, the competing standards in package management is more due to distributions reinventing their own implementation of doing almost the same thing. Debian and Fedora, for example, aren’t doing anything drastically different. Moreover, they don’t make it easy to use outside of their systems.
Why has it caused you regret?
Any configuration coded in, for example, Python 3.12, will cease to be useful in October of 2028, when it will become quite difficult to find an active useable Python 3.12 runtime.
Whereas the same configuration, in an Ansible Playbook, will likely continue to function just fine, under the latest Anbile and Python versions.
I’ve experienced this rodeo enough times that I trust Domain Specific meta-languages much more than direct program code, for configuration.
On the XKCD, yeah. It’s a joke. Randal Munroe isn’t actually advocating to stop trying to fix our standards.
I refer to it here as a reminder that there will always be fatigue with each new RFC, and that’s okay.
Also, not every RFC will turn out as timeless and eternally useful as RFC 2324 And there’s no excuse for my bringing it up here, except that I like it.
what the fuck is sparrow? also salt-stack does all this. nothing sucks you are just doing it wrong.
