Welcome to Docwright

Site orientation

You can read this site in order, or just jump to whichever section interests you. It starts with backups, because that’s what saves you when a laptop dies or a file goes missing. Then it moves to version control, which sounds technical but mostly just means keeping a history of your drafts so you can get back the version you had, say, last Thursday. After that comes security and moving over to Linux. That a big move, so the posts go slowly. Then it moves to running AI models on your own machine, so nothing you write gets sent off to someone else’s servers. And last, how to put your writing online as a website like this, using mdBook. None of these require you to know how to code.

The core problem

Think of a book that changed your mind; not the one you enjoyed, but one that moved you to disavow a position you once held. Now find your notes on it; not the book, the notes, the objection you scrawled in the margin, the passage you copied out, the thought you had and scribbled at at dawn’s nautical twilight.

Good for you if you can. Most do not even takes notes. And many who do, for various reasons, cannot find them when needed. A lot of labour was wasted. And that essentially is what this site attempts to address: Help people work faster and lose less of what they learn.

If you are going to spend years learning – reading, studying, writing, researching, making notes, developing ideas, cultivating a body of work, a garden, if you will – you need tools, weapons even, to keep it. What’s the point of building something you cannot keep and, more importantly, pass down to your children?

For those who don’t read

“Lol, I don’t even read,” one might say, “and I certainly don’t write.” Well, my friend, then I invite you to start today. There’s no other way to a life of significance. Socrates, on trial for his life, said the unexamined life is not worth living. Moreover, watching reels, shorts, and YouTube videos fry your dopamine receptors, damaging your self-control and focus. The antidote to the doomscrolling addiction is reading, slowly at first, with full attention, and smart notetaking. And if you’re a knowledge worker, reading is the only way to truly become skilled at your vocation. You don’t want to reach your 40s and have nothing to show for it. Develop a skill that’s of service to the community. And the more you read, the more skillful you will grow.

“…the majority, though they are sometimes frequent readers, do not set much store by reading. They turn to it as a last resource. They abandon it with alacrity as soon as any alternative pastime turns up. It is kept for railway journeys, illnesses, odd moments of enforced solitude, or for the process called ‘reading oneself to sleep’. They sometimes combine it with desultory conversation; often, with listening to the radio. But literary people are always looking for leisure and silence in which to read and do so with their whole attention. When they are denied such attentive and undisturbed reading even for a few days they feel impoverished.” – C S Lewis, An Experiment in Criticism

Why you should also write

The wise not only read voraciously, but they also write. One, writing helps thinking. In fact, if “people cannot write well, they cannot think well, and if they cannot think well, others will do their thinking for them” (some attribute that quote to George Orwell). So write, even if your memory is perfect. Two, writing helps the next generation stand taller on the shoulders of those who wrote. It doesn’t matter if you’re a farmer, machinist, or plumber. Don’t let your knowledge die with you. Sure, it’s likely that what you have to say has already been said, but you could still in your writing point people to those who’ve said it or said it better. And, finally, write cause you never know when what you considered insignificant becomes valuable to society.

Three, writing is cumulative because ideas intersect. A note written today can become a citation next year, an argument in three years, a chapter in ten, and eventually a book. That happens only if the corpus survives, stays searchable, and stays yours.

With great power comes great responsibility

Our fathers had nothing like the tools now available to us for free. Yet very few ever learn to own the ground their intellectual life. Files live in somebody else’s cloud. Notes are scattered across notebooks, PDFs, Word documents, LLM platforms, browser tabs, and forgotten folders. Research is spread across a dozen applications that do not speak to each other. Private material sits on someone else’s servers, under someone else’s terms. Backups are thin, untested, or absent. Identity rests on passwords held by companies. And publishing waits on someone else’s permission. Why in 2026 would you live like this? You have no excuse. It’s never been easier to spin up a blog or online book and share with others what you’ve learnt, all for free.

The goal is not a resume

The goal is not to become a sysadmin. It’s not to collect technical skills so you can show off on your resume. The goal is wisdom, understanding, a life worth living, maybe passing an inheritance to the next generation. In pursuing those goals, Docwright teaches you about tools that will increase your productivity and power, yet the tools are not the garden.

Site usage-guide

An operational guide to reading docwright

Navigation

There are several methods for navigating through the pages of the knowledge base. The menu button (three horizontal bars) at the top-left of the page to open and close the sidebar. The sidebar lists all the sections and topics of the knowledge base. Click any topic to load that page. The menu bar, on the top, holds all the icons.

The keyboard shortcut b opens and closes the sidebar. The < left and right > arrow keys moves you to the previous and next page; the big arrow buttons at the sides of the page do the same thing. The ∧ up and down v arrow keys scroll up and down a page. The PgUp and PgDn jump high-level headings; Spacebar and Shift + Spacebar keys do the same thing. Clicking any where on the menu bar, other than the icons, will take you to the top of the page.

Note

If the window is too narrow, particularly on mobile displays, sidebar will not automatically appear. In that case, use the menu button to open and close the sidebar.

Menu Bar icons

Theme

There are only two themes available, light or dark. This site’s theme aligns with and changes automatically when you change your system’s theme.

Search

To search for a topic on the site, open the search bar, and enter keywords to see matching topics and headings in real time. In the search list, the ∧ and v arrow keys help you select a result, and Enter opens that section. Esc closes the search bar.

Overview

This is a series for those who have never seriously thought about backup and digital security.

Backups give you a second chance. If a copy doesn’t exist when the disk fails or laptop stolen, no amount of skill can recover it. Why make backups makes the case for backing up, keeping offline copies instead of trusting cloud services, using Borg, and why you mustn’t procrastinate setting up a reliable backup system.

If you’re convinced of Borg, then Getting started with Borg explains how to use Borg. Even if you plan to use the Borg-simple script, a wrapper to make the backup process a breeze, it’s important to know the fundamentals of interacting with Borg.

If you plan to upgrade your backup system – from a simple bring-all-your-data-into-one-place and backup using Borg on an external drive, to a sophisticated well-thought-out system, then see Planning for leveling up; it covers topics on decisions the tool won’t make for you: what actually needs backing up, how many copies and where they live, how long to keep them and how to prune, how often to verify, and how to rehearse a restore so you know the system works before the day it has to.

Once you know what you want, stop typing it out by hand. Leveling up, Part 1 covers setting all of it up once — sources, exclusions, retention, verification — in a config you write on day one and rarely touch again, driven by a single command from then on. Backups you have to remember to do carefully are backups you’ll eventually do carelessly. The aim is to spend an hour now so that this costs you nothing later. Leveling up, Part 2 covers how you can beef up your security by using GPG, so your passphrase is not in plain-text.

Whys of backup

TL;DR: Get a working backup running today. Someday a life could depend on it. Borg is the best backup tool so far. Start simple. Iterate later. Bring your data into a single folder, get a couple of USB sticks; and if you’re convinced of the significance of making backups, the urgency of making it today, the danger of using cloud services, and the dominance of Borg, proceed to the next article.*

Why backup?

It’s the first thing every serious knowledge-worker should do, for data loss is inevitable. If you’re convinced you need to backup immediately, skip to the Why offline or the Why Borg section.

Data loss is inevitable

There are two kinds of people, as an old saying goes, those who have lost important data and those who will. Drives fail. Devices get stolen. Mistakes happen. Malware corrupts or locks what you have. The question is not whether you will face data loss, but whether you will be prepared for it.

Recovery is a pain

Recovery without a backup is painful and time-consuming. The asymmetry between the effort of setting up and running a good backup system and the stress of recovering lost data without one, is vast. And to keep a backup system going costs almost nothing. The hard part is the one-time work to set it up. This section is meant to reduce the effort; and once you’ve understood the principles, setting up a quality backup could take you less than an hour.

Someone’s life may depend on it

Data is not a luxury. It shapes how you work, how you remember, how you prove who you are and what you have done. For most people, the loss of important data is a serious inconvenience. For some it can be catastrophic: a doctor whose patient records vanish, a journalist whose source files are wiped, a small business owner whose client database disappears overnight. And once in a while the right file at the right moment is the difference between life and death: a medical history in an emergency room, the evidence that clears someone, the records that locate a missing person.

It’s a way to honour time

Even if the stakes may not be as high or dramatic, it’s prudent to honour your time. Your data represents time, your most finite and non-renewable resource. Every file created, every note written, every photograph you have taken, is time spent. To be careless with it is to dishonour your time. You never know when you will need it again. A backup plan is, among other things, an act of respect for your own labour and time.

Why offline?

Don’t upload your data to Google Drive or One Drive or some other storage service. That’s a bad idea, even if it’s encrypted. Take responsibility for what’s yours. Don’t be a wuss.

Who says it’s encrypted?

Most consumer cloud is not end-to-end encrypted at all. “Encrypted at rest” means the provider keeps a key and can read your files whenever it chooses. There is nothing for an attacker to break, because the provider can already decrypt, and so can anyone who breaches them, buys them, or compels them. Even the services that are genuinely end-to-end write and ship the software that does the encrypting. A single update can lift your key, or your files, before they are ever encrypted, and you cannot audit every release. Unless it’s a fully open-source app, don’t trust it.

Your custodian is a moving target

The company can be breached, sold, merged, subpoenaed, served a foreign warrant, or simply shut down. It can change its terms or close your account on a morning you had no reason to expect. Furthermore providers consolidate, so the entity holding your data next year may not be the one you chose this year. Legal demands on them rise rather than fall. Their incentive to mine what you store, to train models on it, or to sell access to it, climbs every quarter.

The stored copy only gets easier to open

Then there is the long game. An attacker does not even have to break your encryption now. He copies your encrypted files out of the cloud today and simply waits. Computers keep getting stronger, and the quantum kind is expected to break some if not all encryption algorithms. When that day arrives, the attacker unlocks the copy he took years before. Every cipher and hash in wide use before roughly 2000 has been broken, weakened, or retired; As recently as July 2026, Anthropic published attacks found using Claude Mythos Preview — one significantly weakening HAWK, a post-quantum digital signature scheme, and one identifying a new way to attack round-reduced AES. Now most real-world cryptographic loss came from key sizes aging out, implementation bugs, protocol design, and deliberate sabotage, not from mathematicians cracking the core algorithm; yet handing over your data means you’re still subject to this attack.

Offline removes the remote attack surface

Now an offline drive, has no remote attack surface to speak of, no account to phish, no client to backdoor, no provider to compel, no stored blob to harvest. Your threat model shrinks to the physical: fire, flood, theft, and you deal with physical failures that the simple way: keep more than one copy in more than one place. Physical risks do not scale the way network risks do. Breaching one server affects a million people at once, but how does one burgle a million drawers? The chances of an attacker reaching the drive in your home, the drive at your friend’s home, and the drive buried safely in your ancestral home all on the same afternoon, is slim. Offline backups give you a boundary you can see, inspect, and defend. Later, when the offline drawer of drives isn’t enough, and you want a copy that lives somewhere remote, the answer is still hardware you own: a small home server such as StartOS, which is a self-hosted machine you run yourself, rather than a company’s cloud. But that’s a subject for another day.

Why Borg?

Borg is the best backup-program available today. Don’t take my word for it; do your research, but here are some facts. Nobody bases a product on a backup engine they do not trust. BorgBase a hosting service specialized for BorgBackup, offers append-only repositories and two-factor authentication, and it funds development of the surrounding tooling. Hetzner a German cloud and hosting provider known for affordable, high-performance servers, added native Borg support to its Storage Box product, with an extended SSH service, official documentation, append-only mode, and per-version remote-path pinning. Rsync.net, a longest-standing offsite-storage providers, supports Borg natively as well. On the desktop, Pika Backup, a GNOME application, is powered by BorgBackup, and Vorta provides a Qt front-end over the same Borg backend, while Borgmatic wraps it for scheduling and retention policy. A whole ecosystem of CLIs, Docker servers, and web UIs have grown around the engine.

Borg’s headline accomplishment is reach. The main repository carries roughly 10.9k stars and 739 forks (at the time of this writing). Borg ships as a packaged tool in the Debian, Fedora, and Arch repositories among others, meaning distribution maintainers vetted and adopted it independently rather than leaving it to users. That’s saying a lot. Open Source Everything list names it as the backup tool. These are evidences that experts use it rather than just star it.

On reliability it has earned trust the hard way. Borg deliberately reuses the system SSH client and links only OpenSSL’s libcrypto rather than implementing its own network crypto protocol, and it publishes an explicit threat model spelling out that an attacker cannot modify, rename, remove, or add an archive without the client detecting it. It has run in production for about a decade, descending from Attic, which was accepted into Debian in August 2013 before being forked as Borg in 2015, and it is still actively maintained: lead developer Thomas Waldmann and the project run on community funding through GitHub Sponsors and Open Collective. Across that decade it has had one notable cryptographic flaw, CVE-2023-36811, an archive-forgery issue disclosed and fixed in version 1.2.5 with a documented upgrade procedure, which is the responsible-handling record that matters more than any clean-sheet claim.

Borg uses a technique called deduplication, which means, after the first backup, it only stores the changes of the files being backed up, making it suitable for daily, hourly, and even secondly backups. It supports compression and authenticated encryption, which makes it suitable for storing backups in untrusted locations. To learn more, read the official docs; and verify for yourself if it fits your need.

Why today?

The perfect backup system is worth striving for, but it’s unlikely you’re going to build it all in one day. And the perfect system will always be worse than a working one that’s running today. A single working encrypted backup on an external drive and a password written on paper, is much better than the perfect system you get to eventually. Moreover the perfect backup system is a moving target; the bad guys will never stop trying to find ways to undermine your privacy and sovereignty. And significant improvements in security and resilience does not need the perfect setup; it needs a setup that actually runs today. So start simple. Iterate later.

Now the ‘Levelling up’ pages in this section help you setup a pretty darn good setup – quickly – but they assume you have your data arranged, you’re using Linux, and you understand the principles of a quality backup-system. And if you don’t have a local working backup, it’s a good idea to learn the basics, the bare essentials, before you move on to a complex system. Knowing the basics hopefully gets you out of trouble when the fancy setup fails.

To proceed with the recommend order, all you need is to bring your data into a single folder, and use Borg to back it up on to a USB. The next post helps you do just that. If you’re not convinced of Borg, use dummy data for now. Get a feel of how easy the backup process with Borg really is; a few simple commands in your terminal and you have an encrypted, compressed backup. Then do your research, and when you’re convinced that Borg is the best, you can repeat the steps with your actual data.

Getting started with Borg

Once Borg is set up, making a backup up takes two commands and restoring it takes two.

Note

OS: Borg runs on Mac and Linux, not Windows directly.

Windows users, what are you still doing there? If you really cannot migrate away from Windows, you can run Borg through WSL (Windows Subsystem for Linux): install WSL, then follow the Linux steps below inside it.

Prerequisites: You will need the terminal; if you’ve never used one before, see this brief guide. It’s also a good idea to go through Creating passphrase; it explains the best practices for creating secure passphrases.

Video demo: If you prefer first watching a demo on how Borg works, see the official demo. It’s old but good.

1. Install Borg

On Debian Linux

sudo apt install borgbackup -y

On macOS

Install Homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

When Homebrew finishes it prints a couple of lines starting with eval or export that tell you how to finish setup; copy those, paste them in, and press Enter. Then install Borg:

brew analytics off
brew install borgbackup

2. Create the repo

Borg calls the backup folder a repository: an encrypted, compressed container that holds your data. You have to do this step only once per drive unless, of course, you lose or damage your drive.

2.1. Get the path to your drive

Plug in your external drive or USB stick, and find the path to it.

Note

On macOS

It appears at /Volumes/your-label. To read the exact path on a Mac, open the drive in Finder, press Cmd + Opt + C to copy it as a pathname, and paste it anywhere; you will see something like /Volumes/your-usb-label; e.g. /Volumes/backup1.

On Linux

It usually appears at /media/your-name/your-usb-label; e.g. /media/john/backup1/.

Use the path to your drive.

Tip

If you have no spare drive, you can still test Borg by using a folder on your system, say~/Documents. The point of backing up to a drive is so that if your laptop fails or gets stolen, your backup is not lost with it.

2.2. Name and create the repo

Name the repo anything you like, and run the command to create the repo** (edit the path and repo name to match yours):

borg init --encryption=repokey /Volumes/backup1/borg

On Linux that path should look something like /media/john/backup1/borg, and if you are testing Borg on your system, then it would look something like ~/Documents/borg.

2.3. Enter the passphrase twice

Pick a strong one. For tips on creating a secure passphrase, see Creating passphrase. When it asks whether to show the passphrase for verification, type n.

Caution

Write this passphrase on paper and keep it somewhere away from the laptop. If you lose both the passphrase and the key, your backups are gone for good, with no reset.

Tip

That’s it; that’s the whole one time setup with Borg. From here on, unless you lose your drive, it’s just one command per drive to backup; and two, to restore.

If you level up and start using scripts, then it’s one command to backup to all your drives; and one command to restore from any one of them. See Levelling up details.

3. Back up

Note

From here, every command below uses /Volumes/backup1/borg; replace it with your own path.

Using the terminal, navigate to the folder that contains the folder you want to back up. Say the folder you want to backup is mystuff and it lives in your Documents folder, run:

cd ~/Documents
borg create -s /Volumes/backup1/borg::{now} mystuff

That’s it. You’ve successfully backed up your folder to your borg repo. You should see something like this.

Note

If you don’t cd into Documents but run create with ~/Documents/mystuff, Borg backs up the whole path such that then when you extract you will have Documents and inside it mystuff. While if you cd first, then extract gives you just the folder you backed up.

{now} names this backup with the date and time, so each one stays separate. If you would rather name it yourself, swap {now} for anything; e.g. ::archive. Borg requires archive names within the same repository to be unique. If you try to create a new archive with an existing name, Borg will fail with an error similar to: Archive already exists: <archive-name>. This is by design because each archive is an immutable snapshot.

A common practice used by those who back up more than one machine use ::{hostname}-{now} so each machine’s backups are labelled and sort together; for one folder on one machine, {now} is all you need.

-s stands for stats, shown in the image above. You may remove that if you don’t care about it.

Note

The first backup copies everything, so it takes a while: roughly two to three minutes for 10 GB to a fast external drive, and longer, ten minutes or more, to a cheap USB stick. After that Borg remembers what it already saved and only adds what changed, so every backup after the first is usually done in seconds to a minute.

Tip

Back up after every change you care about. Next time you do not have to retype anything: plug in the drive, open the terminal, press the up arrow until the borg create line appears, and press Enter.

Important

Back up on a schedule you will actually keep, once a day or once a week, but definitely before you switch or wipe a laptop.

If you keep a second drive labelled backup2, set it up and backup the same way (borg init --encryption=repokey /Volumes/backup2/borg); just change backup1 to backup2 in the path.

Keep one of your drives somewhere else: at work, with family, or in a safe. Two drives in the same drawer both die in the same fire or theft; the one stored elsewhere is the one that saves you.

4. Restore

4.1. Select archive

borg list /Volumes/backup1/borg

This lists every backup you have made, newest at the bottom. Each line starts with a timestamp like 2026-05-09T19:15:31 if you used {now} as your archive name. You normally want the latest; older ones matter only if the newest is damaged.

Copy the archive name, which in this case is the timestamp, of the backup you want from the list.

4.2. Extract

Navigate to wherever you want to extract your backup; e.g. Downloads. Then paste your timestamp in place of the one shown:

cd ~/Downloads
borg extract --progress /Volumes/backup1/borg::2026-05-09T19:15:31

You will find your mystuff folder; move it wherever you want it. And that’s how easy it is to use Borg.

Important

The first time you set this up, do this once with any backup to confirm it works. A backup you have never restored from is only a guess.

The closing section of getting-started, after extract, telling the reader to check the archive holds what they think before they rely on it. Raw borg only, since that page has no script. Leading with the listing check and using diff -rq as the thorough follow-up, per 1.1.

5. Verify

You can now make an archive and get it back. That is not the same as knowing the archive holds what you think it holds.

Borg will not tell you when it does not. A mistyped path, a folder you meant to include and did not, an exclude that caught more than you intended: each of those produces a run that finishes cleanly, reports a size, and is quietly missing things. There is no warning, because from borg’s side nothing went wrong.

So before you delete anything, or start relying on this, spend five minutes checking. Do it straight after making an archive and extracting, while the source has not changed underneath you.

5.1 Compare

Compare whichever you got against the original:

diff -rq ~/restore-test/Documents ~/Documents

No output means the two are identical, which is the answer you want.

If there is output, read it before you worry.

Only in /home/john/Documents: something means the live folder has a file the archive does not. This is the line that matters. Either that file was made after the archive was, which is fine, or it was never backed up, which is not.

Only in /home/john/restore-test/Documents: something means the archive holds a file you have since deleted. Normal.

Files ... differ means you have edited that file since archiving. Also normal, and the reason to run this immediately after archiving rather than next week.

What you are hunting is a whole folder on the “only in the live copy” side. Individual files differing after a few days is the system working.

Clean up when you are done:

rm -rf ~/restore-test

5.2 Test on a different machine

Everything above tests the archive. It does not test you.

Restore onto a different machine, or a fresh user account, with nothing but the drive and the passphrase in your head. That is the situation you are buying insurance against, and it is where people find out that the only copy of the passphrase was on the machine that died.

Do it once, deliberately, while nothing is wrong.

Leveling up: Part 1

This is the first of three guides to borg-simple, a shell script that wraps BorgBackup. The command you type is backup. It makes encrypted, versioned backups of the folders you name, to every USB drive you name, from one command.

This part sets everything up with your passphrases in a plain text file that only you can read. That is the simplest arrangement, it is the only one that works unattended from cron (automated backups), and it is where you should start.

Part 2 encrypts that file with GPG, which has stronger security at rest, but it costs you the ability to run automated backups with a timer.

The last section of this guide, Doing all of this without the script, gives the borg command for each thing the tool does. The script is a convenience, not a dependency: your backups are ordinary borg repositories, readable by ordinary borg, on any machine, forever.

What you need

• A Linux machine. These were written on Devuan; Debian, Ubuntu, Mint and their relatives all work the same way.
• One or more USB drives. Two is comfortable, three is generous, four is careful. They do not have to match in size or brand.
• The backup script script, saved somewhere you can find it.
• About fifteen minutes.

Terminology

You will see these words in every message the tool prints, so it is worth ten minutes now.

Drive label. When you plug a USB drive in, your system mounts it, which means it makes the contents appear at a path. On most desktop Linux that path is /media/your-name/something. That last part is the drive label. It is usually the name you gave the drive when you formatted it. d1, d2, d3 are good labels because they are short and you will type them.

Repository, or repo. A folder on the drive where borg keeps backups. Think of it as a vault with one lock. Everything inside one repo shares one passphrase.

Archive. One snapshot inside a repo. Every time you run backup, each repo gets a new archive, named with the date and time. Yesterday’s archive does not go away when today’s is written, which is what “versioned” means.

Deduplication. Borg splits your files into chunks and stores each distinct chunk once. A second archive of a folder you barely touched costs almost nothing. This is why you can keep months of daily snapshots on a drive that is not much bigger than the data.

Retention, or pruning. Deleting old archives on a rule, so the repo does not grow forever. “Keep seven daily, four weekly, every monthly” is a recommended retention rule.

Passphrase. The secret that opens one repo. Borg encrypts everything with it. Lose it and the backup is a pile of noise to you or anyone who steals the drive.

Why not just type borg commands

Borg is a good program, and you can drive it by hand; the last section shows you how. With the script, this is what you stop doing.

1. You don’t have to install borg first; if it is missing, the first command that needs it installs borgbackup for you.
2. You don’t have to run a command per folder; backup archives every repo you configured.
3. You don’t have to rerun the command per drive; each repo goes to all of its drives in the same run.
4. You don’t have to remember which folder belongs in which repo, or which repo lives on which drive; you wrote it down once.
5. You don’t have to type a passphrase per repo per drive; it is handed to borg automatically.
6. You don’t have to leave a passphrase in your shell history or in an exported variable; a short-lived helper process passes it straight to borg and exits.
7. You don’t have to assemble the repository path out of the mount point, the drive label and the repo name.
8. You don’t have to abandon the run because one drive is not plugged in; the missing drive is reported and the others still get their backup.
9. You don’t have to spell out --pattern rules on the command line; your excludes live in the config file.
10. You don’t have to know that borg matches those rules against the source path rather than against the name a folder is stored under, which is the difference between an exclude that works and one that silently does nothing.
11. You don’t have to remember borg’s /./ trick, the one that makes a folder land at the top of the archive under its own name no matter where it lives on disk, nor that it silently does nothing before borg 1.4.
12. You don’t have to run borg prune as a second command; retention runs straight after the archive, for every repo that has a keep line.
13. You don’t have to remember that pruning alone frees nothing, and that borg compact is what actually returns the space.
14. You don’t have to interpret borg’s exit codes; a file that changed mid-copy is reported as a warning on a successful run, and a real error is reported as a failure.
15. You don’t have to add up sizes; the run tells you how much it added and how large the repo is now, counting the same data once even when it went to four drives.
16. You don’t have to reconstruct what happened from a screen of output; the run ends in one line that counts repos by outcome and stamps the time.
17. You don’t have to run borg init once per repo per drive; backup init creates all of them.
18. You don’t have to write a new repo’s config block by hand either; backup init <repo> <path>... writes the block, asks for the passphrase, stores it, and creates the repo on every drive.
19. You don’t have to look up an archive’s name to restore it; backup extract <repo> takes the newest.
20. You don’t have to count backwards through a listing to reach an older one; -3 means the third newest on that drive.
21. You don’t have to cd into the right parent directory before restoring; -i puts each folder back where it came from.
22. You don’t have to restore repos one at a time; backup extract with no repo named does every one it can reach.
23. You don’t have to own a config file at all to restore; point extract at a repo path on the drive and borg asks for the passphrase itself.
24. You don’t have to invoke borg once per repo per drive to see what you have; backup check walks every mounted drive.
25. You don’t have to rotate a passphrase on each drive separately and then remember to update your notes; backup pass-change rotates on every drive and rewrites the passphrase file.
26. You don’t have to mv a repo directory on four drives and then edit two files; backup rename does the directories, the config and the passphrase file, or refuses and changes nothing.
27. You don’t have to make sure two backups never overlap; the tool takes a lock and the second one says so and stops.
28. You don’t have to remember to lock down the passphrase file; the tool refuses to read one that anybody else on the machine can open.
29. You don’t have to worry about a debug session printing a passphrase into a log; the tool refuses to run under bash -x at all.
30. You don’t have to notice on your own that a mistyped filter backed up nothing; an archive that comes out empty is called out, and a repo whose allowlist matches everything is refused before anything is written.
31. You don’t have to take a stranger’s word for any of this; every command it runs is in the last section, and you can do the whole job yourself.

Numbers 5, 6, 10, 28 and 29 are the reason the file layout below is worth following exactly. The rest is convenience.

The two files you keep

Everything you configure lives in two files in your home directory. Nothing else is stored anywhere.

The config file, always at ~/.borg-config. It answers three questions: where do drives appear, what folders go into which repo, and which drives does that repo live on. The path is fixed. The tool does not take a --config flag and does not read one from anywhere else, so a config that is not at ~/.borg-config is not in use.

The passphrase file, wherever you point PASSPHRASE_PATH. It holds one line per repo, and nothing else:

set_pass documents 'a long passphrase for the documents repo'
set_pass photos    'a different long passphrase for photos'

Give it any name you like. The tool works out whether it is plain text or GPG-encrypted by reading the file itself, not by looking at the name, so ~/.borg-pass is as good as ~/.borg-pass.gpg. Part 2 uses that: you encrypt this file in place and change nothing else.

Two constraints on the path. It must be readable and writable by you alone, because the tool refuses a file that grants any access to group or other. And it must contain no spaces, quotes or backslashes, because borg runs the command that fetches your passphrase without a shell to unpick them.

In this guide, the file is plain text. Its only protection is its permissions and whatever disk encryption you have. Not a big deal but it is a real risk, and the subject of Part 2, and it is also the only arrangement that can run without you present.

Setup

Five steps, in order.

Step 1 — save the script and run it once

Copy the whole backup script. Open a text editor, paste it in, and save it somewhere you will not lose it. ~/Documents/backup.sh is a good choice, and it is the one the installer’s examples use later.

Pick a folder whose path has no spaces, quotes or backslashes in it. The script re-invokes itself by absolute path to fetch a passphrase, and borg runs that without a shell, so a path like ~/My Documents/backup.sh will stop the tool with an error the first time it needs a passphrase.

Mark it runnable:

chmod +x ~/Documents/backup.sh

Then check that it works:

bash ~/Documents/backup.sh help

You should get a list of commands.

That bash ~/Documents/backup.sh is how you run it until you install it properly, which is a section near the end of this guide. For the rest of this guide, commands are written the short way, as backup init and backup check. Until you have installed it, type bash ~/Documents/backup.sh init and so on instead.

You do not need to install borg. The first command that needs it will run sudo apt-get install borgbackup for you and ask for your password. If you would rather do it yourself, or you are not on a Debian-family system, install it now:

sudo apt install borgbackup

Then check which version you got:

borg --version

It has to be 1.4 or newer, and below 2.0. The tool checks this itself on every command that uses borg and stops if it is outside that range, so you will find out immediately rather than at the worst possible moment. Debian trixie, Devuan Excalibur and their relatives ship 1.4. Debian bookworm and Devuan Daedalus ship 1.2.4, which is too old; bookworm-backports has 1.4 if you cannot move the whole machine.

This matters more than a version requirement usually does, so it is worth one paragraph. Borg 1.4 is the first version that understands the trick this tool uses to put a folder at the top of an archive under its own name. On 1.2 that trick is ignored without a word, every file is filed under its full path instead, and nothing at all goes wrong until the day you restore and get a nest of empty directories with your folder at the bottom. The full explanation is in the reference.

Step 2 — find your drive labels

Plug in every drive you intend to back up to, and wait a few seconds. Then:

ls /media/$USER

You will see one entry per drive:

d1   d2

or, if you never renamed them:

MY-BACKUP   SEAGATE-2TB

Write these down. They go in the config in step 4. Short names are easier to live with; you can rename a drive in your file manager if you want to.

The directory those labels sit in, /media/your-name, is the mount base. If your system puts drives somewhere else, note that path instead.

Step 3 — write the passphrase file

Pick a passphrase for each repo you are about to create. Make them long. A passphrase you cannot remember is fine here, because you are about to write it down; a short one is not.

Create the file:

touch ~/.borg-pass
chmod 600 ~/.borg-pass

Then open it in a text editor and put one line per repo in it:

set_pass documents 'a long passphrase for the documents repo'
set_pass photos    'a different long passphrase for photos'

Single quotes around the passphrase. If the passphrase itself contains a single quote, write it as '\''.

You must write the first line by hand. Later on, backup init <repo> <path>... will prompt you for a new repo’s passphrase and append it here for you, but it will only do that if this file already exists.

The chmod 600 is not decoration. Without it the tool stops and tells you to run it.

Step 4 — write the config

Create ~/.borg-config with the template below, then chmod 600 ~/.borg-config.

# ~/.borg-config   (chmod 600)

# ── topology: where the drives mount, and where things go ──────────────────
MOUNT_BASE="/media/john"                # each drive is MOUNT_BASE/<label>
ALL_DRIVES="d1 d2 d3 d4"                # the drive pool; `backup_drives all` expands to these
REPO_SUBDIR=""                          # optional folder between the drive and the repo ("" = repo at the drive root)
RESTORE_PATH="$HOME/Downloads"          # where `extract` drops restored folders
PASSPHRASE_PATH="$HOME/.borg-pass"      # the one passphrase file; "" lets borg prompt for every repo
SRC_HOME=""                             # in-place restore only: a foreign home prefix to remap onto yours

# ── one block per repo: repo_name opens a block, the lines under it configure it ──
repo_name documents
backup_data "$HOME/Documents" "$HOME/Notes"
backup_drives all
keep daily=7 weekly=4 monthly=-1
exclude "*.tmp" "**/cache/**"
compression auto,zstd
# restore_to "$HOME/Notes" "/mnt/spare"  # optional: send one folder somewhere other than RESTORE_PATH

repo_name photos
backup_data "$HOME/Pictures"
backup_drives d1 d2
keep last=20

Step 5 — create the repos and take the first backup

Plug in the drives and create the repositories:

backup init

That walks every repo in the config, creates it on each of its mounted drives, and skips the ones that already exist. Then:

backup

The first run is the slow one, because nothing has been seen before. Every run after it only stores what changed.

What to edit, and why

Of the six settings at the top, two are yours and four can usually stay as they are.

MOUNT_BASE is the directory your drives appear in, from step 2. Get this wrong and the tool reports that no drives are mounted, because it is looking in the wrong place.

ALL_DRIVES is the list of labels from step 2. It exists so you write your drives down once; a repo that says backup_drives all gets this whole list, so adding a fourth drive is one edit rather than one per repo.

REPO_SUBDIR is for people who keep their repos inside a folder on the drive rather than at its top. If /media/john/d1/borg/photos is where your repo goes, this is borg. Empty means /media/john/d1/photos. It applies to every drive; the tool cannot handle a subfolder on some drives and not others.

RESTORE_PATH is where a restore drops its files, under a directory named after the repo. It defaults to your Downloads folder, which is a good place for it, because a restore should land somewhere you can inspect before you trust it.

PASSPHRASE_PATH points at the file from step 3. Leave it empty and borg will prompt you for every repo on every drive, which defeats most of the list above.

SRC_HOME is for one situation only: restoring in place on a machine where your username changed. If the archives were made under /home/john and you are now /home/jane, set it to /home/john and an in-place restore will redirect to your home. Leave it empty otherwise.

Below the settings, a repo_name line opens a block and every line under it belongs to that repo until the next repo_name. To stop backing something up, comment out or delete its whole block. Comment out the repo_name line alone and the tool will stop and tell you, rather than quietly attach the orphaned lines to the repo above.

The directives inside a repo block

backup_data <path>... lists the folders this repo archives. Each is stored at the top of the archive under its own basename, so two folders in one repo may not share a name. You may leave it out; a repo with drives but no folders is legal, is reported as “awaiting data”, and archives nothing until you give it some.

backup_drives <label>... lists the drives this repo is written to. The single word all expands to ALL_DRIVES.

keep <key=N>... sets retention. The keys are last, hourly, daily, weekly, monthly, yearly. N is a whole number, or -1 to keep that tier forever. A repo with no keep line is never pruned, which is the safe default and also the one that fills a drive. Unset keys on a line that exists default to daily=7 weekly=4 monthly=-1 yearly=-1 and zero for the rest. A nonzero last=N keeps the N newest archives and ignores the other keys.

exclude <glob>... drops matching paths. A bare glob applies anywhere inside any of this repo’s folders, so exclude "*.tmp" catches a .tmp file at the top of a folder and one six levels down alike. * stays inside one path segment and **/ crosses directories, so **/cache/** and *.tmp are both fine and mean what they look like. A glob written as an absolute path, like /home/john/Documents/scratch, names that one place on disk instead.

include_only <glob>... is the opposite: keep only what matches in this repo, drop everything else.

include_only_in <folder> <glob>... does the same for one backup_data folder and leaves the repo’s other folders whole. These are two directives, not one with an optional first argument; writing a folder after include_only is refused with a message naming the replacement. An exclude always wins over an include_only, so a secret you exclude stays out even if an allowlist would have kept it. Before writing an archive for a repo with an allowlist, the tool asks borg what that allowlist actually does and refuses the run if it drops nothing or keeps nothing, because an allowlist that quietly matches everything is a folder going to your drive whole.

restore_to <folder> <parent path> sends one folder somewhere other than the run’s restore directory. The path names the parent the folder arrives in, the same way -i does, so a folder at ~/Notes given /mnt/spare arrives at /mnt/spare/Notes. Give it the parent, not the destination: /mnt/spare/Notes would put the folder at /mnt/spare/Notes/Notes. Folders with no line fall back to the restore directory for that run. It cannot name the folder’s own parent; that is what -i is for.

compression <spec> is passed to borg untouched, for example auto,zstd,10. Leave it out for borg’s default.

archive no takes the repo out of the backup run while leaving init, check, extract and pass-change working on it. Absent means yes.

If you have an old config using backup yes|no or enabled yes|no, those were renamed to archive, and the tool will stop and tell you rather than quietly ignore the line.

Everyday commands

backup                         back up every repo
backup <repo>                  back up one repo
backup check                   deep-verify every repo on every mounted drive
backup list <repo> <drive>     that copy's archives, newest first
backup extract <repo>          restore the newest archive under RESTORE_PATH
backup extract <repo> d1 -2    restore the second newest on that drive
backup extract <repo> -i       restore in place, over the originals
backup extract                 restore every repo it can reach
backup init <repo> <path>...   create a new repo, block and passphrase and all
backup claim <drive>           take ownership of a drive whose files are another user's
backup pass-change <repo>      rotate a repo's passphrase on every drive
backup rename <old> <new>      rename a repo everywhere it exists
backup version                 which version of the script you are running
backup help                    the list

extract counts archives per drive, so -2 and lower need you to name which drive to count on. -1, the newest, is the default and does not.

pass-change and rename need every one of the repo’s drives plugged in, and refuse to start otherwise. Half a rotation across four drives is worse than no rotation, so they check first and change nothing if they cannot finish.

extract <repo> without -i writes the repo’s folders straight into RESTORE_PATH, under the same names the archive holds, so a repo whose folder is ~/Documents lands at RESTORE_PATH/Documents. It asks first if a name it is about to write is already there, and passing -y accepts that.

Bare extract, the whole-suite form, is different: it gives each repo its own RESTORE_PATH/<repo> directory, because restoring every repo into one directory would merge folders that share a name across repos. That directory belongs to the restore, so it asks about anything already in it, colliding or not. Worth knowing why, since it is the weaker of the two: borg overwrites what the archive holds and leaves everything else alone, so restoring into a directory that already has an older restore in it silently mixes two archives, and every file you deleted or renamed since then sits there looking like live data. Restoring a named repo into RESTORE_PATH cannot be protected that way, because that directory is yours and is never empty, so only name collisions are flagged.

extract -i overwrites the originals, asks you to confirm, and fails rather than proceed if there is no terminal to ask at. Each folder goes back to its own original parent, so a repo holding ~/Documents and /srv/notes puts one in ~ and the other in /srv.

Typing backup instead of bash ~/Documents/backup.sh

Everything above works with the long form. This makes it short, and it is also what the schedule section below needs.

Linux looks for commands in the directories listed in your PATH, and ~/.local/bin is the conventional place for your own. Copy the script there under the name you want to type:

mkdir -p ~/.local/bin
cp ~/Documents/backup.sh ~/.local/bin/backup
chmod +x ~/.local/bin/backup

Open a new terminal and try backup help from any directory.

If your shell cannot find it, ~/.local/bin is not on your PATH. Add this line to the end of ~/.bashrc, then open a new terminal:

export PATH="$HOME/.local/bin:$PATH"

The copy in ~/.local/bin is the one that runs. Editing ~/Documents/backup.sh afterwards changes nothing until you copy it over again, which is the usual way people end up debugging a version they are not running.

Once you have a few scripts to deploy this way, doing it by hand gets tedious and easy to get wrong. install_scripts is a small script that does it for a list: you write down each file and the command name it should install as, and it copies them all into ~/.local/bin and tells you if your PATH is not set up. It has its own page.

If you want something shorter still, an alias in ~/.bashrc is the place for it, not a second copy of the script:

alias bkp='backup'

Aliases work at your keyboard only. Cron and desktop launchers do not see them, which is why the schedule below uses the full path.

Running it on a schedule

This is the part that only works here, in Part 1. Cron runs when you are not there, so nothing may stop to ask you anything, and a plain text passphrase file is the only kind that never does. Once you move to the GPG file in Part 2, unattended runs stop being dependable.

Two things cron cannot do for you. It cannot plug a drive in, and it cannot mount one. So a scheduled backup makes sense for a drive that lives in the machine or stays plugged in, and a drive you carry is still a drive you run backup for by hand.

Open your crontab with crontab -e and add:

30 20 * * * /home/john/.local/bin/backup >> /home/john/.borg-backup.log 2>&1

Use the full path. Cron’s PATH is nearly empty and will not find backup on its own.

Redirect both streams to a log. Without the redirect, cron mails you the output, and on a desktop machine with no mail set up that means it goes nowhere and you never learn that four weeks of backups failed. Check the log now and then; the last line of each run says what happened.

You do not need to guard against overlap. The tool takes an exclusive lock at the start of a run, so if a long backup is still going when the next one fires, the second says another operation is in progress and exits.

A drive that is not mounted at 20:30 is not an error that stops the run. It is reported, the repos on the other drives still get archived, and the run finishes with a nonzero exit code so your log shows it was not a clean night.

Starting over on a new machine

This is the whole reason for the exercise, so it is worth reading before you need it.

Your two files, ~/.borg-config and your passphrase file, are inside the backups. Make sure of that now: at least one repo’s backup_data must cover them, either by backing up your home directory or by putting both in a folder that a repo covers. A backup you cannot open is not a backup.

On the new machine:

Copy the backup script over, from a USB stick, a repo you can clone, or a printout if it comes to that. Save it as in step 1; you can install it onto your PATH later, once things are calm.

Plug in one of your drives and look at what is on it:

ls /media/$USER/d1

Restore the repo that holds your two files, by path, with no config anywhere:

cd ~
bash ~/Documents/backup.sh extract /media/john/d1/documents

Any argument containing a slash is treated as a repo path. The tool skips its config entirely, borg asks you for that repo’s passphrase, and the archive is unpacked into the directory you are standing in. This is the step that needs nothing but the drive and the passphrase in your head or on paper, which is why the passphrase for at least one repo has to be recoverable from outside the backups.

You can do this step with borg alone and no script at all; see the last section.

Now put the two files where they belong. The config path is fixed, so it has to be moved rather than pointed at:

cp ~/Documents/.borg-config ~/.borg-config
chmod 600 ~/.borg-config
cp ~/Documents/.borg-pass ~/.borg-pass
chmod 600 ~/.borg-pass

Open ~/.borg-config and check three things against the new machine. MOUNT_BASE still has your username in it if you wrote it out in full, and the new machine’s username may differ. PASSPHRASE_PATH must point at where you just put the passphrase file. ALL_DRIVES should still match your labels, which it will unless you replaced a drive.

From here everything is normal:

backup check
backup extract

If your username changed and you want folders back at their original paths, set SRC_HOME to the old home directory and use backup extract -i.

If the drive’s files belong to the old machine’s user account, borg cannot write its lock and the tool reads with the lock bypassed and warns. backup claim <drive> takes ownership of the repo directories on that drive, and only those.

Doing all of this without the script

Everything above is borg underneath. This section is the whole tool in plain borg commands, so you can check what is being done on your behalf, work on a machine where the script is not installed, or decide you would rather not run it at all.

Assume one repo called documents, holding ~/Documents and ~/Notes, on a drive labelled d1.

Set the repo and the passphrase for the session.

export BORG_REPO=/media/john/d1/documents

Then let borg prompt you for the passphrase each time, which is the safe default. Do not export BORG_PASSPHRASE: an exported variable is readable through /proc by anything running as you, and it lands in your shell history.

Create the repo.

borg init --encryption=repokey

Once per repo per drive. Four drives means four borg init calls with four different BORG_REPO values, all with the same passphrase if you want the script’s arrangement.

Take an archive.

borg create --stats --compression auto,zstd \
    "::{now}" \
    /home/john/./Documents \
    /home/john/./Notes

The /./ in the middle of each path is the whole trick behind the tool’s archive layout. It tells borg to store the folder under everything after the dot, so Documents/... rather than home/john/Documents/.... Without it a restore recreates the entire path underneath wherever you unpack. It needs borg 1.4 or newer; on older borg it is ignored silently and you get the deep path.

Exclude things.

borg create "::{now}" \
    --pattern '- sh:home/john/Documents/**/*.tmp' \
    --pattern '- sh:home/john/Notes/**/*.tmp' \
    /home/john/./Documents /home/john/./Notes

This is the part most worth reading twice, because it is the one that fails silently. Borg matches these patterns against the source path with its leading slash removed, never against the name the folder is stored under. A folder given as /home/john/Documents reaches the matcher as home/john/Documents/..., and /./ changes only what is stored, not what is matched. So - sh:Documents/**/*.tmp matches nothing at all, and the run reports success while archiving every one of those files. **/ matches zero directories as well as many, which is why one pattern per folder covers every depth including the top. Check any pattern before you trust it:

borg create --dry-run --list "::check" --pattern '- sh:...' /home/john/./Documents

Lines beginning with x were excluded, lines beginning with - would have been archived.

Prune, then compact.

borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly -1
borg compact

Pruning marks archives for removal; it does not return the space on its own. borg compact is what does, and it is a separate command by design.

Verify.

borg check --verify-data

Slow, because it reads and re-hashes everything. This is the command that tells you a drive is going bad before you need it.

See what you have.

borg list
borg list "::2026-08-17T20:30:00"

The first lists archives, the second lists the files inside one.

Restore the newest archive.

mkdir -p ~/Downloads/documents
cd ~/Downloads/documents
borg extract "::$(borg list --last 1 --short)"

Borg always extracts into the current directory; there is no destination flag. That is why the script cds for you, and why restoring “in place” is nothing more than cding to the folder’s original parent first:

cd /home/john
borg extract "::$(borg list --last 1 --short)" Documents

The trailing Documents restores that one folder out of the archive. Restore into an empty directory unless you mean to merge; borg overwrites what the archive holds and leaves everything else where it is.

Restore an older one.

borg list --last 5 --short
borg extract "::the-name-you-picked"

Read a drive that belongs to another user, or is read-only.

borg list --bypass-lock
borg extract --bypass-lock "::archive-name"

Borg wants to write a lock file even to read. When it cannot, this skips the lock. Only use it when you are certain nothing else is writing that repo.

Change a passphrase.

borg key change-passphrase

Once per drive. Every copy of a repo has its own key file, so a rotation that only reaches three of four drives leaves the fourth on the old passphrase.

What you are giving up by doing it this way. The commands above are all of it, and they are not hard. What the script adds is that it runs them for every repo and every drive from one invocation, never puts a passphrase on a command line or in an environment variable, gets the pattern anchoring right, does not stop because one drive is missing, refuses a filter that is not doing what you think, and tells you at the end what actually happened. None of that is magic, and none of it is required. The repositories are ordinary borg repositories either way, and any borg on any machine can open them.

When something looks wrong

borg not found; installing borgbackup... Expected on a fresh machine. It will ask for your password and carry on.

borg <version> is too old or borg <version> is too new The installed borg is outside the 1.4-to-below-2.0 range. See step 1. If it says too new, check whether borgbackup-is-borgbackup2 is installed; that package repoints the borg command at the 2.x line.

<file> is reachable by group or other (mode N); run: chmod 600 <file> The config or the passphrase file is readable by someone else. Run the command it gives you.

MOUNT_BASE '<path>' is not a directory MOUNT_BASE is wrong. Check it with ls /media/$USER.

no drives mounted for the repo(s) to back up (looked for: ...) None of the drives those repos want is plugged in and mounted. Note that this is checked before anything reads your passphrase file.

could not read passphrases from <file> The file has no valid set_pass lines, or, once you are on Part 2, it cannot be decrypted right now.

no passphrase set for <repo>; skipped this run That repo has no set_pass line. Add one, or let backup init <repo> store it.

no passphrase file at <file> yet; create it with your first repo's set_pass line init will add lines to an existing file but will not create the file. Do step 3.

passcommand needs paths free of whitespace, quotes, and backslashes The script, or the passphrase file, is saved somewhere with a space or a quote in the path. Move it somewhere plainer and update PASSPHRASE_PATH if that is what moved.

config: ignoring unknown config directive '<name>' A directive the tool does not have, usually a typo or a line left over from an older version, such as backup yes or enabled yes where archive yes is meant. It is only a warning, so the run continues without whatever that line was supposed to do; fix it, because a mistyped backup_data drops a folder from your backup this way.

<repo> has an allowlist but it dropped nothing An include_only or include_only_in line is matching no path borg sees, so the folder would have gone to the drive whole. Check it with borg create --list --dry-run, and remember that borg matches the source path, not the stored name.

archived <repo> on <drive> but the archive is EMPTY The opposite mistake: a pattern matched everything, so nothing was kept.

<dest> already contains: ... A name this restore is about to write is already there. Move it, or pass -y to overwrite it.

another backup operation is in progress A previous run has not finished. Wait for it.

refusing to run under bash -x Tracing would print your passphrase. Debug a specific section instead.

Where to go next

Part 2 encrypts the passphrase file with GPG, so the passphrases are not sitting in readable text on your disk. Read it before you decide, because it costs you unattended backups and it introduces a key you can lose.

Manual borg and gpg use goes further than the section above, one borg command at a time, with the GPG side as well.

borg-super-simple is a much shorter script that reads the same ~/.borg-config and the same passphrase file, and does two things: back up every repo, and extract one repo from one drive into the current directory. It exists for the day the main script will not run, or you want to read the whole thing before trusting it. It ignores keep, restore_to and the allowlist directives, and skips any repo that has an allowlist rather than guess.

The reference is not a guide and does not need reading in order. Look there for the borg versions the tool accepts and why, how a passphrase gets from your file to borg without passing through anything that logs it, how extract decides whether you gave it a repo name or a path, and how to send a repo to a folder on this disk rather than a USB drive.

Leveling up: Part 2

Part 1 left you with working backups and one weak point: the passphrases to every repo are sitting in a readable file in your home directory. This part encrypts that file with GPG.

Read the last two sections before you start. This rung costs you unattended backups, and it hands you a key that you can lose in a way that destroys every backup you own. It is worth doing anyway, for most people, most of the time.

Doing this without the script, near the end, is the same arrangement in plain gpg and borg commands.

What you are actually fixing

In Part 1, ~/.borg-pass is protected by exactly one thing: its file permissions.

That is enough to stop another ordinary user on the same machine. It is not enough for anything else. Anyone with root can read it. An unencrypted backup of your home directory contains it in the clear. A laptop that is stolen while running, or with an unencrypted disk, gives it up. And the file is the whole game: it holds every repo’s passphrase, so it opens every archive on every drive at once.

Encrypting it moves the secret from “protected by a permission bit” to “protected by a key that lives somewhere else.”

It does not protect you from a machine that is compromised while you are using it. When you run a backup, the file is decrypted, in memory, on that machine. Nothing in this part changes that, and no arrangement of files can.

What GPG is, in one paragraph

GPG gives you a key pair. The public key encrypts, the private key decrypts, and only the private key can undo what the public key did. The private key is itself protected by a passphrase, which you type. A background program, the GPG agent, remembers that passphrase for a while so you are not retyping it every few seconds.

So after this part you have two layers. The borg passphrases are encrypted to your GPG key, and your GPG key is protected by its own passphrase. That second passphrase is the one thing you must never lose and must never write into the backups.

What changes in the tool: nothing

This is the good news, and it is why this part is short.

The tool does not decide what kind of passphrase file it has from the filename. It reads the file. A file whose lines start with set_pass is plain text. A file that begins with -----BEGIN PGP MESSAGE-----, or that contains bytes that are not printable text, is treated as GPG-encrypted and decrypted before use.

So you encrypt the file where it already sits. PASSPHRASE_PATH does not change. ~/.borg-config does not change. No command changes. The next backup run notices the file is now ciphertext and asks GPG to open it.

You may rename it to something like ~/.borg-pass.gpg if you find that clearer, and if you do, update PASSPHRASE_PATH to match. The tool does not care either way.

Doing it

Step 1 — install gpg and make a key

sudo apt install gnupg

The backup script installs borg for you but not gpg, so this one is yours to run.

If you do not already have a key:

gpg --full-generate-key

Take the defaults, use a real email address as the identity, and give it a long passphrase. Then find its identifier:

gpg --list-secret-keys --keyid-format=long

Step 2 — get the private key out of the machine, first

Do this before you encrypt anything. The order matters, because after step 3 the key is the only route to your backups.

gpg --export-secret-keys --armor you@example.com > secret-key.asc
gpg --export --armor you@example.com > public-key.asc
gpg --gen-revoke you@example.com > revoke.asc

Export the public key as well as the secret one. It is not sensitive, and you need it to rebuild from paper.

Put both on a USB stick that is not one of your backup drives, and keep it somewhere physically separate. Consider a paper copy as well:

sudo apt install paperkey
gpg --export-secret-keys you@example.com | paperkey --output secret-key-paper.txt

paperkey strips out everything reconstructible from the public key, leaving a much smaller amount to print. The output is a hex dump with line numbers and a checksum on each line, so it can be typed back in and will tell you which line you got wrong. A printed key in a drawer survives things a USB stick does not.

Rebuilding later needs the printout and the public key together:

paperkey --pubring public-key.asc --secrets secret-key-paper.txt --output secret-key.asc

Two things a paper copy does not do. It does not remove the passphrase, so if the secret key was protected by one the rebuilt key is too; paper rescues you from a dead disk, not from a forgotten passphrase. And it is not self-contained, so store public-key.asc with the printout or there is nothing to rebuild against.

Then delete secret-key.asc from your disk once it is safely elsewhere.

Step 3 — encrypt the passphrase file in place

gpg --encrypt --recipient you@example.com --output ~/.borg-pass.new ~/.borg-pass
chmod 600 ~/.borg-pass.new

Check that it actually decrypts before you throw the original away:

gpg --decrypt ~/.borg-pass.new

You should see your set_pass lines. If you do, swap the files in:

mv ~/.borg-pass.new ~/.borg-pass

Confirm the whole thing still works end to end:

backup check

If that verifies your repos, the tool decrypted the file and used the passphrases.

Step 4 — get rid of the plain text copy

The original is gone from that path, but it may still exist elsewhere. Check your editor’s backup files, your Downloads folder, and any place you copied it while following Part 1.

More importantly, the plaintext version is already inside your existing archives. Every backup you took during Part 1 contains it. That is not fixable by deleting a file; those archives are what they are. If it matters to you, take the view that the old archives are as sensitive as the old file was, and let retention age them out.

What it costs you

Cron becomes unreliable. An unattended run needs the GPG agent already primed with your key passphrase, and an agent’s cache expires. A backup that fires at 20:30 on a machine you have not touched since morning will find no primed agent and fail, or hang waiting for a prompt that nobody will answer. This is why the cron section lives in Part 1 and not here. If you want both, keep the plain text file and rely on full-disk encryption instead; that is a legitimate choice, not a lesser one.

Every borg call reopens the file. This surprises people, so it is worth being exact. The tool never holds your passphrases in its own long-running process. It hands borg a command to fetch a passphrase, and borg runs that command fresh for each borg operation that needs the key. One repo on one drive involves an archive, then a prune if you set retention, then a verification pass, and each of those is a separate borg process that fetches the passphrase again. Add one more at the start of every run, when the tool works out which repos have a stored passphrase at all.

With the agent holding your key passphrase, you will not notice; each decryption is silent. With a hardware token that requires a physical touch per operation, you will notice a great deal, because it is one touch per decryption rather than one per run.

One more thing to install on a new machine. Recovery now needs gpg present and your private key imported before the passphrase file is of any use.

The danger, stated plainly

If you lose the GPG private key, or forget its passphrase, the passphrase file cannot be opened. If the passphrase file cannot be opened, the borg repos cannot be opened. If the borg repos cannot be opened, every backup you have is a directory of random-looking data, permanently.

There is no recovery, no reset, and nobody to appeal to. This is not a flaw in the design; it is what encryption is.

Three rules follow from that, and none of them is optional.

The private key must exist outside the backups. If your only copy of the key is inside an encrypted repo, you have built a lock whose key is inside the box. The repo needs a passphrase, the passphrase is encrypted to the key, the key is in the repo. Keep the export and the paper copy on separate media, in a separate place. Note that neither covers a forgotten passphrase: both rebuild a key that is still protected by it.

At least one repo’s passphrase must be recoverable without any of this. Written down, in a safe, in a password manager on your phone, on paper in an envelope, whatever suits you. On a dead machine you restore that one repo by pointing borg at it directly and typing the passphrase, which gets your config and passphrase file back, and everything else follows. Without that, a lost GPG key is total.

Test the recovery, once, on purpose. Import your key onto a different machine or a fresh user account, copy a drive’s repo over, and restore something. An untested recovery is a belief, not a backup.

Rotating a passphrase after this

backup pass-change <repo> still works exactly as before.

It changes the repo’s passphrase on every drive, then rewrites the passphrase file. When that file is encrypted it re-encrypts to the same recipients the file already had, checks that the result decrypts before replacing anything, and leaves the original in place if that check fails. So a failed re-encryption cannot destroy a good file.

If the tool cannot work out who the file was encrypted to, it stops and tells you to re-encrypt by hand rather than guess.

Doing this without the script

The script’s only role here is reading the encrypted file for you. Everything in this part is gpg, and you can keep the same arrangement without it.

Keep the passphrases wherever you like and hand one to borg by hand. The simplest version has no file at all: let borg prompt.

export BORG_REPO=/media/john/d1/documents
borg create "::{now}" /home/john/./Documents

Borg asks, you type, nothing is stored. This is the most secure arrangement there is, and the least convenient.

Or keep one encrypted file and feed it to borg without the script. Borg takes a command to run whenever it needs a passphrase:

export BORG_REPO=/media/john/d1/documents
export BORG_PASSCOMMAND="gpg --quiet --decrypt /home/john/.borg-documents.gpg"
borg create "::{now}" /home/john/./Documents

Where that file holds nothing but the one repo’s passphrase, encrypted:

printf '%s' 'the passphrase' | gpg --encrypt --recipient you@example.com \
    --output ~/.borg-documents.gpg
chmod 600 ~/.borg-documents.gpg

One file per repo, rather than one file holding them all. That is more files but a smaller blast radius, and it removes the parsing step entirely. Note the two rules the script enforces and you now have to keep yourself: the file must be readable by you alone, and the path in BORG_PASSCOMMAND must have no spaces, quotes or backslashes, because borg splits that string without a shell.

Never do this instead.

export BORG_PASSPHRASE='the passphrase'

It reaches borg, and it also reaches your shell history, anything that dumps your environment, and every process running as you through /proc. BORG_PASSCOMMAND exists so you do not have to.

Rotate a passphrase by hand.

BORG_REPO=/media/john/d1/documents borg key change-passphrase
BORG_REPO=/media/john/d2/documents borg key change-passphrase

One call per drive, because every copy has its own key file, and then re-encrypt whatever file holds it. A rotation that reaches three drives out of four leaves the fourth on the old passphrase with nothing to tell you.

Re-encrypt the combined file after an edit.

gpg --decrypt ~/.borg-pass > /dev/shm/pass.tmp
# edit /dev/shm/pass.tmp
gpg --encrypt --recipient you@example.com --output ~/.borg-pass.new /dev/shm/pass.tmp
gpg --decrypt ~/.borg-pass.new                 # confirm it opens before you swap
mv ~/.borg-pass.new ~/.borg-pass
shred -u /dev/shm/pass.tmp

Decrypt to /dev/shm, which is memory rather than disk, and check the new file decrypts before replacing the old one. Those two habits are what the script’s rotation does for you, and they are the two that people skip.

When something looks wrong

could not read passphrases from <file> (for a .gpg: can gpg decrypt it now ...) The agent is not primed, the key is not present, or the file is not what you think it is. Test it directly with gpg --decrypt <file>.

refusing to archive: <file> is present but unreadable, so every repo would be silently skipped The right behaviour, and worth understanding. Rather than treat an undecryptable file as “no passphrases stored” and quietly back up nothing, the run stops.

cannot read the GPG recipients of <file> to re-encrypt it A rotation got as far as the drives but could not rewrite the passphrase file. The drives are on the new passphrase and the file is not, so fix the file by hand before the next run, using the message the tool printed.

Prompts that never appear under cron. See the cost section above. This is expected, not a bug.

Where to go next

Manual borg and gpg use drops the script entirely and shows you every command underneath, borg and GPG alike, in more depth than the section above.

borg-super-simple is a much shorter script that reads the same config and the same passphrase file, GPG or plain, and decides which from the file’s own bytes rather than its name. It backs up every repo and extracts one repo from one drive, and nothing else. Keep it for the day the main script will not run.

The reference covers the borg versions the tool accepts, how a passphrase reaches borg without touching a command line, and the parts these three guides do not need.

Borg-simple script

To devs and reviewers: This script is about 3129 lines long and the result of vibe-coding with Opus 4.7, 4.8, 5, a few times with Fable 5, and about 200 iterations (for real). Anthropic helped but also exasperates; I can’t wait to upgrade my rig and install Qwen or something. Anyway I’ve tested all the user-facing functions, and they work. I’m not a coder, so please review this script if you can.

If this is too long, then please consider reviewing the core part of the script which contains only the core functions for easy review. Hopefully, there’s nothing malicious. I’ve been using the simple script for my daily backup-workflow.

#!/bin/bash
# borg-simple, v192
#
# Encrypted, versioned borg backups to USB drives. You keep one config file,
# ~/.borg-config (what to back up, to which drives, how long to keep it); the
# script reads it on every run, and init and rename are the only commands that
# write to it. Passphrases live in one file you point PASSPHRASE_PATH at, named
# whatever you like (chmod 600), gpg-encrypted or plaintext: the script reads
# which from the file's own bytes. With no path set, borg prompts per repo.
# Run it with no arguments to back up every repo, with a repo name to back up one,
# and with --help for the command list. The name you type is whatever this file was
# installed as, and the help and error messages show that name back to you.
# When a drive's files belong to another user, 'claim' takes ownership of that
# drive's mountpoint and its repo directories after showing what it will take
# and asking; it is the only place this tool uses sudo on a backup path.
# If borg itself is missing, the first command that needs it installs
# borgbackup with apt, so a fresh machine needs only this script. borg 1.4 or
# newer and below 2.0 is required, and anything outside that is refused.
# Setup, the config format, and cron use are in the guide.

set -euo pipefail

## ─── config location ─────────────────────────────────────────────────
# Every path this tool resolves hangs off $HOME, and under set -u an unset HOME
# aborts on the next line with a raw bash error, before help or version can run.
# die() is not defined yet here, so this says it the long way.
if [[ -z "${HOME:-}" ]]; then
    printf '%s\n' "HOME is not set, so the config path cannot be resolved; run from a login shell, or set HOME" >&2
    exit 1
fi
readonly CONFIG_FILE="$HOME/.bc"

# The single source of truth for the CLI surface. The dispatch case lists these,
# _reject_reserved_name forbids them as repo names by reading this list, and
# _selfcheck asserts each has a handler, a dispatch branch and a help-banner line.
readonly SUBCOMMANDS=(init extract list check claim pass-change rename version)

## ─── output ──────────────────────────────────────────────────────────
say()  { printf '%s\n' "$*"; }
warn() { printf '%s\n' "$*" >&2; }
die()  { printf '%s\n' "$*" >&2; exit 1; }
die_usage() { printf '%s\n' "$*" >&2; exit 2; }
need() { command -v "$1" >/dev/null 2>&1 || die "$1 not found; install it first"; }

## ─── baked defaults: the settings you edit ───────────────────────────
# Built-in settings, so the script runs with no config file. Your ~/.borg-config
# overrides any line it sets; lines it omits keep the value here.
MOUNT_BASE="/media/$(id -un)"   # drives mount under here, so drive d1 is MOUNT_BASE/d1; override for non-udisks layouts
RESTORE_PATH="$HOME/Downloads"  # extract restores a named repo into here; the whole-suite form uses RESTORE_PATH/<repo>
# SRC_HOME remaps an in-place restore of another machine's backup: folders go
# back to the parent named on each backup_data line, so when those sit under a
# foreign home (say /home/john) set this to it and they land under yours instead.
# "" = off. It cannot help a repo whose backup_data is a home directory itself:
# the archive stores that folder under its own name, so nothing is left to remap.
SRC_HOME=""
REPO_SUBDIR=""                  # optional folder between the drive and the repo; "" = repo at the drive root
PASSPHRASE_PATH=""              # one passphrase file, any name, gpg-encrypted or plaintext (detected from its content); "" = borg asks
ALL_DRIVES="d1 d2 d3 d4"        # the drive labels; a repo using `backup_drives all` gets all of these

## ─── internal state ──────────────────────────────────────────────────
# Repo blocks. CURRENT_REPO tracks the open block while sourcing the config.
CURRENT_REPO=""
declare -a REPOS=()           # repo names, in config order
declare -A REPO_SEEN=()       # name -> 1
declare -A REPO_SRC=()        # name -> newline-joined absolute source folders
declare -A REPO_DRIVES=()     # name -> space-joined drive labels
declare -A REPO_EXCLUDES=()   # name -> newline-joined exclude globs (emitted as '- sh:' patterns)
declare -A REPO_INCLUDES=()   # name -> newline-joined include_only globs
declare -A REPO_ARCHIVE=()    # name -> 1 archived (default), 0 if `archive no`
declare -A REPO_COMPRESSION=() # name -> borg compression spec; empty = borg default (lz4)
declare -A REPO_FINCLUDE_GLOBS=() # "name + newline + abs-folder" -> newline-joined folder-scoped include_only globs
declare -A REPO_HAS_FINCLUDE=()   # name -> 1 if the repo has any folder-scoped include_only line
declare -A REPO_RESTORE_TO=()     # "name + newline + abs-folder" -> absolute parent the folder restores into
declare -a PASS_SKIPPED=()    # repos skipped this run for a missing passphrase
declare -A PASS=()            # name -> passphrase; set only in the _emit-pass/_pass-names children and the writers, never on a routine run
declare -A PASS_NAMES=()      # name -> 1 for repos with a set_pass entry; learned via the names-only child, values never enter the parent
_PASS_NAMES_LOADED=0
_PASS_NAMES_UNREADABLE=0      # PASSPHRASE_PATH is present but the names child failed: no exec, no decrypt, or no set_pass lines

# Holds '--bypass-lock' for the current read when the repo directory is not
# writable, so borg can read a repo whose files another user owns, or one on
# read-only media, without creating a lock file. Empty for a writable repo, so a
# genuine stale lock on your own repo still fails loudly and tells you to
# break-lock. Set per read by _set_read_lock; never on a write path.
declare -a RO_LOCKOPT=()
declare -A _RO_WARNED=()

# Deduplicated bytes added across one archive run, counted once per repo (its
# first archived drive) so mirror drives don't multiply it.
RUN_DEDUP_BYTES=0
declare -A RUN_REPO_DEDUP_COUNTED=()

# Repos whose allowlist has already been checked against borg's matcher this
# run. The answer is a property of the config and the source tree, not of the
# drive being written, so one check per repo covers all its copies.
declare -A RUN_REPO_FILTER_CHECKED=()

# Every drive-level reason a repo came out short, in order. Each was already
# warned, but warn goes to stderr while the summary goes to stdout, so a cron job
# keeping only stdout got the verdict and none of the causes. Replayed on stdout
# just above the summary, so the run is answerable from one stream.
declare -a RUN_ISSUES=()

# What the repo loop last printed, so a blank line can separate a run's per-repo
# blocks: "block", "skip", or "" before anything. Consecutive skips stay grouped,
# since a blank line around every one-liner is noisier than the run it separates.
_RUN_LAST=""

# Retention. A repo is pruned only if it has a keep line; these defaults fill in
# any key such a line leaves unset.
readonly KEEP_LAST_DEFAULT=0
readonly KEEP_HOURLY_DEFAULT=0
readonly KEEP_DAILY_DEFAULT=7
readonly KEEP_WEEKLY_DEFAULT=4
readonly KEEP_MONTHLY_DEFAULT=-1
readonly KEEP_YEARLY_DEFAULT=-1
declare -A SEEN_KEEP=()
declare -A KEEP_LAST=()
declare -A KEEP_HOURLY=()
declare -A KEEP_DAILY=()
declare -A KEEP_WEEKLY=()
declare -A KEEP_MONTHLY=()
declare -A KEEP_YEARLY=()

## ─── helpers ─────────────────────────────────────────────────────────
_note_issue() { RUN_ISSUES+=("$1"); }

# Comma-join the arguments into "a, b, c"; nothing for an empty list.
_csv() {
    (( $# > 0 )) || return 0
    local acc="$1" x; shift
    for x in "$@"; do acc+=", $x"; done
    printf '%s' "$acc"
}

# One wording for an absent drive, everywhere. The path is load-bearing: a drive
# that is plugged in and mounted still reports "not mounted" whenever MOUNT_BASE
# points somewhere else, and only the path tells that apart from an absent drive.
_not_mounted() { printf '%s is not mounted at %s' "$1" "$MOUNT_BASE/$1"; }

# borg is the one dependency a fresh machine will not have, and a fresh machine
# is exactly where this tool has to work, so install it rather than refuse.
ensure_borg() {
    if command -v borg >/dev/null 2>&1; then
        _require_borg_version
        return 0
    fi
    say "borg not found; installing borgbackup..."
    sudo apt-get update                || die "apt-get update failed"
    sudo apt-get install -y borgbackup || die "could not install borgbackup"
    command -v borg >/dev/null 2>&1 || die "borgbackup installed but borg still not on PATH"
    _require_borg_version
}

# Every source folder is handed to borg as <parent>/./<folder> so it lands at the
# archive root under its own basename. That prefix strip is borg's slashdot hack,
# added in the 1.4 series; 1.2.x ignores the /./ silently and stores every item
# under its full path instead. Nothing fails at backup time, so the damage shows
# up only at restore, which is the worst failure direction this tool has.
# The upper bound matters as much: this tool parses deduplicated_size and
# unique_csize out of `borg create --json` (both gone in 2.0), calls a bare
# `borg prune` (2.0 wants an archive glob) and uses the ::archive syntax 2.0
# changed, and Debian trixie ships a borgbackup-is-borgbackup2 package that
# repoints the `borg` command at 2.x. Refuse both ends, and say which was hit.
_require_borg_version() {
    local out ver major minor
    out=$(borg --version 2>/dev/null | head -n1) || die "could not run 'borg --version'"
    [[ -n "$out" ]] || die "'borg --version' printed nothing; cannot confirm borg is 1.4 or newer"
    ver="${out##* }"
    major="${ver%%.*}"
    minor="${ver#*.}"; minor="${minor%%.*}"
    case "$major$minor" in
        ''|*[!0-9]*) die "could not read a version number from 'borg --version' (got: '$out'); this tool needs borg 1.4 or newer, below 2.0" ;;
    esac
    (( major == 1 && minor >= 4 )) && return 0
    (( major >= 2 )) \
        && die "borg $ver is too new; this tool is written against borg 1.x and would fail partway through on 2.x (changed create --json fields, prune arguments, and repo::archive syntax). Install the borgbackup 1.4 package, or check whether borgbackup-is-borgbackup2 has repointed the borg command"
    die "borg $ver is too old; this tool needs borg 1.4 or newer, which is where borg learned to strip the /./ prefix this tool's archive layout depends on. Upgrade borgbackup, or read these repos with borg directly"
}

# Absolute, symlink-resolved path to this script, for the BORG_PASSCOMMAND that
# re-invokes it. A bare $0 (PATH-resolved, e.g. cron) goes through command -v.
_resolve_self() {
    local src="$0" pth
    case "$src" in
        */*) pth="$src" ;;
        *)   pth=$(command -v -- "$src" 2>/dev/null) || pth="$src" ;;
    esac
    readlink -f -- "$pth" 2>/dev/null || printf '%s\n' "$pth"
}
SELF=$(_resolve_self)
readonly SELF

# The name this was invoked as, used by every message that tells you what to
# type. install_scripts deliberately lets a file be stored under one name and
# installed as another, so a message hardcoding either one is wrong for somebody.
CMD="${0##*/}"
readonly CMD

# One backup operation at a time; the fd-9 lock auto-releases on process exit.
# The lock lives beside the config in $HOME, not under $XDG_RUNTIME_DIR or /tmp:
# a $HOME path is per-user by construction, so no other local user can pre-create
# the name as a symlink and have this exec 9> truncate a file you own.
_acquire_lock() {
    local lock="${CONFIG_FILE}.lock"
    exec 9>"$lock"
    flock -n 9 || die "another backup operation is in progress"
}

# The mount-level reasons a write or a chown cannot work, said once for both
# callers so the two wordings can never drift apart. $1 is the mountpoint, $2 its
# fstype, $3 its options, $4 the indent every line carries, $5 the consequence
# clause the caller wants on the read-only line. Returns 1 for a read-only mount,
# 2 for a filesystem that carries no ownership, 0 when neither applies.
_fs_blocker() {
    local mnt="$1" fstype="$2" opts="$3" pad="${4:-}" ro_because="${5:-}"
    case ",$opts," in
        *,ro,*)
            warn "$pad$mnt is mounted read-only, $ro_because"
            warn "$pad  remount it:  sudo mount -o remount,rw $mnt"
            return 1 ;;
    esac
    case "$fstype" in
        vfat|exfat|ntfs|ntfs3|fuseblk)
            warn "$pad$mnt is $fstype, which stores no ownership; the mount's uid= option decides it, so chown cannot help"
            warn "$pad  remount as you:  sudo mount -o remount,uid=\"\$(id -u)\",gid=\"\$(id -g)\" $mnt"
            return 2 ;;
    esac
    return 0
}

# The remedy for an unwritable directory, named for the cause rather than offered
# as a menu. Three things produce the same symptom and only one is fixed by
# taking ownership: a read-only mount refuses the chown too, and vfat/exfat/ntfs
# carry no on-disk ownership at all (the mount's uid= option decides it). Ask the
# mount which case this is and print only that remedy. $1 is the directory, $2
# the drive label when the caller knows it. Advice only; nothing is run here.
_unwritable_hint() {
    local dir="${1:-}" label="${2:-}" fstype="" opts="" mnt=""
    if [[ -n "$dir" ]] && command -v findmnt >/dev/null 2>&1; then
        read -r fstype opts mnt < <(findmnt -no FSTYPE,OPTIONS,TARGET -T "$dir" 2>/dev/null) || true
    fi
    # A chown or remount aimed at / would be aimed at the whole machine, which
    # means the drive is not mounted where the caller thought. Say that instead
    # of printing a command that must not be run.
    if [[ "$mnt" == "/" ]]; then
        warn "  $dir is on your root filesystem, so that drive is not mounted; mount it and re-run"
        return 0
    fi
    if [[ -z "$mnt" ]]; then
        local show="${dir:-<the repo directory>}"
        warn "  inspect:  findmnt -no FSTYPE,OPTIONS -T $show   and   ls -ld $show"
        return 0
    fi
    local blocker=0
    _fs_blocker "$mnt" "$fstype" "$opts" "  " "so nothing can be written to it" || blocker=$?
    case "$blocker" in
        1) warn "  if that fails, the drive may have a physical write-lock switch, or a damaged filesystem"
           return 0 ;;
        2) return 0 ;;
    esac
    warn "  the files under $mnt belong to another user, and borg needs the repo directory itself writable"
    if [[ -n "$label" ]]; then
        warn "  take it:  $CMD claim $label"
    else
        warn "  take it:  $CMD claim <drive>   (or: sudo chown -R \"\$(id -un):\$(id -gn)\" $mnt)"
    fi
    return 0
}

# The first path under a borg repo that cannot be written, printed on stdout, or
# nothing when every one of them can. The top directory is only where borg puts
# its lock file: the archive goes into data/, and config and nonce are rewritten
# on every run, so a repo whose directory is yours and whose contents are not
# passes a top-directory test and then fails several steps into borg with a
# Python traceback. That is what a partly-claimed drive leaves behind. Only paths
# that already exist are tested, so a directory that is not a borg repo at all is
# not accused of a permission problem. Returns 0 either way; read the output.
_first_unwritable() {
    local repo="$1" p
    [[ -w "$repo" ]] || { printf '%s\n' "$repo"; return 0; }
    for p in data config nonce; do
        if [[ -e "$repo/$p" && ! -w "$repo/$p" ]]; then
            printf '%s\n' "$repo/$p"
            return 0
        fi
    done
    return 0
}

# The read mirror of _first_unwritable, same contract and same reason. One extra
# point: --bypass-lock cannot rescue an unreadable repo, since the lock is not
# the obstacle there, which is why _set_read_lock distinguishes the two.
_first_unreadable() {
    local repo="$1" p
    [[ -r "$repo" && -x "$repo" ]] || { printf '%s\n' "$repo"; return 0; }
    for p in config data; do
        [[ -e "$repo/$p" ]] || continue
        if [[ ! -r "$repo/$p" ]] || { [[ -d "$repo/$p" ]] && [[ ! -x "$repo/$p" ]]; }; then
            printf '%s\n' "$repo/$p"
            return 0
        fi
    done
    return 0
}

# Decide the lock mode for a read of repo dir $1: bypass borg's lock when the
# directory is not writable, warning once per directory so a scan across several
# drives does not repeat itself. Read-only callers only (list, extract, check);
# a write path must hold the lock, never this.
_set_read_lock() {
    local repo="$1" label="${2:-}"
    if [[ -w "$repo" ]]; then
        RO_LOCKOPT=()
        return 0
    fi
    RO_LOCKOPT=(--bypass-lock)
    [[ -z "${_RO_WARNED[$repo]:-}" ]] || return 0
    _RO_WARNED["$repo"]=1
    # Say which of the two it is. Bypassing the lock reads an unwritable repo
    # fine; it does nothing for one that cannot be read, and claiming otherwise
    # promises a workaround that borg then fails on a few steps later.
    local blocked; blocked=$(_first_unreadable "$repo")
    if [[ -n "$blocked" ]]; then
        warn "$blocked cannot be read, and no lock option changes that"
    else
        warn "$repo is not writable; reading with borg's lock bypassed (assumes nothing else is writing it)"
    fi
    _unwritable_hint "$repo" "$label"
}

# Offer to take ownership of a drive when there is a terminal, and print the
# remedy when there is not. Returns 0 only when a claim actually ran and
# succeeded, so the caller can re-test rather than assume.
_offer_claim() {
    local drive="$1" dir="$2"
    if [[ -t 0 ]] && _claim_drive "$drive"; then return 0; fi
    _unwritable_hint "$dir" "$drive"
    return 1
}

# Refuse an all-or-nothing operation unless every listed directory is writable,
# offering to claim the drives that are not. $1 names the operation; the rest are
# "<drive>TAB<dir>" pairs. Dies rather than proceeding on a subset: a rename or a
# rotation that lands on some drives and not others leaves the config and the
# repos out of step, which then has to be repaired by hand.
_require_writable() {
    local op="$1"; shift
    local pair drive dir
    local -a blocked=() still=()
    for pair in "$@"; do
        dir="${pair#*$'\t'}"
        [[ -z "$(_first_unwritable "$dir")" ]] || blocked+=("$pair")
    done
    (( ${#blocked[@]} > 0 )) || return 0
    warn "$op needs these to be writable, and they are not:"
    for pair in "${blocked[@]}"; do warn "  ${pair#*$'\t'}"; done
    for pair in "${blocked[@]}"; do
        drive="${pair%%$'\t'*}"; dir="${pair#*$'\t'}"
        _offer_claim "$drive" "$dir" || true
        [[ -z "$(_first_unwritable "$dir")" ]] || still+=("$drive")
    done
    (( ${#still[@]} == 0 )) \
        || die "$op refused: still not writable on ${still[*]}; nothing was changed"
    return 0
}

## ─── config: validators and path resolution ──────────────────────────
_validate_segment() {
    local kind="$1" value="$2"
    case "$value" in
        ''|.|..)           die "config: invalid $kind '$value'" ;;
        -*)                die "config: $kind may not start with '-': '$value'" ;;
        *[!A-Za-z0-9._-]*) die "config: $kind has an unsafe character (allowed: letters, digits, . _ -): '$value'" ;;
    esac
    return 0
}

# Repo names share the command line with subcommands, so a repo named like one
# would be unreachable: `<cmd> <name>` would dispatch to the subcommand instead
# of archiving the repo. Reserve the visible dispatch words, the help flags, and
# the hidden dispatch words. Drive labels are never dispatched, so this guards
# repo names only, at the three places a repo_name line is born: the config
# directive, `init` creating a repo, and `rename`.
_reject_reserved_name() {
    local name="$1" w
    for w in "${SUBCOMMANDS[@]}" help -h --help _emit-pass _pass-names _selfcheck; do
        [[ "$name" == "$w" ]] \
            && die "repo name '$name' is reserved for the '$name' command; pick another name"
    done
    return 0
}

_require_absolute_path() {
    local kind="$1" path="$2"
    [[ "$path" == /* ]] || die "config: $kind path must be absolute (start with /): '$path'"
}

# One validator for a backup_drives value ('all' or a list of labels), used by
# the config directive and by init's --drives, so a bad drive is rejected the
# same way wherever it is written. $1 prefixes every message with its context.
_validate_drives_list() {
    local what="$1" drives="$2" d seen=""
    if [[ "$drives" == all ]]; then
        [[ "$ALL_DRIVES" =~ [^[:space:]] ]] \
            || die "$what: 'all' needs ALL_DRIVES set in $CONFIG_FILE, but it is empty"
        what="$what: ALL_DRIVES"
        drives="$ALL_DRIVES"
    fi
    # shellcheck disable=SC2086  # space-separated label list; intentional split
    for d in $drives; do
        [[ "$d" != all ]] || die "$what: 'all' is reserved for the ALL_DRIVES pool; a drive may not be named all"
        _validate_segment "drive" "$d"
        case " $seen " in *" $d "*) die "$what lists drive '$d' twice" ;; esac
        seen+="$d "
    done
    return 0
}

# Resolve a config path: a leading ~/ or a bare relative path is taken under
# $HOME, an absolute path passes through, and empty stays empty.
_resolve_home_path() {
    local p="$1" out
    [[ -n "$p" ]] || return 0
    # A shellcheck directive is only honoured in front of a complete command, so
    # it sits above the whole case rather than above the one branch it is for:
    # the '~/' pattern below is a literal, expanded by hand on the right-hand
    # side, not a failed tilde expansion. Do not move it onto the branch. There
    # it is a parse error that stops shellcheck and leaves the rest unchecked.
    # shellcheck disable=SC2088
    case "$p" in
        /*)    out="$p" ;;
        '~')   out="$HOME" ;;
        '~/'*) out="$HOME/${p#'~/'}" ;;
        *)     out="$HOME/$p" ;;
    esac
    # Drop a trailing slash so a folder written with or without one resolves
    # identically; never reduce the root "/" to empty.
    [[ "$out" == / ]] || out="${out%/}"
    printf '%s\n' "$out"
}

## ─── config: directives ──────────────────────────────────────────────
# repo_name opens a block; the directives under it attach to this repo until
# the next repo_name. To remove a repo, comment or delete its whole block.
repo_name() {
    (( $# == 1 )) || die "config: repo_name needs exactly one name; got: $*"
    _validate_segment "repo name" "$1"
    _reject_reserved_name "$1"
    [[ -z "${REPO_SEEN[$1]+x}" ]] || die "config: repo $1 declared more than once"
    REPO_SEEN["$1"]=1
    REPOS+=("$1")
    CURRENT_REPO="$1"
}

# A directive with no open repo means a repo_name was commented out but its
# lines were left behind; fail loudly rather than silently misattach them.
_require_open_repo() {
    [[ -n "$CURRENT_REPO" ]] \
        || die "config: $1 with no repo_name above it (did you comment out a repo_name but leave its lines?)"
}

# Folders to back up for the current repo. Each is stored at the archive root
# under its own basename, so two folders in one repo must not share a basename.
backup_data() {
    _require_open_repo "backup_data"
    (( $# >= 1 )) || die "config: backup_data needs at least one folder path"
    local p abs base
    for p in "$@"; do
        abs=$(_resolve_home_path "$p")
        _require_absolute_path "backup_data folder" "$abs"
        base=$(basename "$abs")
        _validate_segment "backup_data folder name (basename)" "$base"
        case $'\n'"${REPO_SRC[$CURRENT_REPO]:-}" in
            *$'\n'*"/$base"$'\n'*)
                die "config: repo $CURRENT_REPO has two folders named '$base'; they would collide at the archive root" ;;
        esac
        REPO_SRC["$CURRENT_REPO"]+="$abs"$'\n'
    done
}

backup_drives() {
    _require_open_repo "backup_drives"
    (( $# >= 1 )) || die "config: backup_drives needs at least one drive (or 'all')"
    if [[ "$1" == all ]]; then
        (( $# == 1 )) || die "config: backup_drives all takes no other arguments (got: backup_drives $*)"
        _validate_drives_list "config: backup_drives all" all
        REPO_DRIVES["$CURRENT_REPO"]="$ALL_DRIVES"
        return 0
    fi
    _validate_drives_list "config: backup_drives" "$*"
    REPO_DRIVES["$CURRENT_REPO"]="$*"
}

# Retention for the repo. Unset keys inherit: daily=7 weekly=4 monthly=-1
# yearly=-1, rest 0 (-1 keeps a tier forever, 0 none). 'last' nonzero keeps the
# N most recent and ignores the bucket keys.
keep() {
    _require_open_repo "keep"
    (( $# >= 1 )) || die "config: keep needs at least one key=N"
    [[ -z "${SEEN_KEEP[$CURRENT_REPO]+x}" ]] || die "config: keep set more than once for repo $CURRENT_REPO"
    SEEN_KEEP["$CURRENT_REPO"]=1
    local pair key value saw_last=0 saw_bucket=0
    for pair in "$@"; do
        [[ "$pair" == *=* ]] || die "config: keep '$pair' must be key=N"
        key="${pair%%=*}"; value="${pair#*=}"
        [[ "$value" =~ ^(-1|[0-9]+)$ ]] \
            || die "config: keep $key value '$value' must be -1 or a non-negative integer"
        case "$key" in
            last)     KEEP_LAST["$CURRENT_REPO"]="$value";     saw_last=1   ;;
            hourly)   KEEP_HOURLY["$CURRENT_REPO"]="$value";   saw_bucket=1 ;;
            daily)    KEEP_DAILY["$CURRENT_REPO"]="$value";    saw_bucket=1 ;;
            weekly)   KEEP_WEEKLY["$CURRENT_REPO"]="$value";   saw_bucket=1 ;;
            monthly)  KEEP_MONTHLY["$CURRENT_REPO"]="$value";  saw_bucket=1 ;;
            yearly)   KEEP_YEARLY["$CURRENT_REPO"]="$value";   saw_bucket=1 ;;
            *) die "config: keep unknown key '$key' (use last|hourly|daily|weekly|monthly|yearly)" ;;
        esac
    done
    if (( saw_last && saw_bucket && ${KEEP_LAST[$CURRENT_REPO]:-0} != 0 )); then
        warn "config: keep for $CURRENT_REPO: last=${KEEP_LAST[$CURRENT_REPO]} active; bucket keys on this line ignored"
    fi
}

# Denylist for the repo: each pattern becomes a '- sh:' borg pattern (shell-style:
# * stops at /, **/ crosses). Excludes are emitted before any allowlist, so an
# explicit exclude always wins; they compose with both kinds of include_only.
exclude() {
    _require_open_repo "exclude"
    (( $# >= 1 )) || die "config: exclude needs at least one pattern"
    local pat
    for pat in "$@"; do
        [[ -n "$pat" ]] || die "config: exclude empty pattern"
        REPO_EXCLUDES["$CURRENT_REPO"]+="$pat"$'\n'
    done
}

# Repo-wide allowlist: each glob becomes a '+ sh:' include and a global '- sh:**'
# drops the rest. Excludes are emitted before any allowlist, so an explicit
# exclude always wins. Mutually exclusive with include_only_in over one repo
# (checked in _require_repos_valid); both compose with exclude.
#
# The two guards below detect a scoped allowlist written with this directive.
# Neither of them picks a scope, since the directive's name does that; they exist
# because getting the scope wrong silently shortens a backup and is found only at
# restore. So a first token that names one of this repo's folders, or that holds
# no wildcard and therefore cannot be a glob, is refused rather than taken as a
# repo-wide pattern.
include_only() {
    _require_open_repo "include_only"
    (( $# >= 1 )) || die "config: include_only needs at least one glob"
    local first; first=$(_resolve_home_path "$1")
    case $'\n'"${REPO_SRC[$CURRENT_REPO]:-}" in
        *$'\n'"$first"$'\n'*)
            die "config: include_only is repo-wide; to scope an allowlist to one folder write: include_only_in $1 <glob>..." ;;
    esac
    [[ "$1" == *'*'* || "$1" == *'?'* || "$1" == *'['* ]] \
        || die "config: include_only '$1' contains no wildcard, so it is not a glob; for one folder's allowlist write: include_only_in $1 <glob>..."
    local g
    for g in "$@"; do
        [[ -n "$g" ]] || die "config: include_only empty glob"
        REPO_INCLUDES["$CURRENT_REPO"]+="$g"$'\n'
    done
}

# Allowlist scoped to one backup_data folder: its globs are anchored to the
# folder's own name and a '- sh:<base>/**' drops only that folder's rest, leaving
# the repo's other folders whole. One line per folder; mutually exclusive with
# the repo-wide include_only over the same repo.
include_only_in() {
    _require_open_repo "include_only_in"
    (( $# >= 2 )) || die "config: include_only_in needs a backup_data folder and at least one glob"
    local folder="$1"; shift
    local first; first=$(_resolve_home_path "$folder")
    case $'\n'"${REPO_SRC[$CURRENT_REPO]:-}" in
        *$'\n'"$first"$'\n'*) ;;
        *) die "config: include_only_in folder '$folder' is not a backup_data folder of repo $CURRENT_REPO (list it in backup_data above this line)" ;;
    esac
    local key="$CURRENT_REPO"$'\n'"$first"
    [[ -z "${REPO_FINCLUDE_GLOBS[$key]+x}" ]] || die "config: two include_only_in lines for folder '$folder' in repo $CURRENT_REPO"
    REPO_HAS_FINCLUDE["$CURRENT_REPO"]=1
    local g globs=""
    for g in "$@"; do
        [[ -n "$g" ]] || die "config: include_only_in empty glob for '$folder'"
        globs+="$g"$'\n'
    done
    REPO_FINCLUDE_GLOBS["$key"]="$globs"
}

# Where one folder lands on a restore that is not in place. The path names the
# parent the folder arrives in, not the folder itself, so it reads the same way
# -i does: a folder at ~/Documents/g1 given a parent of /mnt/spare arrives at
# /mnt/spare/g1. Folders with no line fall back to the run's restore directory;
# a line present but wrong dies here, at config load, rather than at restore time
# when you are least able to deal with it.
restore_to() {
    _require_open_repo "restore_to"
    (( $# == 2 )) || die "config: restore_to needs a backup_data folder and one absolute parent path"
    local folder="$1" target="$2"
    local first; first=$(_resolve_home_path "$folder")
    case $'\n'"${REPO_SRC[$CURRENT_REPO]:-}" in
        *$'\n'"$first"$'\n'*) ;;
        *) die "config: restore_to folder '$folder' is not a backup_data folder of repo $CURRENT_REPO (list it in backup_data above this line)" ;;
    esac
    local key="$CURRENT_REPO"$'\n'"$first"
    [[ -z "${REPO_RESTORE_TO[$key]+x}" ]] || die "config: two restore_to lines for folder '$folder' in repo $CURRENT_REPO"
    [[ -n "$target" ]] || die "config: restore_to for '$folder' has an empty path"
    local abs; abs=$(_resolve_home_path "$target")
    _require_absolute_path "restore_to path for '$folder'" "$abs"
    # Restoring a folder on top of itself is what -i is for, and doing it from
    # this directive would overwrite live data on a command that promises not to.
    [[ "$abs" != "$(dirname "$first")" ]] \
        || die "config: restore_to for '$folder' names its own parent, which would overwrite the live folder; use '$CMD extract $CURRENT_REPO -i' for that"
    REPO_RESTORE_TO["$key"]="$abs"
}

# archive no excludes the repo from the archive run and drops its passphrase
# need; init, check, extract, claim and pass-change ignore it. Absent means yes.
archive() {
    _require_open_repo "archive"
    (( $# == 1 )) || die "config: archive needs one value: yes or no"
    [[ -z "${REPO_ARCHIVE[$CURRENT_REPO]+x}" ]] || die "config: archive set more than once for repo $CURRENT_REPO"
    case "$1" in
        yes) REPO_ARCHIVE["$CURRENT_REPO"]=1 ;;
        no)  REPO_ARCHIVE["$CURRENT_REPO"]=0 ;;
        *)   die "config: archive must be 'yes' or 'no', got '$1'" ;;
    esac
}

# Passed to borg create untouched; borg validates the spec. Omit for lz4 default.
compression() {
    _require_open_repo "compression"
    (( $# == 1 )) || die "config: compression needs one spec, e.g. auto,zstd,10"
    [[ -z "${REPO_COMPRESSION[$CURRENT_REPO]+x}" ]] || die "config: compression set more than once for repo $CURRENT_REPO"
    [[ -n "$1" ]] || die "config: compression empty spec"
    REPO_COMPRESSION["$CURRENT_REPO"]="$1"
}

# True unless the repo was turned off with `archive no`.
_repo_archives() { [[ "${REPO_ARCHIVE[$1]:-1}" != 0 ]]; }

# Lives in the passphrase file, keyed by repo name.
set_pass() {
    (( $# == 2 )) || die "pass: set_pass needs: <repo> '<passphrase>'; got $# args"
    [[ -z "${PASS[$1]+x}" ]] || die "pass: passphrase for $1 set more than once in $PASSPHRASE_PATH"
    PASS["$1"]="$2"
}

# A file holding secrets, or sourced as shell, must be owner-only. A loose mode
# is tightened here rather than refused: chmod only ever narrows, so there is
# nothing destructive to gate. Two cases are refused instead, both for security:
# chmod follows symlinks, so a symlink here would tighten whatever it points at,
# and a file owned by someone else is not one to quietly adjust or trust. The
# chmod does not undo an exposure that already happened, so when the old mode
# granted read to group or other, say so: rotate what was in it.
require_private() {
    local f="$1" mode
    [[ -e "$f" ]] || die "$f not found; create it (chmod 600) before running"
    [[ -r "$f" ]] || die "$f exists but is not readable"
    mode=$(stat -c '%a' "$f") || die "cannot stat $f"
    if (( (8#$mode & 8#77) == 0 )); then return 0; fi
    [[ ! -L "$f" ]] \
        || die "$f is reachable by group or other (mode $mode) and is a symlink; this will not chmod through it, so fix the file it points at"
    [[ "$(stat -c '%u' "$f")" == "$(id -u)" ]] \
        || die "$f is reachable by group or other (mode $mode) and is not owned by you; its owner must run: chmod 600 $f"
    chmod 600 "$f" || die "$f is reachable by group or other (mode $mode) and chmod 600 failed; fix it by hand"
    warn "$f was mode $mode, reachable by group or other; tightened it to 600"
    if (( (8#$mode & 8#44) != 0 )); then
        warn "it was readable by them until now, so treat anything in it as already read; for the passphrase file that means rotating: $CMD pass-change <repo>"
    fi
    return 0
}

# Source the config with universal validation. Subcommands add their own.
parse_config() {
    local cfg="$CONFIG_FILE"
    # An absent config is fine: the baked defaults stand, and a repo named on the
    # command line can still be adopted from a mounted drive by _adopt_drive_repo.
    # When present, an unrecognised directive (a sibling tool's sync_*
    # line, a stale passphrase_file line) is tolerated rather than fatal, but
    # never silent: a mistyped backup_data would otherwise drop a folder from
    # the backup with no output at all, found only at restore time.
    if [[ -e "$cfg" ]]; then
        require_private "$cfg"
        # shellcheck disable=SC2317  # bash calls this indirectly while sourcing the config; not dead code
        command_not_found_handle() { warn "config: ignoring unknown directive '$1'"; return 0; }
        # shellcheck source=/dev/null
        source "$cfg" || die "failed to load $cfg (check it for a shell syntax error, e.g. an unbalanced quote)"
        unset -f command_not_found_handle
        # set_pass belongs in the passphrase file. A set_pass line here would land
        # in PASS (which nothing reads on a routine run) while the repo is then
        # skipped as "no passphrase set"; catch the misplaced line loudly.
        (( ${#PASS[@]} == 0 )) || die "config: set_pass belongs in the passphrase file, not $CONFIG_FILE; move the line there"
    fi

    MOUNT_BASE=$(_resolve_home_path "$MOUNT_BASE")
    [[ -n "$MOUNT_BASE" ]] || die "MOUNT_BASE is empty; set it in $CONFIG_FILE or keep the baked default"
    [[ -d "$MOUNT_BASE" ]] || die "MOUNT_BASE '$MOUNT_BASE' is not a directory"
    [[ -z "$REPO_SUBDIR" ]] || _validate_segment "REPO_SUBDIR" "$REPO_SUBDIR"
    RESTORE_PATH=$(_resolve_home_path "$RESTORE_PATH")
    # SRC_HOME is a literal foreign-home prefix for in-place remap, deliberately
    # not run through _resolve_home_path (that resolves against the running
    # user's $HOME, which would defeat its purpose).
    if [[ -n "$SRC_HOME" ]]; then
        _require_absolute_path "SRC_HOME" "$SRC_HOME"
        SRC_HOME="${SRC_HOME%/}"
    fi
    PASSPHRASE_PATH=$(_resolve_home_path "$PASSPHRASE_PATH")
    # The file's name carries no meaning: _pass_source_kind reads the format out
    # of the file itself. The one real constraint is that a set path be
    # space-free, because borg splits BORG_PASSCOMMAND itself.
    case "$PASSPHRASE_PATH" in
        '') ;;
        *[[:space:]]*) die "PASSPHRASE_PATH must be space-free for the passcommand (got: '$PASSPHRASE_PATH')" ;;
    esac
    # A path set to a file that is not there reads as 'none' everywhere
    # downstream, which is the same state as PASSPHRASE_PATH="" and produces the
    # same silent outcome. Those are very different intentions, so name the gap.
    if [[ -n "$PASSPHRASE_PATH" && ! -e "$PASSPHRASE_PATH" ]]; then
        warn "PASSPHRASE_PATH names $PASSPHRASE_PATH, but no file is there; every repo will be treated as having no stored passphrase"
    fi
}

# A repo with no backup_data is allowed (it archives nothing yet, reported as
# awaiting data); backup_drives and the filtering rules are required.
_require_repos_valid() {
    (( ${#REPOS[@]} > 0 )) || die "no repos configured; add repo_name blocks to $CONFIG_FILE"
    local r
    for r in "${REPOS[@]}"; do
        [[ -n "${REPO_DRIVES[$r]:-}" ]] || die "config: repo $r has no backup_drives line"
        # Two allowlist scopes over one repo are ambiguous; excludes compose with
        # either, so they are not part of this test.
        if [[ -n "${REPO_INCLUDES[$r]:-}" && -n "${REPO_HAS_FINCLUDE[$r]:-}" ]]; then
            die "config: repo $r has both include_only and include_only_in; use one repo-wide allowlist, or scope every allowlist to a folder"
        fi
    done
}

## ─── repo paths and drive membership ─────────────────────────────────
_repo_declared() { [[ -n "${REPO_SEEN[$1]+x}" ]]; }

# The repo directory on a given drive, and the directory holding it.
_repo_path()   { printf '%s\n' "$MOUNT_BASE/$1${REPO_SUBDIR:+/$REPO_SUBDIR}/$2"; }
_repo_parent() { printf '%s\n' "$MOUNT_BASE/$1${REPO_SUBDIR:+/$REPO_SUBDIR}"; }

# What init must be able to write to create a repo on a drive: the REPO_SUBDIR
# when it is already there, otherwise the mountpoint init would create it under.
_init_target_dir() {
    local p; p=$(_repo_parent "$1")
    if [[ -d "$p" ]]; then printf '%s\n' "$p"; else printf '%s\n' "$MOUNT_BASE/$1"; fi
}

# True when a drive label is one of a repo's configured drives. extract, list,
# check and claim all take a drive, and a typo in it should say so rather than
# fall through to "no mounted drive had it", which points at the wrong problem.
_repo_has_drive() {
    local repo="$1" want="$2" d
    # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
    for d in ${REPO_DRIVES[$repo]}; do
        [[ "$d" == "$want" ]] && return 0
    done
    return 1
}

# A repo's configured drives, comma-joined, for those error messages.
_repo_drives_csv() {
    # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
    _csv ${REPO_DRIVES[$1]}
}

# One wording for a name the config does not declare, and one for a drive the
# named repo does not use, so every command that takes them says the same thing.
_require_declared_repo() {
    _repo_declared "$1" || die "$1 is not a configured repo (check $CONFIG_FILE)"
}
_require_repo_drive() {
    _repo_has_drive "$1" "$2" \
        || die "$2 is not a configured drive for $1 (it has: $(_repo_drives_csv "$1"))"
}

# Last resort for a bare repo name no config declares. MOUNT_BASE, REPO_SUBDIR
# and ALL_DRIVES are all baked, so a repo directory is fully derivable from the
# name alone: look for one on each mounted drive and, when any turn up, declare
# the repo exactly as a config block would. Nothing is written and no config is
# touched; an in-place restore still refuses, correctly, because there are no
# backup_data lines saying where the folders came from. Returns 1 when nothing
# is found, leaving the caller's existing errors in place.
_adopt_drive_repo() {
    local name="$1" d found=()
    # Cheap shape test rather than _validate_segment, which dies: a name that
    # cannot be a path segment should fall through to the caller's message, not
    # abort with a config-parse error for a word that came from the command line.
    case "$name" in
        ''|.|..|-*|*[!A-Za-z0-9._-]*) return 1 ;;
    esac
    [[ "$ALL_DRIVES" =~ [^[:space:]] ]] || return 1
    # shellcheck disable=SC2086  # space-separated label list; intentional split
    for d in $ALL_DRIVES; do
        mountpoint -q "$MOUNT_BASE/$d" || continue
        [[ -d "$(_repo_path "$d" "$name")" ]] || continue
        found+=("$d")
    done
    (( ${#found[@]} > 0 )) || return 1
    repo_name "$name"
    backup_drives "${found[@]}"
    say "'$name' is not in $CONFIG_FILE; using the repo directory found on: $(_csv "${found[@]}")"
    return 0
}

## ─── passphrase file: read, write, load ──────────────────────────────
# Backend read from the file's content, not its name, so the file may be called
# anything. The test is for what actually distinguishes the two: a plaintext file
# carries set_pass lines in the clear and ciphertext does not. Neither the suffix
# nor the first byte decides it, and a UTF-8 BOM is allowed for explicitly or a
# one-repo file whose only set_pass line is line 1 would read as ciphertext. A
# file merely missing its set_pass lines, or empty, stays plaintext so the caller
# reports that rather than a decrypt failure for a file that was never encrypted.
_pass_source_kind() {
    [[ -n "$PASSPHRASE_PATH" && -e "$PASSPHRASE_PATH" ]] || { printf 'none\n'; return; }
    local bom=$'\xef\xbb\xbf'
    if LC_ALL=C grep -qaE "^(${bom})?[[:space:]]*set_pass[[:space:]]" -- "$PASSPHRASE_PATH"; then
        printf 'plain\n'; return
    fi
    if LC_ALL=C grep -qa '^-----BEGIN PGP MESSAGE-----' -- "$PASSPHRASE_PATH" \
       || LC_ALL=C grep -qa '[^[:print:][:space:]]' -- "$PASSPHRASE_PATH"; then
        printf 'gpg\n'
    else
        printf 'plain\n'
    fi
}

# Emit the passphrase file's cleartext on stdout. For .gpg this decrypts
# through a pipe, so cleartext never lands on disk.
_pass_cleartext() {
    case "$(_pass_source_kind)" in
        plain) require_private "$PASSPHRASE_PATH"; cat -- "$PASSPHRASE_PATH" ;;
        gpg)   need gpg; require_private "$PASSPHRASE_PATH"; gpg --quiet --decrypt "$PASSPHRASE_PATH" || die "could not decrypt $PASSPHRASE_PATH" ;;
        none)  die "no passphrase file at $PASSPHRASE_PATH to read" ;;
    esac
}

# Replace the passphrase file atomically (same-dir tempfile). For .gpg, re-encrypt
# to the file's existing recipients and verify the result decrypts before the
# swap, so a bad re-encrypt cannot replace a good file; cleartext is piped, never
# on disk. The EXIT trap covers a die from anywhere below, which would otherwise
# leave the tempfile behind; no caller of this holds an EXIT trap of its own.
_pass_write() {
    local tmp
    tmp=$(mktemp "${PASSPHRASE_PATH}.XXXXXX") || die "could not create tempfile for $PASSPHRASE_PATH"
    trap 'rm -f "$tmp"' EXIT
    trap 'rm -f "$tmp"; exit 130' INT TERM
    case "$(_pass_source_kind)" in
        plain)
            cat > "$tmp" || die "could not write $tmp"
            ;;
        gpg)
            need gpg
            local kids=() kid
            while IFS= read -r kid; do
                [[ -n "$kid" ]] && kids+=( -r "$kid" )
            done < <(gpg --list-packets "$PASSPHRASE_PATH" 2>/dev/null \
                       | sed -n 's/.*keyid \([0-9A-Fa-f]\{8,\}\).*/\1/p')
            (( ${#kids[@]} > 0 )) \
                || die "cannot read the GPG recipients of $PASSPHRASE_PATH to re-encrypt it; re-encrypt the file to your key by hand"
            gpg --quiet --yes --encrypt "${kids[@]}" --output "$tmp" \
                || die "re-encryption of $PASSPHRASE_PATH failed"
            gpg --quiet --decrypt "$tmp" >/dev/null 2>&1 \
                || die "re-encrypted $PASSPHRASE_PATH failed to decrypt; original kept"
            ;;
        none)
            die "no passphrase file at $PASSPHRASE_PATH to update; create it first"
            ;;
    esac
    # chmod after the write: gpg --output can recreate the file under the umask,
    # so set owner-only just before the swap, not before the encrypt.
    chmod 600 "$tmp" || die "could not make $PASSPHRASE_PATH owner-only"
    mv "$tmp" "$PASSPHRASE_PATH" || die "could not write $PASSPHRASE_PATH"
    trap - EXIT INT TERM
}

# The combined passphrase file is never sourced into this long-running process
# on a routine run. borg gets a BORG_PASSCOMMAND that re-invokes the hidden
# _emit-pass child to print one repo's passphrase, and a separate _pass-names
# child reports which repos have an entry, so this process learns the names
# without ever holding a value. The writers below still read the file at write
# time; those are the deliberate cleartext moments.

# Print a BORG_PASSCOMMAND for repo $1, or nothing (borg prompts) when that repo
# has no stored entry. borg runs it with no shell and no guaranteed PATH, so this
# invokes a fixed bash on the script by absolute path, and all three tokens must
# be free of whitespace and quotes because borg splits the string itself.
_repo_passcommand() {
    local repo="$1"
    _repo_has_pass "$repo" || return 0
    case "$BASH$SELF$PASSPHRASE_PATH" in
        *[[:space:]\'\"\\]*) die "passcommand needs paths free of whitespace, quotes, and backslashes (borg splits BORG_PASSCOMMAND with shlex), but one of BASH='$BASH', SELF='$SELF', PASSPHRASE_PATH='$PASSPHRASE_PATH' contains one" ;;
    esac
    printf '%s %s _emit-pass %s %s\n' "$BASH" "$SELF" "$PASSPHRASE_PATH" "$repo"
}

# Hidden value child: decrypt the file, evaluate its set_pass lines, print the
# one repo's passphrase; exit non-zero (borg fails loudly) if absent. The only
# place a passphrase is materialised on a routine run, in a short-lived process
# handing it straight to borg; tracing is refused in main.
_emit_pass() {
    (( $# == 2 )) || { printf '_emit-pass needs <file> <repo>\n' >&2; exit 2; }
    local file="$1" repo="$2" ct
    PASSPHRASE_PATH="$file"
    [[ "$(_pass_source_kind)" != none ]] || { printf 'no passphrase file at %s\n' "$file" >&2; exit 1; }
    ct=$(_pass_cleartext) || exit 1
    eval "$ct"; unset ct
    [[ -n "${PASS[$repo]:-}" ]] || { printf 'no passphrase for %s in %s\n' "$repo" "$file" >&2; exit 1; }
    printf '%s' "${PASS[$repo]}"
}

# Hidden names child: decrypt the file and print the repo names that have a
# set_pass entry, one per line. Names only; values stay in _emit-pass.
_pass_names() {
    (( $# == 1 )) || { printf '_pass-names needs <file>\n' >&2; exit 2; }
    local file="$1" ct k
    PASSPHRASE_PATH="$file"
    [[ "$(_pass_source_kind)" != none ]] || exit 0
    ct=$(_pass_cleartext) || exit 1
    eval "$ct"; unset ct
    for k in "${!PASS[@]}"; do printf '%s\n' "$k"; done
}

# Populate PASS_NAMES once via the names child, so this process learns which
# repos have a passphrase without holding a value. Tolerant: an absent or
# undecryptable file leaves PASS_NAMES empty and sets _PASS_NAMES_UNREADABLE.
# Call this from the parent before anything reaches _repo_passcommand: that runs
# inside a command substitution, and a first load from there would set these
# globals in a subshell that is then discarded, so the file would be re-read once
# per repo per drive and the unreadable flag would never reach the parent.
_load_pass_names() {
    (( _PASS_NAMES_LOADED )) && return 0
    _PASS_NAMES_LOADED=1
    [[ "$(_pass_source_kind)" != none ]] || return 0
    local out rc=0
    out=$("$BASH" "$SELF" _pass-names "$PASSPHRASE_PATH") || rc=$?
    if (( rc != 0 )); then
        _PASS_NAMES_UNREADABLE=1
        warn "could not read passphrases from $PASSPHRASE_PATH (for a .gpg: can gpg decrypt it now, i.e. is gpg-agent primed and the key present? otherwise: does the file contain 'set_pass <repo> ...' lines?)"
        return 0
    fi
    local n
    while IFS= read -r n; do
        [[ -n "$n" ]] && PASS_NAMES["$n"]=1
    done <<< "$out"
    # The child ran and printed nothing: the file parsed but defined no repo.
    # That is what a line written as an assignment produces; only
    # `set_pass <repo> <pass>`, a directive with two arguments, reaches this tool.
    if (( ${#PASS_NAMES[@]} == 0 )); then
        _PASS_NAMES_UNREADABLE=1
        warn "$PASSPHRASE_PATH defines no repo passphrase; every line must read: set_pass <repo> '<passphrase>' (single quotes: the file is read by bash, so a \$ or a backtick in a double-quoted passphrase is expanded and the passphrase is silently altered)"
    fi
    # Explicit, and load-bearing. The loop above ends in a conditional, so an $out
    # with no usable line would otherwise make this function return 1, which
    # aborts every bare caller under set -e with nothing printed. Do not drop it.
    return 0
}

# True when repo $1 has a stored passphrase (names only; no value read).
_repo_has_pass() { _load_pass_names; [[ -n "${PASS_NAMES[$1]:-}" ]]; }

# Require a stored passphrase for each named repo (presence by name only).
_require_pass() {
    local r missing=()
    for r in "$@"; do _repo_has_pass "$r" || missing+=("$r"); done
    (( ${#missing[@]} == 0 )) \
        || die "no passphrase set for ${missing[*]} (add set_pass lines to $PASSPHRASE_PATH)"
}

# One wording for every "borg does the asking" notice. $1 is why nothing is
# stored, $2 what the prompts are for, and $3 is 'twice' when the operation lists
# before it extracts, so the second prompt is expected rather than a retry.
# Always stderr: it is an advisory about how the run will behave, not a result.
_prompt_notice() {
    local why="$1" what="$2" how="${3:-once}"
    local when="once"
    [[ "$how" != twice ]] || when="twice: once to list, once to extract"
    warn "$why; borg will ask for $what, $when"
}

# True when no passphrase file is configured and there is a terminal, i.e. the
# documented "borg asks" fallback is live for this run. Every command honours it
# the same way, by emitting no BORG_PASSCOMMAND. Tty-gated so an unattended run
# fails closed rather than stalling on a prompt nobody can answer. The prompt is
# per borg invocation, not per repo, so a multi-repo pool is a lot of typing;
# that is the price of keeping the passphrase off disk and out of this process.
_pass_prompt_mode() { [[ -z "$PASSPHRASE_PATH" && -t 0 ]]; }

# Archive preflight for the repos passed in: load the names, then mark those that
# are archived and hold folders but have no stored passphrase as skipped (warned).
# In prompt mode nothing is skipped and no names are loaded; borg does the asking.
# The permission check happens in the parent, before the names child reads the
# file, so a mode notice lands on the run's own output rather than in a process
# whose stdout is being captured.
_archive_pass_preflight() {
    if _pass_prompt_mode; then
        _prompt_notice "no passphrase file configured" "each repo on each drive"
        return 0
    fi
    [[ "$(_pass_source_kind)" == none ]] || require_private "$PASSPHRASE_PATH"
    _load_pass_names
    (( _PASS_NAMES_UNREADABLE )) && die "refusing to archive: $PASSPHRASE_PATH is present but unreadable, so every repo would be silently skipped; fix the cause in the warning above and re-run"
    local label missing=()
    for label in "$@"; do
        _repo_archives "$label" || continue            # archive no
        [[ -n "${REPO_SRC[$label]:-}" ]] || continue   # no folders yet (awaiting data)
        _repo_has_pass "$label" || missing+=("$label")
    done
    if (( ${#missing[@]} > 0 )); then
        if [[ -z "$PASSPHRASE_PATH" ]]; then
            warn "no passphrase file configured and no terminal to prompt; skipped this run: ${missing[*]}"
        else
            warn "no passphrase set for ${missing[*]}; skipped this run (add set_pass lines to $PASSPHRASE_PATH)"
        fi
        PASS_SKIPPED=("${missing[@]}")
    fi
}

# Replace the set_pass value for $1 with $2 in the passphrase file. Matching on
# the line rather than splitting it keeps the old value out of any temp file, and
# printf -v keeps the new one out of a command substitution.
_update_pass_value() {
    local label="$1" new_pass="$2" quoted
    printf -v quoted '%q' "$new_pass"
    local ct; ct=$(_pass_cleartext) || die "could not read $PASSPHRASE_PATH; file left untouched"
    local found=0 line out=""
    while IFS= read -r line || [[ -n "$line" ]]; do
        if [[ "$line" =~ ^[[:space:]]*set_pass[[:space:]]+([^[:space:]]+)[[:space:]] ]] \
            && [[ "${BASH_REMATCH[1]}" == "$label" ]]; then
            out+="set_pass ${label} ${quoted}"$'\n'
            found=1
        else
            out+="$line"$'\n'
        fi
    done <<< "$ct"
    (( found )) \
        || die "no set_pass line for $label in $PASSPHRASE_PATH; rotation finished on drives but the file was not updated"
    printf '%s' "$out" | _pass_write
}

# Rewrite the label on the set_pass line, preserving the value and its spacing.
_update_pass_label() {
    local old="$1" new="$2"
    local ct; ct=$(_pass_cleartext) || die "could not read $PASSPHRASE_PATH; file left untouched"
    local found=0 line out=""
    while IFS= read -r line || [[ -n "$line" ]]; do
        if [[ "$line" =~ ^([[:space:]]*set_pass[[:space:]]+)([^[:space:]]+)([[:space:]]+.+)$ ]] \
            && [[ "${BASH_REMATCH[2]}" == "$old" ]]; then
            out+="${BASH_REMATCH[1]}${new}${BASH_REMATCH[3]}"$'\n'
            found=1
        else
            out+="$line"$'\n'
        fi
    done <<< "$ct"
    (( found )) \
        || { warn "no set_pass line for $old in $PASSPHRASE_PATH; nothing to relabel there"; return 0; }
    printf '%s' "$out" | _pass_write
}

## ─── config mutation: append a repo block, rename a repo ─────────────
# init appends a repo block; rename rewrites a repo_name line. Both write
# atomically through a same-dir tempfile, preserving the file's mode, and touch
# only their target line, so a sibling tool's lines, comments and blanks are left
# in place. Everything else in the config is edited by hand.

# Write new whole-file content over the config atomically, preserving its mode.
_write_config() {
    local content="$1" tmp
    tmp=$(mktemp "${CONFIG_FILE}.XXXXXX") || die "cannot create a temp file beside $CONFIG_FILE"
    trap 'rm -f "$tmp"' EXIT
    trap 'rm -f "$tmp"; exit 130' INT TERM
    printf '%s' "$content" > "$tmp" || die "failed to write $tmp"
    if [[ -e "$CONFIG_FILE" ]]; then
        chmod --reference="$CONFIG_FILE" "$tmp" || die "failed to set mode on $tmp"
    else
        chmod 600 "$tmp" || die "failed to set mode on $tmp"
    fi
    mv -f "$tmp" "$CONFIG_FILE" || die "failed to update $CONFIG_FILE"
    trap - EXIT INT TERM
}

# Rewrite the repo_name block header in the config, preserving indentation and
# any trailing comment. Dies if the line is not found, so the directories are
# never silently left out of step with the config.
_update_repo_name() {
    local old="$1" new="$2"
    local found=0 line out=""
    while IFS= read -r line || [[ -n "$line" ]]; do
        if [[ "$line" =~ ^([[:space:]]*repo_name[[:space:]]+)([^[:space:]#]+)([[:space:]].*)?$ ]] \
            && [[ "${BASH_REMATCH[2]}" == "$old" ]]; then
            out+="${BASH_REMATCH[1]}${new}${BASH_REMATCH[3]:-}"$'\n'
            found=1
        else
            out+="$line"$'\n'
        fi
    done < "$CONFIG_FILE"
    (( found )) \
        || die "no 'repo_name $old' line in $CONFIG_FILE; drives renamed but the config was not updated"
    _write_config "$out"
}

# Append a new repo block: a backup_data line per resolved path, a backup_drives
# line, and the optional directives as commented templates. No archive line is
# templated: a new repo is archive-on by default. Paths are validated like
# backup_data before any write.
_append_repo_block() {
    local repo="$1" drives="$2"; shift 2
    local p abs base seen="" dq
    local -a body=("repo_name $repo")
    for p in "$@"; do
        abs=$(_resolve_home_path "$p")
        _require_absolute_path "backup_data folder" "$abs"
        base=$(basename "$abs")
        _validate_segment "backup_data folder name (basename)" "$base"
        case $'\n'"$seen" in
            *$'\n'"$base"$'\n'*) die "two folders named '$base' would collide at the archive root: $abs" ;;
        esac
        seen+="$base"$'\n'
        printf -v dq 'backup_data %q' "$abs"
        body+=("$dq")
    done
    body+=("backup_drives $drives")
    body+=("# keep daily=30 weekly=12 monthly=-1")
    body+=("# exclude '**/cache/**' '**/node_modules/**'     # denylist (shell-style globs)")
    body+=("# include_only '**/wanted'                          # repo-wide allowlist")
    body+=("# include_only_in \"\$HOME/sub\" '**/wanted'          # allowlist one folder, others whole")
    body+=("# compression auto,zstd,10")
    local existing block content
    existing=$(cat "$CONFIG_FILE")
    printf -v block '%s\n' "${body[@]}"
    # One blank line between the prior content and the new block.
    printf -v content '%s\n\n%s' "$existing" "$block"
    _write_config "$content"
}

# Prompt for repo $1's passphrase twice and store it through the atomic,
# recipient-preserving writer, then record the name so the caller can use it at
# once. Fails closed with no terminal so a non-interactive run never hangs, and
# requires the passphrase file to exist (the first passphrase is hand-created).
_store_pass() {
    local repo="$1"
    [[ "$(_pass_source_kind)" != none ]] \
        || die "no passphrase file at $PASSPHRASE_PATH yet; create it with your first repo's set_pass line, then init adds the rest"
    [[ -t 0 ]] \
        || die "no passphrase stored for $repo and no terminal to prompt; add a 'set_pass $repo' line to $PASSPHRASE_PATH, then re-run"
    local current repo_re
    current=$(_pass_cleartext) || die "could not read $PASSPHRASE_PATH"
    repo_re="${repo//./\\.}"
    if printf '%s\n' "$current" | grep -qE "^[[:space:]]*set_pass[[:space:]]+${repo_re}([[:space:]]|\$)"; then
        return 0                                     # already stored
    fi
    local p1 p2 line
    read -rsp "passphrase for $repo: " p1; printf '\n' >&2
    read -rsp "confirm: " p2;             printf '\n' >&2
    [[ -n "$p1" ]]       || die "empty passphrase; nothing stored"
    [[ "$p1" == "$p2" ]] || die "passphrases did not match; nothing stored"
    printf -v line 'set_pass %s %q' "$repo" "$p1"
    printf '%s\n%s\n' "$current" "$line" | _pass_write
    unset p1 p2
    PASS_NAMES["$repo"]=1
    say "stored a passphrase for $repo in $PASSPHRASE_PATH"
}

## ─── claim: take ownership of one drive's repos ──────────────────────
# The one place this tool uses sudo on a backup path. Deliberately not a chown -R
# of the whole mountpoint: a drive can hold someone else's files, and taking the
# lot would take those too. What borg needs is the mountpoint itself (its lock
# file is created beside the repo), any REPO_SUBDIR, and each repo directory in
# full, so that is exactly what is taken.
#
# Refuses rather than escalating wherever a chown cannot work or cannot be
# trusted: a path that is not the drive's own mountpoint, so a chown can never
# walk the root filesystem when a drive is not mounted where you thought; a
# read-only mount; a filesystem that carries no ownership. Requires a terminal,
# so a cron run can never sit on a sudo password prompt. Returns 1 on any refusal
# so the caller re-tests rather than assumes. It takes the drive and nothing
# else: naming a subset of the repos leaves the partly-claimed drive borg fails
# several steps into.
_claim_drive() {
    local drive="$1"
    local mount="$MOUNT_BASE/$drive"
    command -v findmnt >/dev/null 2>&1 || { warn "findmnt not found; cannot confirm what $mount is before changing ownership"; return 1; }
    command -v sudo    >/dev/null 2>&1 || { warn "sudo not found; take ownership of $mount by hand"; return 1; }
    mountpoint -q "$mount" || { warn "$(_not_mounted "$drive")"; return 1; }

    local fstype="" opts="" mnt=""
    read -r fstype opts mnt < <(findmnt -no FSTYPE,OPTIONS,TARGET -T "$mount" 2>/dev/null) || true
    [[ "$mnt" == "$mount" ]] \
        || { warn "$mount is not itself a mountpoint (it resolves to ${mnt:-an unknown mount}); refusing to change ownership there"; return 1; }
    _fs_blocker "$mount" "$fstype" "$opts" "" "so ownership cannot be changed" || return 1
    [[ -t 0 ]] || { warn "claiming $drive needs sudo and a terminal to confirm on; run it interactively"; return 1; }

    # What to take: the mountpoint, any REPO_SUBDIR, and every repo the config
    # puts on this drive. A repo directory that is not there yet is not chowned.
    local -a shallow=("$mount") deep=() names=()
    [[ -z "$REPO_SUBDIR" ]] || shallow+=("$mount/$REPO_SUBDIR")
    local r dir
    for r in ${REPOS[@]+"${REPOS[@]}"}; do
        _repo_has_drive "$r" "$drive" || continue
        dir=$(_repo_path "$drive" "$r")
        [[ -d "$dir" ]] && { deep+=("$dir"); names+=("$r"); }
    done

    # The paths share one prefix and the mountpoint is printed above them, so the
    # preview names the scope rather than spelling every path out: what is taken
    # shallow, what is taken in full, and by whom. The refusals above have already
    # established that $mount is this drive's own mountpoint.
    local owner; owner="$(id -un):$(id -gn)"
    say "claiming $drive for $owner, as root, under $mount:"
    if [[ -n "$REPO_SUBDIR" ]]; then
        say "  the mountpoint and $REPO_SUBDIR/ themselves, not their other contents"
    else
        say "  the mountpoint itself, not its other contents"
    fi
    if (( ${#names[@]} > 0 )); then
        say "  these repo directories, recursively: $(_csv "${names[@]}")"
    else
        say "  no configured repo of yours is there yet, so nothing else"
    fi
    local ans
    read -rp "take them? [y/N] " ans
    [[ "$ans" == [Yy] || "$ans" == [Yy][Ee][Ss] ]] || { warn "nothing changed"; return 1; }

    sudo chown "$owner" "${shallow[@]}" || { warn "could not take $mount"; return 1; }
    if (( ${#deep[@]} > 0 )); then
        sudo chown -R "$owner" "${deep[@]}" || { warn "could not take the repo directories on $drive"; return 1; }
    fi
    say "claimed $drive"
}

cmd_claim() {
    (( $# == 1 )) || die_usage "usage: $CMD claim <drive>"
    [[ "$1" != -* ]] || die_usage "usage: $CMD claim <drive>"
    need mountpoint
    local drive="$1"
    _validate_segment "drive" "$drive"
    parse_config
    _claim_drive "$drive" || exit 1
}

## ─── init: create the repos ──────────────────────────────────────────
_check_usb() {
    mountpoint -q "$MOUNT_BASE/$1" || { warn "$(_not_mounted "$1")"; return 1; }
}

_init_one() {
    local repo="$1" drive="$2" dir parent
    dir=$(_repo_path "$drive" "$repo")
    if [[ -d "$dir" ]]; then
        say "$dir already exists"
        return 0
    fi
    # Writability before borg, and before anything is created: an unwritable
    # target fails several steps into borg with a Python traceback otherwise.
    parent=$(_init_target_dir "$drive")
    if [[ -n "$(_first_unwritable "$parent")" ]]; then
        warn "$parent is not writable, so $repo cannot be created on $drive"
        _offer_claim "$drive" "$parent" || true
        [[ -z "$(_first_unwritable "$parent")" ]] || return 1
    fi
    if [[ -n "$REPO_SUBDIR" ]]; then
        mkdir -p "$MOUNT_BASE/$drive/$REPO_SUBDIR" \
            || { warn "could not create $MOUNT_BASE/$drive/$REPO_SUBDIR"; return 1; }
    fi
    local -x BORG_REPO="$dir"
    local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"
    if borg init --encryption=repokey "$dir"; then
        say "initialised $dir"
    else
        warn "failed to initialise $dir"
        return 1
    fi
}

cmd_init() {
    ensure_borg; need mountpoint
    # <cmd> init [<repo> [<paths>...] [--drives <labels>...]]. A repo with paths
    # creates a new block; a repo without paths inits an existing repo; no repo
    # inits every configured repo. --drives sets the new block's drives (default
    # 'all'); everything after --drives is a drive label.
    local repo="" sawdrives=0
    local -a paths=() dr=()
    while (( $# )); do
        case "$1" in
            --drives)    sawdrives=1; shift ;;
            --drives=*)  die_usage "use '--drives d1 d2'; '--drives=...' is not supported" ;;
            -*)          die_usage "unknown flag: $1 (usage: $CMD init [<repo> [<paths>...] [--drives <labels>...]])" ;;
            *)
                if   (( sawdrives )); then dr+=("$1")
                elif [[ -z "$repo" ]]; then repo="$1"
                else paths+=("$1"); fi
                shift ;;
        esac
    done

    parse_config
    # Serialize against a concurrent archive/rename: init writes the config block
    # and creates borg repos, so it must not race another config writer.
    _acquire_lock
    local -a todo=()
    # A repo name not yet in the config means create it: with the folders given,
    # or as an awaiting-data repo when none are.
    local creating=0
    [[ -n "$repo" ]] && ! _repo_declared "$repo" && creating=1
    if (( creating )); then
        _validate_segment "repo name" "$repo"
        _reject_reserved_name "$repo"
        local drives="all"
        if (( sawdrives )); then
            (( ${#dr[@]} > 0 )) || die_usage "--drives needs at least one drive label"
            drives="${dr[*]}"
        fi
        _validate_drives_list "--drives" "$drives"
        # Every drive, mounted and writable, before anything is written. A repo
        # created on two of its four drives is a false redundancy belief: the
        # config says four copies exist and two do, and nothing says so again
        # until a restore needs the drive that was never written. pass-change and
        # rename already refuse a partial set for the same reason.
        local -a want=() missing=() pairs=()
        # shellcheck disable=SC2086  # validated space-separated labels; intentional split
        if [[ "$drives" == all ]]; then want=($ALL_DRIVES); else want=($drives); fi
        local d
        for d in "${want[@]}"; do
            mountpoint -q "$MOUNT_BASE/$d" || missing+=("$d")
        done
        (( ${#missing[@]} == 0 )) \
            || die "creating $repo needs every one of its drives mounted, so all copies start together; missing: ${missing[*]} (looked under $MOUNT_BASE)"
        for d in "${want[@]}"; do pairs+=("$d"$'\t'"$(_init_target_dir "$d")"); done
        _require_writable "creating $repo" "${pairs[@]}"
        # The passphrase is prompted after the block is written, so confirm that
        # can succeed before changing anything on disk.
        if [[ "$(_pass_source_kind)" == none ]]; then
            if [[ -z "$PASSPHRASE_PATH" ]]; then
                die "no passphrase file configured yet; create $CONFIG_FILE with a PASSPHRASE_PATH line pointing at a passphrase file (any name, gpg-encrypted or plaintext, chmod 600), put a 'set_pass $repo' line in that file, then re-run init"
            fi
            die "no passphrase file at $PASSPHRASE_PATH yet; create it with your first repo's set_pass line, then init adds the rest"
        fi
        [[ -t 0 ]] || die "creating $repo prompts for its passphrase; no terminal to prompt, so run it interactively"
        if (( ${#paths[@]} == 0 )); then
            # No folders given: a typo'd name would otherwise provision a junk repo
            # on every drive, so confirm the name and drives before writing.
            say "no folders given: this creates '$repo' as an awaiting-data repo on drives: $drives"
            local ans
            read -rp "create $repo? [y/N] " ans
            [[ "$ans" == [Yy] || "$ans" == [Yy][Ee][Ss] ]] || die "aborted; nothing changed"
        fi
        _append_repo_block "$repo" "$drives" "${paths[@]}"
        if (( ${#paths[@]} == 0 )); then
            say "added repo $repo to $CONFIG_FILE (awaiting data; add 'backup_data <path>' lines to its block)"
        else
            say "added repo $repo to $CONFIG_FILE"
        fi
        REPOS+=("$repo"); REPO_SEEN["$repo"]=1
        # The config line keeps 'all'; the in-memory drive list this init uses
        # must be the expanded pool, matching what backup_drives does at parse.
        REPO_DRIVES["$repo"]="${want[*]}"
        todo=("$repo")
    elif (( ${#paths[@]} > 0 )); then
        # Folders given for a name that is already a repo, or with no name at all.
        [[ -n "$repo" ]] || die_usage "creating a repo needs a name: $CMD init <repo> <path>..."
        die "repo $repo is already in $CONFIG_FILE; edit its block by hand, or pick a new name"
    else
        (( ! sawdrives )) || die_usage "--drives only applies when creating a repo"
        _require_repos_valid
        if [[ -n "$repo" ]]; then
            todo=("$repo")
        else
            todo=("${REPOS[@]}")
        fi
    fi

    # A repo's passphrase is born at init: store a missing one (prompted, failing
    # closed with no terminal) rather than refusing to proceed. Presence comes
    # from the names-only child, so no value enters this process here.
    local r
    for r in "${todo[@]}"; do
        _repo_has_pass "$r" || _store_pass "$r"
    done

    local drive available configured rc=0
    for r in "${todo[@]}"; do
        say "initialising repo $r..."
        available=(); configured=0
        # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
        for drive in ${REPO_DRIVES[$r]}; do
            configured=$(( configured + 1 ))
            if _check_usb "$drive"; then available+=("$drive"); fi
        done
        if (( ${#available[@]} == 0 )); then
            warn "no drives available for $r"
            rc=1
            continue
        fi
        for drive in "${available[@]}"; do
            _init_one "$r" "$drive" || rc=1
        done
        # An existing repo may legitimately be completed one drive at a time, but
        # the gap should not be silent: until every drive has a copy, the config
        # promises redundancy the disks do not have.
        if (( ${#available[@]} < configured )); then
            warn "$r is not on all ${configured} of its drives yet; re-run '$CMD init $r' with the missing drive(s) mounted"
            rc=1
        fi
    done
    exit "$rc"
}

## ─── archive: the default action ─────────────────────────────────────
_fmt_duration() {
    local secs="$1" h m s
    h=$(( secs / 3600 )); m=$(( (secs % 3600) / 60 )); s=$(( secs % 60 ))
    if   (( h > 0 )); then printf '%dh%dm%ds' "$h" "$m" "$s"
    elif (( m > 0 )); then printf '%dm%ds' "$m" "$s"
    else                   printf '%ds' "$s"
    fi
}

# Bytes to a short human size via numfmt SI (decimal, powers of 1000), dropping
# a trailing .0 so a round value reads 2MB rather than 2.0MB.
_fmt_size() {
    local s
    s=$(numfmt --to=si --suffix=B --format='%.1f' "$1" 2>/dev/null) || s="${1}B"
    printf '%s' "${s/.0/}"
}

# One summary group as ", <count> <label> (name, name)", or nothing when the
# list is empty, so an unused category drops out of the line cleanly.
_group() {
    local label="$1"; shift
    (( $# > 0 )) || return 0
    printf ', %d %s (%s)' "$#" "$label" "$(_csv "$@")"
}

# Confirm with borg that a repo's allowlist actually bites before writing an
# archive under it. Args: <repo> <pattern opt>... -- <source>... ; BORG_REPO is
# already set by the caller. Returns 0 when the allowlist both keeps something
# and drops something, 1 otherwise, warning with the reason.
#
# borg's matcher is the only authority on what these patterns do, so this asks
# it rather than reasoning about the globs. `create --dry-run` reads no file data
# and does not unlock the repo key, so the cost is one metadata walk of the
# sources and no passphrase use: no agent prompt, no hardware token touch.
# `--list` writes its per-file status to stderr, not stdout, hence the swap
# below. The archive name is never created, so reusing one is harmless.
#
# Two outcomes are refused. Nothing dropped means the allowlist is inert and the
# folder goes to the drive whole, which is a privacy failure that reports as
# success. Nothing kept means an empty archive; catching it here means nothing is
# written and no retention decision hangs on it.
_assert_filter_bites() {
    local repo="$1"; shift
    local -a popts=() srcs=()
    local a seen=0
    for a in "$@"; do
        if   (( seen ));       then srcs+=("$a")
        elif [[ "$a" == -- ]]; then seen=1
        else                        popts+=("$a"); fi
    done
    (( ${#srcs[@]} > 0 )) || return 0
    local out rc=0
    out=$(borg create --dry-run --list ${popts[@]+"${popts[@]}"} "::{now}" "${srcs[@]}" 2>&1 >/dev/null) || rc=$?
    if (( rc != 0 )); then
        # A dry run that cannot even walk the sources says nothing about the
        # patterns. Do not block the backup on it; the post-create checks stand.
        warn "could not dry-run the filter for $repo (borg exited $rc); the allowlist was not verified"
        return 0
    fi
    local kept dropped
    kept=$(printf '%s\n' "$out" | grep -c '^- ') || true
    dropped=$(printf '%s\n' "$out" | grep -c '^x ') || true
    if (( dropped == 0 )); then
        warn "$repo has an allowlist but it dropped nothing, so the folders would be archived whole; the globs are matching no path borg sees"
        warn "borg matches the source path with its leading slash removed, so a folder at /home/you/proj is matched as home/you/proj/...; check the globs with: borg create --list --dry-run"
        return 1
    fi
    if (( kept == 0 )); then
        warn "$repo has an allowlist and it kept nothing, so the archive would be empty; check the globs with: borg create --list --dry-run"
        return 1
    fi
    return 0
}

# Append one repo-wide glob list to the caller's pattern array as '<sign> sh:'
# patterns. One function for both signs because exclude and include_only must
# anchor identically: an edit reaching one loop and not the other would change
# what a backup holds and say nothing. $1 names the caller's array, $2 is '+' or
# '-', $3 the newline-joined globs, the rest the matcher roots.
_add_repo_wide_patterns() {
    local -n _out="$1"
    local sign="$2" globs="$3"; shift 3
    local p mroot
    while IFS= read -r p; do
        [[ -n "$p" ]] || continue
        if [[ "$p" == /* ]]; then
            _out+=( --pattern "$sign sh:${p#/}" )
        else
            for mroot in "$@"; do _out+=( --pattern "$sign sh:$mroot/**/$p" ); done
        fi
    done <<< "$globs"
}

# One repo to one drive. Returns 0 on archived-and-verified, 1 if the archive was
# not created, 2 if it was created but failed its quick verification.
_archive_one_drive() {
    local repo="$1" drive="$2"
    local mount="$MOUNT_BASE/$drive"
    if ! mountpoint -q "$mount"; then
        warn "$(_not_mounted "$drive")"
        _note_issue "$repo on $drive: not mounted at $mount"
        return 1
    fi
    local -x BORG_REPO; BORG_REPO=$(_repo_path "$drive" "$repo")
    if [[ ! -d "$BORG_REPO" ]]; then
        warn "no $repo repo on $drive"
        _note_issue "$repo on $drive: no repo directory at $BORG_REPO"
        return 1
    fi
    # borg's first act on a write is to create a lock file inside the repo dir,
    # so an unwritable repo fails with a Python traceback several steps in. The
    # preflight already refused the run if no drive had a writable repo; this
    # catches the mixed case, where some drives are fine.
    local blocked; blocked=$(_first_unwritable "$BORG_REPO")
    if [[ -n "$blocked" ]]; then
        warn "$repo on $drive is not writable ($blocked)"
        _unwritable_hint "$BORG_REPO" "$drive"
        _note_issue "$repo on $drive: not writable ($blocked)"
        return 1
    fi
    local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"

    # Each folder is anchored with /./ so it lands at the archive root under
    # its own basename, regardless of where it lives on disk. The same loop
    # records each folder's matcher root, because the two are not the same path
    # and the filtering below needs the second one.
    local -a srcs=() popts=() roots=()
    local p key g mroot
    while IFS= read -r p; do
        [[ -n "$p" ]] || continue
        srcs+=( "$(dirname "$p")/./$(basename "$p")" )
        roots+=( "${p#/}" )
    done <<< "${REPO_SRC[$repo]}"

    # Filtering. borg does the matching; this assembles one ordered list of
    # '+'/'- sh:' patterns (first match wins, unmatched kept). This is the one
    # place the matcher's addressing is written down; everything else points here.
    #
    # THE MATCHER ROOT. borg matches the source path made relative, i.e. the
    # absolute path with its leading slash off: /home/john/proj reaches the
    # matcher as home/john/proj/... . The /./ anchor changes only the name the
    # item is stored under, never what the matcher sees, so a pattern written
    # against the stored name matches nothing. That failure is silent in the
    # dangerous direction: an allowlist whose '-' never fires archives its folder
    # whole and reports success. Every pattern here is anchored on a folder's
    # matcher root, and _assert_filter_bites has borg confirm it before create.
    #
    # ORDER. Every exclude is a plain drop emitted before any '+', so an explicit
    # exclude always beats an allowlist. Then either a repo-wide allowlist ('+'
    # per glob, global '- sh:**' for the rest) or a folder-scoped one ('+' on
    # that folder's matcher root, '- sh:<root>/**' for its rest, other folders
    # left whole). The two scopes are mutually exclusive, checked at parse.
    #
    # DEPTH, deliberately different per directive. exclude and include_only name
    # no folder, so a bare glob means "anywhere in this repo" and is emitted per
    # root behind '**/', which borg matches at zero directories as well as many.
    # include_only_in has named its folder, so its globs stay anchored there;
    # widening them would let 'src/**' keep a vendored src/ levels down. A glob
    # written as an absolute path is already a matcher path once the leading
    # slash is off, so it passes through unprefixed.
    _add_repo_wide_patterns popts '-' "${REPO_EXCLUDES[$repo]:-}" ${roots[@]+"${roots[@]}"}
    if [[ -n "${REPO_INCLUDES[$repo]:-}" ]]; then
        _add_repo_wide_patterns popts '+' "${REPO_INCLUDES[$repo]}" ${roots[@]+"${roots[@]}"}
        popts+=( --pattern '- sh:**' )
    else
        while IFS= read -r p; do
            [[ -n "$p" ]] || continue
            key="$repo"$'\n'"$p"
            [[ -n "${REPO_FINCLUDE_GLOBS[$key]:-}" ]] || continue
            mroot="${p#/}"
            while IFS= read -r g; do
                [[ -n "$g" ]] && popts+=( --pattern "+ sh:$mroot/$g" )
            done <<< "${REPO_FINCLUDE_GLOBS[$key]}"
            popts+=( --pattern "- sh:$mroot/**" )
        done <<< "${REPO_SRC[$repo]}"
    fi

    # Once per repo per run: the answer is a property of the config and the
    # source tree, not of the drive being written.
    if [[ -n "${REPO_INCLUDES[$repo]:-}${REPO_HAS_FINCLUDE[$repo]:-}" ]] \
       && [[ -z "${RUN_REPO_FILTER_CHECKED[$repo]:-}" ]]; then
        RUN_REPO_FILTER_CHECKED["$repo"]=1
        if ! _assert_filter_bites "$repo" ${popts[@]+"${popts[@]}"} -- "${srcs[@]}"; then
            _note_issue "$repo on $drive: allowlist matched nothing, so nothing was archived"
            return 2
        fi
    fi

    # borg's default compression is lz4; a `compression` directive overrides it
    # with the user's spec, passed through for borg to validate and apply.
    [[ -n "${REPO_COMPRESSION[$repo]:-}" ]] && popts+=( --compression "${REPO_COMPRESSION[$repo]}" )

    # borg's rc: 0 clean, 1 warnings (a source file changed or vanished mid-run,
    # common for live dotfiles) with the archive still written, 2+ a real error.
    # Treat warnings as success so a changing profile isn't a failure. Progress
    # goes to stderr, so gate it on stderr being a terminal: interactive runs get
    # feedback, cron logs stay clean.
    local -a progress=()
    [[ -t 2 ]] && progress=( --progress )
    local create_rc=0 create_json
    create_json=$(borg create --json ${progress[@]+"${progress[@]}"} ${popts[@]+"${popts[@]}"} "::{now}" "${srcs[@]}") || create_rc=$?
    if (( create_rc >= 2 )); then
        warn "borg create failed (rc=$create_rc) for $repo on $drive"
        # borg's own error reached stderr above this line, unredirected, so any
        # remedy printed before it has already scrolled away. The preflight
        # cleared this repo minutes ago; re-test, because a drive that remounts
        # read-only mid-run lands here.
        blocked=$(_first_unwritable "$BORG_REPO")
        if [[ -n "$blocked" ]]; then
            warn "$blocked is not writable now, which is very likely the error above"
            _unwritable_hint "$BORG_REPO" "$drive"
        fi
        _note_issue "$repo on $drive: borg create failed (rc=$create_rc)"
        return 1
    fi
    if (( create_rc == 1 )); then
        warn "borg create finished with warnings for $repo on $drive; archive was still created"
    fi

    # borg's own figures: this snapshot's deduplicated size is how much the repo
    # grew, and unique_csize is the repo's on-disk total. Best-effort; a parse
    # miss just drops the size from the line. borg 2.0 renames both fields.
    local added_bytes repo_bytes size_note=""
    added_bytes=$(printf '%s' "$create_json" | sed -n 's/.*"deduplicated_size"[^0-9]*\([0-9]\{1,\}\).*/\1/p')
    repo_bytes=$(printf '%s' "$create_json"  | sed -n 's/.*"unique_csize"[^0-9]*\([0-9]\{1,\}\).*/\1/p')
    if [[ -n "$added_bytes" ]]; then
        # The same logical data goes to every drive, so count each repo's growth
        # once; summing per drive would multiply the run total by the drive count.
        if [[ -z "${RUN_REPO_DEDUP_COUNTED[$repo]:-}" ]]; then
            RUN_DEDUP_BYTES=$(( RUN_DEDUP_BYTES + added_bytes ))
            RUN_REPO_DEDUP_COUNTED["$repo"]=1
        fi
        if [[ -n "$repo_bytes" ]]; then
            size_note=" (+$(_fmt_size "$added_bytes"), repo $(_fmt_size "$repo_bytes"))"
        else
            size_note=" (+$(_fmt_size "$added_bytes"))"
        fi
    fi

    # An archive can come out empty from a wrong include_only glob or from source
    # folders that hold nothing, and an empty archive is structurally valid, so
    # borg and the verification below both accept it. create --json already
    # reported the file count, so this costs nothing; do not replace it with a
    # listing call. A folder-scoped include_only that matches nothing only
    # empties its one folder, which a whole-archive count cannot see.
    local empty_archive=0 nfiles
    nfiles=$(printf '%s' "$create_json" | sed -n 's/.*"nfiles"[^0-9]*\([0-9]\{1,\}\).*/\1/p')
    if [[ -z "$nfiles" ]]; then
        warn "could not read nfiles from borg create --json for $repo on $drive; the empty-archive check did not run"
    elif (( nfiles == 0 )); then
        warn "archived $repo on $drive but the archive is EMPTY (0 files): check the include_only globs with 'borg create --list --dry-run', or confirm the folders are not empty"
        empty_archive=1
    fi

    # Verify before retention, not after. Prune deletes archives and compact
    # frees their space, so running them first would destroy known-good history
    # on the assumption that the archive just written is good, and test that
    # assumption afterwards. Nothing is pruned unless the new archive both
    # verifies and holds files.
    local check_out="" check_rc=0
    if [[ -n "$pc" ]]; then
        check_out=$(borg check --archives-only --last 1 2>&1) || check_rc=$?
    else
        # With no passcommand borg asks for the passphrase on stderr, so
        # capturing stderr here would hide the prompt and the run would look hung.
        borg check --archives-only --last 1 || check_rc=$?
    fi
    if (( check_rc != 0 )); then
        warn "archived $repo on $drive but the new archive FAILED verification; run: $CMD check $repo"
        if [[ -n "$check_out" ]]; then warn "$check_out"; fi
        warn "retention skipped for $repo on $drive; no archive was pruned"
        _note_issue "$repo on $drive: archive written but failed verification"
        return 2
    fi
    if (( empty_archive )); then
        # Structurally valid but empty: report unverified so it is not counted as
        # a good backup, and keep retention off so it cannot rotate a real one out.
        warn "retention skipped for $repo on $drive; no archive was pruned"
        _note_issue "$repo on $drive: archive written but empty, so it is not counted as a backup"
        return 2
    fi

    # Pruned only if the repo has a keep line; compact frees the space prune
    # marks. borg 2.0 change point: prune there requires an archive glob.
    if [[ -n "${SEEN_KEEP[$repo]+x}" ]]; then
        local kl="${KEEP_LAST[$repo]:-$KEEP_LAST_DEFAULT}"
        if (( kl != 0 )); then
            borg prune --keep-last "$kl" || warn "prune failed for $repo on $drive"
        else
            borg prune \
                --keep-hourly   "${KEEP_HOURLY[$repo]:-$KEEP_HOURLY_DEFAULT}" \
                --keep-daily    "${KEEP_DAILY[$repo]:-$KEEP_DAILY_DEFAULT}" \
                --keep-weekly   "${KEEP_WEEKLY[$repo]:-$KEEP_WEEKLY_DEFAULT}" \
                --keep-monthly  "${KEEP_MONTHLY[$repo]:-$KEEP_MONTHLY_DEFAULT}" \
                --keep-yearly   "${KEEP_YEARLY[$repo]:-$KEEP_YEARLY_DEFAULT}" \
                || warn "prune failed for $repo on $drive"
        fi
        borg compact || warn "compact failed for $repo on $drive"
    fi

    say "archived $repo on $drive$size_note"
    return 0
}

# Set to 1 once a run has printed its summary, so the EXIT trap stays quiet on a
# normal exit. The trap must not disturb the exit status, so it never calls exit
# and never returns non-zero.
_ARCHIVE_REPORTED=0
_archive_exit_notice() {
    local rc=$?
    # Only when stdout is not a terminal. The line exists so a redirected run can
    # be read for its outcome from that one stream: a fatal exit prints its reason
    # on stderr and nothing on stdout, which would otherwise be indistinguishable
    # from a run that never happened.
    if (( rc != 0 && _ARCHIVE_REPORTED == 0 )) && [[ ! -t 1 ]]; then
        printf '%s failed before it could report; the reason is on stderr above (exit %d)\n' "$CMD" "$rc"
    fi
    return 0
}

# One repo across its drives. The return code is the repo's outcome for the
# summary, not a plain pass/fail: 0 verified clean on every drive, 3 backed up
# (verified on at least one) but a drive was missing or failed verification, 4 an
# archive was written but none verified anywhere, 1 no archive written anywhere.
_archive_repo() {
    local repo="$1"
    # borg treats a missing source path as a warning, and the create step maps
    # borg's warning code to success so a dotfile changing mid-run is not a
    # failure. Together those would let a deleted or unmounted folder produce a
    # short archive that reports as good and then rotates a real one out under
    # retention. Refuse the repo instead: the run loses one backup, where
    # proceeding loses an old one.
    local missing_src=() s
    while IFS= read -r s; do
        [[ -n "$s" ]] || continue
        [[ -e "$s" ]] || missing_src+=("$s")
    done <<< "${REPO_SRC[$repo]}"
    if (( ${#missing_src[@]} > 0 )); then
        warn "repo $repo not archived: configured folder(s) missing: $(_csv "${missing_src[@]}")"
        warn "fix the path in $CONFIG_FILE or mount what holds it; existing archives are untouched"
        _note_issue "$repo: configured folder(s) missing: $(_csv "${missing_src[@]}")"
        _RUN_LAST=block
        return 1
    fi
    if [[ -n "$_RUN_LAST" ]]; then say ""; fi
    _RUN_LAST=block
    say "archiving repo $repo..."
    local start drive r any_clean=0 any_unverified=0 any_missing=0
    start=$(date +%s)
    # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
    for drive in ${REPO_DRIVES[$repo]}; do
        r=0
        _archive_one_drive "$repo" "$drive" || r=$?
        case "$r" in
            0) any_clean=1 ;;
            2) any_unverified=1 ;;
            *) any_missing=1 ;;
        esac
    done
    local dur; dur=$(_fmt_duration $(( $(date +%s) - start )))
    if (( any_clean )); then
        say "repo $repo done in $dur"
        if (( any_unverified || any_missing )); then return 3; fi
        return 0
    fi
    if (( any_unverified )); then
        warn "repo $repo: an archive was written but failed verification on every drive; run: $CMD check $repo"
        return 4
    fi
    warn "repo $repo not archived to any drive"
    return 1
}

# Refuse a run that cannot succeed, before the passphrase is touched. A repo that
# would archive is archive-on and holds backup_data. Mounted is not enough:
# Mounted is not enough: archive has no read-only fallback the way check and
# extract do. For a GPG passphrase file the passphrase is an agent prompt or a
# hardware-token touch, so nothing is solicited for work that cannot succeed. When the repos on a drive
# are simply owned by someone else, offer to claim it and re-scan once; with no
# terminal there is no offer, and the run fails with the remedy printed.
# Returns 0 when at least one repo would archive, 1 when none would.
_archive_preflight_drives() {
    local r d rp has_target=0 any_drive=0 any_writable=0 offered=0 claimed=0 ever_claimed=0
    local -a want=() unwritable=()
    local -A unwritable_drives=()
    while :; do
        has_target=0; any_drive=0; any_writable=0; claimed=0
        want=(); unwritable=(); unwritable_drives=()
        for r in "$@"; do
            _repo_archives "$r" || continue
            [[ -n "${REPO_SRC[$r]:-}" ]] || continue
            has_target=1
            # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
            for d in ${REPO_DRIVES[$r]}; do
                want+=("$d")
                mountpoint -q "$MOUNT_BASE/$d" || continue
                any_drive=1
                rp=$(_repo_path "$d" "$r")
                [[ -d "$rp" ]] || continue     # reported per drive by _archive_one_drive
                if [[ -z "$(_first_unwritable "$rp")" ]]; then
                    any_writable=1
                else
                    unwritable+=("$r on $d")
                    unwritable_drives["$d"]=1
                fi
            done
        done
        (( ${#unwritable[@]} > 0 )) || break
        warn "repo directory not writable: $(_csv "${unwritable[@]}")"
        (( offered )) && break
        offered=1
        # The cause is a property of the drive, not the repo, so several repos on
        # one drive share one offer and one remedy.
        for d in "${!unwritable_drives[@]}"; do
            if _offer_claim "$d" "$MOUNT_BASE/$d"; then claimed=1; ever_claimed=1; fi
        done
        (( claimed )) || break
    done
    if (( has_target && ! any_drive )); then
        local uniq
        uniq=$(printf '%s\n' "${want[@]}" | awk '!seen[$0]++' | paste -sd' ' -)
        die "no drives mounted for the repo(s) to back up (looked for: ${uniq}); plug one in and re-run"
    fi
    (( has_target && any_drive && ! any_writable )) \
        && die "no writable repo on any mounted drive; nothing can be archived, so the passphrase was not touched"
    (( has_target )) || return 1
    # A claim ran, so its output is finished with; start the run's own on a fresh
    # line rather than running the two together.
    if (( ever_claimed )); then say ""; fi
    return 0
}

cmd_archive() {
    (( $# <= 1 )) || die_usage "$CMD archives all repos, or one named repo; run '$CMD --help' for the subcommands"
    # Every ordinary outcome of a run, including total failure, ends with a
    # summary line on stdout; a fatal exit printed nothing there at all, so a run
    # that died on a config or passphrase problem was indistinguishable from a
    # run that never happened. This is the backup log for a cron-driven tool.
    _ARCHIVE_REPORTED=0
    trap '_archive_exit_notice' EXIT
    # First line of the run, before the config is even read, so a log identifies
    # what produced it even when the run dies in config parsing or the preflight.
    say "$(_identity)"
    local only=""
    (( $# == 1 )) && only="$1"
    ensure_borg; need mountpoint
    parse_config
    _require_repos_valid
    if [[ -n "$only" ]]; then
        _repo_declared "$only" \
            || die_usage "$only is not a configured repo or a known subcommand; run '$CMD --help' for the list"
    fi

    local -a run_repos=()
    if [[ -n "$only" ]]; then run_repos=("$only"); else run_repos=("${REPOS[@]}"); fi

    local has_target=0
    _archive_preflight_drives "${run_repos[@]}" && has_target=1

    # The lock comes before the passphrase, not after: the preflight below runs
    # the _pass-names child, which for a GPG file means an agent prompt or a
    # hardware-token touch, and a second concurrent run should be turned away
    # before it makes you touch anything. Everything above is read-only, so
    # nothing is lost by holding the lock from here.
    _acquire_lock

    # borg reads these from the environment and they override anything decided
    # here, so an exported leftover from an earlier shell would supply the
    # passphrase while the run reports it came from the file or a prompt. Not
    # neutralised, since setting one deliberately is legitimate, but never silent.
    if [[ -n "${BORG_PASSPHRASE:-}" || -n "${BORG_PASSPHRASE_FD:-}" || -n "${BORG_PASSCOMMAND:-}" ]]; then
        warn "a borg passphrase variable is set in the environment and takes precedence over $CONFIG_FILE's passphrase source"
    fi

    (( has_target )) && _archive_pass_preflight "${run_repos[@]}"

    RUN_DEDUP_BYTES=0
    RUN_REPO_DEDUP_COUNTED=()
    RUN_REPO_FILTER_CHECKED=()
    _RUN_LAST=""
    local script_start repo r rc=0
    # Each repo lands in one outcome by name. A partial repo is listed in both
    # backed_up and partial: it has a usable archive on a drive and is also
    # flagged for the drive it missed or that failed.
    local -a backed_up=() partial=() unverified=() failed=() awaiting_data=()
    script_start=$(date +%s)
    for repo in "${run_repos[@]}"; do
        if ! _repo_archives "$repo"; then
            if [[ "$_RUN_LAST" == block ]]; then say ""; fi
            say "skipping archive-off repo $repo"
            _RUN_LAST=skip
            continue
        fi
        if [[ -z "${REPO_SRC[$repo]:-}" ]]; then
            if [[ "$_RUN_LAST" == block ]]; then say ""; fi
            say "skipping $repo: no backup_data yet (awaiting data)"
            _RUN_LAST=skip
            awaiting_data+=("$repo")
            continue
        fi
        # No stored passphrase: skipped, warned and counted via PASS_SKIPPED in
        # the preflight. In prompt mode there is nothing to store, so run it.
        _pass_prompt_mode || _repo_has_pass "$repo" || continue
        r=0
        _archive_repo "$repo" || r=$?
        case "$r" in
            0) backed_up+=("$repo") ;;
            3) backed_up+=("$repo"); partial+=("$repo") ;;
            4) unverified+=("$repo") ;;
            *) failed+=("$repo") ;;
        esac
    done

    # An error run is anything short of a verified archive on every drive of
    # every archived repo. Archive-off and awaiting-data repos are intentional.
    if (( ${#failed[@]} > 0 || ${#unverified[@]} > 0 || ${#partial[@]} > 0 || ${#PASS_SKIPPED[@]} > 0 )); then
        rc=1
    fi

    if (( ${#backed_up[@]} == 0 && ${#failed[@]} == 0 && ${#unverified[@]} == 0 && ${#PASS_SKIPPED[@]} == 0 && ${#awaiting_data[@]} == 0 )); then
        if [[ -n "$_RUN_LAST" ]]; then say ""; fi
        if [[ -n "$only" ]]; then
            say "$only is archive-off; set 'archive yes' or remove its 'archive no' line in $CONFIG_FILE"
        else
            say "all configured repos are archive-off; nothing to back up"
        fi
        _ARCHIVE_REPORTED=1
        exit 0
    fi

    local dur ts added noun=repos backed_names="" extra=""
    dur=$(_fmt_duration $(( $(date +%s) - script_start )))
    ts=$(date '+%a %d %b, %H:%M')
    added=$(_fmt_size "$RUN_DEDUP_BYTES")
    (( ${#backed_up[@]} == 1 )) && noun=repo
    (( ${#backed_up[@]} > 0 )) && backed_names=" ($(_csv "${backed_up[@]}"))"
    extra+=$(_group failed     "${failed[@]}")
    extra+=$(_group unverified "${unverified[@]}")
    extra+=$(_group partial    "${partial[@]}")
    extra+=$(_group skipped    "${PASS_SKIPPED[@]}")
    # An archive-off repo is a standing config fact, already said once per repo
    # in the body above, so it is not restated in the line a cron log is read for.
    extra+=$(_group "without backup_data" "${awaiting_data[@]}")
    if [[ -n "$_RUN_LAST" ]]; then say ""; fi
    # On stdout, immediately above the summary: this is the answer to "why was
    # that repo partial", and it has to reach whoever reads the stream the
    # verdict lands in. Only on an error run, so a clean run keeps its one line.
    if (( rc != 0 && ${#RUN_ISSUES[@]} > 0 )); then
        local issue
        for issue in "${RUN_ISSUES[@]}"; do say "  $issue"; done
        say ""
    fi
    if (( rc == 0 )); then
        printf 'backed up and verified %d %s%s, +%s in %s%s, %s\n' \
            "${#backed_up[@]}" "$noun" "$backed_names" "$added" "$dur" "$extra" "$ts"
    else
        printf 'completed with errors: backed up and verified %d %s%s, +%s in %s%s, %s\n' \
            "${#backed_up[@]}" "$noun" "$backed_names" "$added" "$dur" "$extra" "$ts"
    fi
    _ARCHIVE_REPORTED=1
    exit "$rc"
}

## ─── extract: restore ────────────────────────────────────────────────
# Said by both extract routes that can write over live folders, so it is one
# string: the whole-suite form and the named-repo form must not drift apart.
readonly IN_PLACE_WARNING='in-place restore overwrites files at their original locations'

# True when any of the repo's folders carries a restore_to line. Kept as a scan
# rather than a second map so the two cannot disagree.
_repo_has_restore_to() {
    local repo="$1" src
    while IFS= read -r src; do
        [[ -n "$src" ]] || continue
        [[ -z "${REPO_RESTORE_TO["$repo"$'\n'"$src"]:-}" ]] || return 0
    done <<< "${REPO_SRC[$repo]:-}"
    return 1
}

# Several helpers below build borg's environment in a subshell so BORG_REPO,
# BORG_PASSCOMMAND and RO_LOCKOPT cannot reach the next drive. shellcheck reports
# that as SC2030/SC2031, which is the design rather than a defect, so those
# functions carry a one-line disable; it is never disabled globally.

# True when the spec makes the caller list the archives before extracting, so
# with no stored passphrase the second borg prompt is expected, not a retry.
_spec_lists_first() {
    local spec="$1"
    [[ -z "$spec" || "$spec" =~ ^-[0-9]+$ ]]
}

_require_restore_path() {
    [[ -n "$RESTORE_PATH" ]] || die "RESTORE_PATH is not set in $CONFIG_FILE"
    [[ -d "$RESTORE_PATH" ]] || die "RESTORE_PATH '$RESTORE_PATH' is not a directory; create it first"
}

# `borg list --short` with borg's own reason surfaced rather than left on a
# stderr stream a multi-repo run scrolls past: a wrong passphrase and an
# archive-less repo otherwise both arrive several layers up as "no usable
# archive", which sends you looking at the wrong thing. Only captured when a
# passphrase is already in the environment; with none, borg asks on stderr, and
# redirecting that would hide the prompt and the run would look hung.
_list_archives() {
    local what="$1"; shift
    local rc=0 errf="" reason
    if [[ -n "${BORG_PASSCOMMAND:-}${BORG_PASSPHRASE:-}${BORG_PASSPHRASE_FD:-}" ]]; then
        errf=$(mktemp) || errf=""
    fi
    if [[ -n "$errf" ]]; then
        # BORG_SHOW_SYSINFO=no is what makes the line below the reason. borg logs
        # its own message first and, on an unexpected exception, a traceback and
        # then an environment dump, whose last line is 'SSH_ORIGINAL_COMMAND:
        # None', so the tail of the stream is the dump and never the cause. With
        # the dump off, the last line is the message on an ordinary error and the
        # exception itself on a crash, which is the line worth showing either way.
        BORG_SHOW_SYSINFO=no borg list ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --short "$@" 2>"$errf" || rc=$?
        if (( rc != 0 )); then
            reason=$(grep -v '^[[:space:]]*$' "$errf" | tail -n1)
            warn "could not list $what: ${reason:-borg exited $rc}"
        fi
        rm -f "$errf"
    else
        borg list ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --short "$@" || rc=$?
        (( rc == 0 )) || warn "could not list $what (borg exited $rc)"
    fi
    return "$rc"
}

# Resolve a spec to a concrete archive name, with BORG_REPO already set. Empty
# or -1 is the latest; -N is N back; anything else is a literal archive name,
# passed through unverified, since checking it would cost another borg call and,
# with no stored passphrase, another prompt per drive.
_resolve_archive() {
    local what="$1" spec="$2"
    if [[ -z "$spec" || "$spec" == "-1" ]]; then
        local latest
        latest=$(_list_archives "$what" --last 1) || return 1
        [[ -n "$latest" ]] || { warn "no archives found in $what, aborting"; return 1; }
        printf '%s\n' "$latest"
    elif [[ "$spec" =~ ^-([0-9]+)$ ]]; then
        local n="${BASH_REMATCH[1]}"
        (( n >= 1 )) || { warn "invalid -N: '$spec' (use -1, -2, ...)"; return 1; }
        local listing arr=()
        listing=$(_list_archives "$what" --last "$n") || return 1
        mapfile -t arr <<< "$listing"
        [[ -n "${arr[-1]:-}" ]] || unset 'arr[-1]'
        if (( ${#arr[@]} < n )); then
            warn "$what has only ${#arr[@]} archive(s); cannot extract $spec"
            return 1
        fi
        printf '%s\n' "${arr[0]}"
    else
        printf '%s\n' "$spec"
    fi
}

# Report what actually arrived under $dest for the archive-relative paths asked
# for. borg's positional paths start at a source folder's own basename, because
# every source is anchored with /./ at create time, so a typo matches nothing
# while borg still reports success; check what landed rather than the exit status.
# $2 is the hint printed after a miss. Returns 1 when nothing arrived.
_report_arrivals() {
    local dest="$1" hint="$2"; shift 2
    local p arrived=() missing=()
    for p in "$@"; do
        if [[ -e "$dest/$p" ]]; then arrived+=("$p"); else missing+=("$p"); fi
    done
    if (( ${#missing[@]} > 0 )); then
        warn "nothing was restored for: $(_csv "${missing[@]}")"
        warn "$hint"
    fi
    (( ${#arrived[@]} > 0 )) || return 1
    say "restored into $dest:"
    for p in "${arrived[@]}"; do say "  $p"; done
    return 0
}

# Config-free extract: name the repo by path, the archive lands in the current
# directory, and borg prompts for the passphrase since nothing is stored.
_borgex_path() {
    local repo="$1" spec="$2"; shift 2
    local paths=("$@")
    if [[ ! -d "$repo" ]]; then
        if [[ "$repo" == /* ]]; then die "no directory at $repo"
        else die "no directory at $repo (relative to the current directory, $PWD)"; fi
    fi
    local resolved; resolved=$(readlink -f -- "$repo") || die "cannot resolve path $repo"
    local -x BORG_REPO="$resolved"
    _set_read_lock "$resolved"
    if [[ -z "${BORG_PASSPHRASE:-}" && -z "${BORG_PASSPHRASE_FD:-}" && -z "${BORG_PASSCOMMAND:-}" ]]; then
        local how=once
        _spec_lists_first "$spec" && how=twice
        _prompt_notice "a repo named by path has nothing stored" "$resolved" "$how"
    fi
    local archive; archive=$(_resolve_archive "$resolved" "$spec") || return 1
    local dest="$PWD"
    if (( ${#paths[@]} > 0 )); then
        say "extracting ${#paths[@]} path(s) from $resolved (archive: $archive) into $dest..."
        if borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --progress "::$archive" "${paths[@]}"; then
            _report_arrivals "$dest" "a path starts at a source folder's own name, case included; list the archive with: borg list '$resolved::$archive'" "${paths[@]}" \
                || { warn "nothing was restored from $resolved"; return 1; }
            return 0
        fi
        warn "extraction from $resolved failed; see borg output above"; return 1
    fi
    say "extracting $resolved (archive: $archive) into $dest..."
    if borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --progress "::$archive"; then say "restored $resolved into $dest"; return 0; fi
    warn "extraction from $resolved failed; see borg output above"; return 1
}

# Restore one repo's folders to their original parent directories. BORG_REPO and
# the passphrase environment must already be set. Returns non-zero on failure.
_restore_folders_in_place() {
    local repo="$1" archive="$2" drive="$3"
    say "restoring $repo (archive: $archive) in place from $drive..."
    # An in-place restore covering ~/.borg-config or the passphrase file would
    # overwrite the running tool's own inputs. This run is unaffected (it holds
    # them in memory), but snapshot and warn so you re-check before the next.
    local cfg_before="" pass_before=""
    if [[ -e "$CONFIG_FILE" ]]; then cfg_before=$(sha256sum -- "$CONFIG_FILE" 2>/dev/null) || true; fi
    if [[ -n "$PASSPHRASE_PATH" && -e "$PASSPHRASE_PATH" ]]; then pass_before=$(sha256sum -- "$PASSPHRASE_PATH" 2>/dev/null) || true; fi
    if [[ -z "${REPO_SRC[$repo]:-}" ]]; then
        warn "$repo has no backup_data folders, so there is nothing to restore in place"
        return 1
    fi
    local rc=0 src parent base
    while IFS= read -r src; do
        [[ -n "$src" ]] || continue
        parent=$(dirname "$src"); base=$(basename "$src")
        if [[ -n "$SRC_HOME" && ( "$parent" == "$SRC_HOME" || "$parent" == "$SRC_HOME"/* ) ]]; then
            parent="$HOME${parent#"$SRC_HOME"}"
        elif [[ -n "$SRC_HOME" && "$src" == "$SRC_HOME" ]]; then
            # A source that is the foreign home itself has nothing left to remap:
            # the archive holds it under its own name, so it can only land beside
            # the real one. Say so rather than let the write fail unexplained.
            warn "SRC_HOME cannot remap '$src': the archive stores it as '$base', so it restores to '$parent/$base', not under your home"
        fi
        say "  $base -> $parent/"
        if ! ( mkdir -p "$parent" && cd "$parent" && borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --progress "::$archive" "$base" ); then
            warn "failed to restore '$base' into '$parent'"; rc=1
        fi
    done <<< "${REPO_SRC[$repo]}"
    local cfg_after="" pass_after=""
    if [[ -e "$CONFIG_FILE" ]]; then cfg_after=$(sha256sum -- "$CONFIG_FILE" 2>/dev/null) || true; fi
    if [[ -n "$PASSPHRASE_PATH" && -e "$PASSPHRASE_PATH" ]]; then pass_after=$(sha256sum -- "$PASSPHRASE_PATH" 2>/dev/null) || true; fi
    [[ "$cfg_before"  == "$cfg_after"  ]] || warn "this restore overwrote $CONFIG_FILE; this run still uses the config loaded at start, but re-check that file before your next run"
    [[ "$pass_before" == "$pass_after" ]] || warn "this restore overwrote $PASSPHRASE_PATH; this run still uses the passphrases loaded at start, but re-check it before your next run"
    (( rc == 0 )) && say "restored $repo in place (existing files overwritten, others left as-is)"
    return "$rc"
}

# The one gate every destructive extract path goes through. $1 is the -y flag and
# $2 states what is about to be overwritten, without trailing punctuation, so the
# same sentence serves the prompt and the refusal. -y proceeds, a terminal asks,
# and no terminal fails closed rather than prompting into a void.
_confirm() {
    local assume_yes="$1" what="$2" ans
    (( assume_yes )) && return 0
    [[ -t 0 ]] || die "$what; re-run with -y to confirm (no terminal to prompt)"
    read -rp "$what; continue? [y/N] " ans
    [[ "$ans" == [Yy] || "$ans" == [Yy][Ee][Ss] ]] || die "aborted"
}

# True if the path is a directory holding at least one entry.
_dir_nonempty() {
    [[ -d "$1" ]] || return 1
    local out
    out=$(ls -A -- "$1" 2>/dev/null) || true
    [[ -n "$out" ]]
}

# Every mounted drive holding this repo, in config order, one label per line.
# Empty output means no copy is reachable. Called with 'reasons' as $2 it prints
# why each unusable drive was dropped instead of the labels; both answers come
# from one loop so they cannot drift. Nothing is warned here, because extract
# needs one good copy: the reasons are asked for at the one moment they are the
# answer, when no copy is left. archive and check warn per drive; that is their job.
_extract_candidates() {
    local repo="$1" want="${2:-labels}" d probe parent blocked
    # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
    for d in ${REPO_DRIVES[$repo]}; do
        if ! mountpoint -q "$MOUNT_BASE/$d"; then
            if [[ "$want" == reasons ]]; then printf '%s\n' "$(_not_mounted "$d")"; fi
            continue
        fi
        # Before the directory test, not after it: an untraversable parent makes
        # the repo look absent, and "no repo there" would send you looking for a
        # missing copy when the copy is there and closed to you.
        parent=$(_repo_parent "$d")
        if [[ ! -r "$parent" || ! -x "$parent" ]]; then
            if [[ "$want" == reasons ]]; then printf '%s\n' "cannot look inside $parent; take the drive with '$CMD claim $d'"; fi
            continue
        fi
        probe=$(_repo_path "$d" "$repo")
        if [[ ! -d "$probe" ]]; then
            if [[ "$want" == reasons ]]; then printf '%s\n' "no $repo repo on $d (looked at $probe)"; fi
            continue
        fi
        # A copy borg cannot read is not a candidate. Calling borg on it buys a
        # Python traceback in place of a sentence, and the copy is unusable
        # either way, so it is dropped here with the remedy attached.
        blocked=$(_first_unreadable "$probe")
        if [[ -n "$blocked" ]]; then
            if [[ "$want" == reasons ]]; then printf '%s\n' "cannot read $blocked; take the drive with '$CMD claim $d'"; fi
            continue
        fi
        if [[ "$want" == labels ]]; then printf '%s\n' "$d"; fi
    done
    return 0
}

# Resolve the spec against one drive's copy and print the archive name.
# shellcheck disable=SC2030,SC2031  # per-drive borg env is subshelled by the caller's $( )
_archive_on_drive() {
    local repo="$1" drive="$2" spec="$3"
    local -x BORG_REPO; BORG_REPO=$(_repo_path "$drive" "$repo")
    _set_read_lock "$BORG_REPO" "$drive"
    local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"
    _resolve_archive "$BORG_REPO" "$spec"
}

# One complete restore attempt from a single drive's copy, into $dest or, with
# in_place set, back to the source folders. Returns 1 rather than exiting so the
# caller can fall through to the next copy; the subshell is needed here because
# this cd's, and it also keeps the borg environment off the next drive.
# shellcheck disable=SC2030,SC2031  # per-drive borg env is subshelled on purpose
_extract_from_drive() {
    local repo="$1" drive="$2" archive="$3" in_place="$4" dest="$5"; shift 5
    local paths=("$@")
    (
        local -x BORG_REPO; BORG_REPO=$(_repo_path "$drive" "$repo")
        _set_read_lock "$BORG_REPO" "$drive"
        local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"

        if (( in_place )); then
            _restore_folders_in_place "$repo" "$archive" "$drive" || exit 1
            exit 0
        fi

        mkdir -p "$dest" || { warn "cannot create $dest"; exit 1; }
        cd "$dest"       || { warn "cannot cd to $dest"; exit 1; }

        # restore_to is folder-level, so it applies to a whole-repo restore only.
        # With --path the caller has named archive paths directly and those land
        # under $dest, which is what the flag has always meant.
        if (( ${#paths[@]} == 0 )) && _repo_has_restore_to "$repo"; then
            say "extracting $repo from $drive (archive: $archive)..."
            _restore_folders_to_targets "$repo" "$archive" "$dest" || exit 1
            say "restored $repo"
            exit 0
        fi

        if (( ${#paths[@]} > 0 )); then
            say "extracting ${#paths[@]} path(s) from $repo on $drive (archive: $archive) into $dest..."
            borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --progress "::$archive" "${paths[@]}" || exit 1
            # The archive's own starting names are the answer to a miss, and this
            # repo's backup_data lines hold them, so print them rather than
            # sending the reader off to run list. An adopted repo has none.
            local hint rsrc roots=()
            while IFS= read -r rsrc; do
                [[ -n "$rsrc" ]] || continue
                roots+=("$(basename "$rsrc")")
            done <<< "${REPO_SRC[$repo]:-}"
            if (( ${#roots[@]} > 0 )); then
                hint="a path starts at one of this repo's own folder names, case included: $(_csv "${roots[@]}")"
            else
                hint="a path starts at a source folder's own name, case included; check it against '$CMD list $repo $drive'"
            fi
            # Every requested path missed. That is the same on every copy, since a
            # path is archive-relative, so exit 2 to tell the caller not to spend
            # the other drives on it; a partial hit is a real success and stays 0.
            _report_arrivals "$dest" "$hint" "${paths[@]}" || exit 2
            exit 0
        fi

        say "extracting $repo from $drive (archive: $archive) into $dest..."
        borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --progress "::$archive" || exit 1
        say "restored $repo into $dest"
        exit 0
    )
}

# Restore one repo's folders, each into the parent its restore_to names, and the
# rest into $dest in a single call. BORG_REPO and the passphrase environment must
# already be set, and the caller must not have cd'd anywhere that matters: this
# cd's per folder. Returns non-zero if any folder failed.
#
# Only reached when the repo has at least one restore_to line, so the plain bare
# `borg extract` is untouched for every other repo. A folder dropped from
# backup_data since the archive was written lands in $dest with the remainder.
_restore_folders_to_targets() {
    local repo="$1" archive="$2" dest="$3"
    local rc=0 src base target key
    local -a skip=()
    while IFS= read -r src; do
        [[ -n "$src" ]] || continue
        base=$(basename "$src")
        key="$repo"$'\n'"$src"
        target="${REPO_RESTORE_TO[$key]:-}"
        [[ -n "$target" ]] || continue
        say "  $base -> $target/"
        if ! ( mkdir -p "$target" && cd "$target" && borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} --progress "::$archive" "$base" ); then
            warn "failed to restore '$base' into '$target'"; rc=1
        fi
        skip+=( --exclude "sh:$base" --exclude "sh:$base/**" )
    done <<< "${REPO_SRC[$repo]}"
    # Everything without a restore_to, plus whatever else the archive holds, in
    # one call into $dest. The excludes above are what keep the folders already
    # placed from being written a second time here.
    say "  the rest -> $dest/"
    if ! ( mkdir -p "$dest" && cd "$dest" && borg extract ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} ${skip[@]+"${skip[@]}"} --progress "::$archive" ); then
        warn "failed to restore the remainder of $repo into $dest"; rc=1
    fi
    return "$rc"
}

# Order the reachable copies of a repo best-first, printing "<archive>TAB<drive>"
# lines. A copy whose -N or default spec will not resolve is dropped here rather
# than at extract time, so a stub or archive-less repo directory cannot win the
# pick; a literal archive name resolves trivially on every copy, so for that spec
# nothing is dropped and a copy lacking the archive is found at extract time.
#
# Ranking is on the archive name's first ten characters, the date, descending,
# with the config-order index breaking every tie. That ten-character slice is its
# own sort field rather than a character offset into the name: a key range that
# runs past a short field spills into the next one and inverts the tiebreak.
# Compare only the date, never the whole name: names come from borg's {now},
# evaluated once per borg create, and _archive_repo walks a repo's drives in
# config order, so within one run the drive written last always carries the later
# name, and a whole-name sort would rank write order rather than currency.
# Residual: two runs on one day landing on different drives tie on the date and
# go to config order, so the copy chosen can be hours older than the freshest.
_extract_plan() {
    local repo="$1" spec="$2"; shift 2
    local -a cand=("$@")
    local -a found=()
    local i=0 d a
    for d in "${cand[@]}"; do
        if a=$(_archive_on_drive "$repo" "$d" "$spec"); then
            found+=("$(printf '%.10s\t%03d\t%s\t%s' "$a" "$i" "$a" "$d")")
        else
            warn "$repo on $d: no usable archive there, skipping this copy"
        fi
        i=$(( i + 1 ))
    done
    (( ${#found[@]} > 0 )) || return 0

    printf '%s\n' "${found[@]}" | sort -t$'\t' -k1,1r -k2,2n | cut -f3,4
}

# Try each copy in the plan until one restores. Args: <repo> <in_place> <dest>
# <plan entry>... -- <path>..., a plan entry being "<archive>TAB<drive>".
# Returns 0 on success, 1 when every copy failed, 2 when none of the requested
# paths exist in the archive, which is the same on every copy.
_restore_from_plan() {
    local repo="$1" in_place="$2" dest="$3"; shift 3
    local -a plan=() paths=()
    local a seen=0
    for a in "$@"; do
        if   (( seen ));       then paths+=("$a")
        elif [[ "$a" == -- ]]; then seen=1
        else                        plan+=("$a"); fi
    done
    local i archive drive drc
    for (( i = 0; i < ${#plan[@]}; i++ )); do
        archive="${plan[i]%%$'\t'*}"; drive="${plan[i]##*$'\t'}"
        drc=0
        _extract_from_drive "$repo" "$drive" "$archive" "$in_place" "$dest" ${paths[@]+"${paths[@]}"} || drc=$?
        (( drc == 0 )) && return 0
        if (( drc == 2 )); then
            warn "none of the requested paths exist in $repo; nothing was restored, and the other copies hold the same paths"
            return 2
        fi
        warn "extraction of $repo from $drive failed; see borg output above"
        if (( in_place )); then
            warn "whatever borg wrote before the failure is already in place; the rest of $repo was not restored"
        else
            warn "$dest may now hold a partial $repo tree"
        fi
        if (( i + 1 < ${#plan[@]} )); then
            # Name the next copy's archive: it is resolved per drive, so it can be
            # a different point in time from the one that just failed.
            warn "falling through to ${plan[i+1]##*$'\t'} (archive: ${plan[i+1]%%$'\t'*}); it overwrites what the failed attempt left"
        fi
    done
    return 1
}

# Every repo with a mounted drive and a stored passphrase. -i restores in place,
# else into a per-repo subdirectory of RESTORE_PATH. `archive no` gates archiving,
# not restore, so archive-off repos are included. Args: <in_place> <assume_yes>;
# --path is rejected by the caller, so there are no paths to carry.
_extract_all_repos() {
    local in_place="$1" assume_yes="$2"; shift 2
    # No archive spec here, and that includes -1. Counting back needs a drive to
    # count on and no one drive holds every repo, so every -N but the newest is
    # unanswerable; -1 is the newest, which is what this form already does, so
    # accepting it would be a token that changes nothing. One rule, either way.
    (( $# == 0 )) \
        || die_usage "restoring every repo always uses the newest archive; name the repo to choose another: $CMD extract <repo> $1"
    need mountpoint
    parse_config
    _require_repos_valid
    (( in_place )) || _require_restore_path
    # Both branches ask: one omitted word is the whole difference between
    # restoring a repo and restoring every one of them. Asked above the
    # passphrase block, not below it, so nothing physical is asked of you for a
    # restore you have not agreed to yet. The two checks above are free and stay
    # first, so a confirmed run is not turned away a moment later.
    if (( in_place )); then
        _confirm "$assume_yes" "$IN_PLACE_WARNING"
    else
        _confirm "$assume_yes" "restoring every configured repo (${#REPOS[@]}) under $RESTORE_PATH"
    fi
    # An unreadable passphrase file would make every repo below report "no stored
    # passphrase" and end the run on a summary blaming drives and passphrases,
    # when the one cause is this file. Load the names up front and refuse.
    if [[ "$(_pass_source_kind)" != none ]]; then
        _load_pass_names
        (( _PASS_NAMES_UNREADABLE )) && die "refusing to restore: $PASSPHRASE_PATH is present but unreadable, so every repo would be skipped as if it had no passphrase; fix the cause in the warning above, or restore one repo at a time and let borg prompt"
    fi
    # The documented fallback (no passphrase file configured, borg asks) is
    # honoured here as everywhere else, keeping to one copy per repo so the count
    # of prompts is two per repo rather than one per drive plus one.
    local prompt_mode=0 pass_gap=""
    if _pass_prompt_mode; then
        prompt_mode=1
        _prompt_notice "no passphrase file configured" "each repo" twice
    elif [[ -z "$PASSPHRASE_PATH" ]]; then
        pass_gap="no passphrase file configured and no terminal to prompt"
    elif [[ ! -e "$PASSPHRASE_PATH" ]]; then
        pass_gap="no passphrase file at $PASSPHRASE_PATH"
    else
        pass_gap="no set_pass line in $PASSPHRASE_PATH"
    fi
    local rc=0 r rdest rfirst no_drive=0 prc w
    local -a restored=() skipped=() failed=() rcand=() rplan=() rwhy=()
    for r in "${REPOS[@]}"; do
        rcand=()
        mapfile -t rcand < <(_extract_candidates "$r")
        if (( ${#rcand[@]} == 0 )); then
            # Ask the same loop why, rather than calling every empty answer a
            # missing drive: a mounted drive without this repo on it, and one
            # whose copy cannot be read, are different problems with different
            # remedies, and the single-repo path already reports them properly.
            rwhy=()
            mapfile -t rwhy < <(_extract_candidates "$r" reasons)
            for w in ${rwhy[@]+"${rwhy[@]}"}; do
                case "$w" in *"is not mounted at"*) no_drive=1 ;; esac
            done
            skipped+=("$r ($(_csv "${rwhy[@]}"))")
            continue
        fi
        # Only in-place needs the folder list; a restore under RESTORE_PATH
        # unpacks whatever the archive holds and does not consult it.
        if (( in_place )) && [[ -z "${REPO_SRC[$r]:-}" ]]; then
            skipped+=("$r (no backup_data folders to put back)"); continue
        fi
        if ! (( prompt_mode )) && ! _repo_has_pass "$r"; then skipped+=("$r ($pass_gap)"); continue; fi
        rdest="$RESTORE_PATH/$r"
        if (( ! in_place && ! assume_yes )) && _dir_nonempty "$rdest"; then
            skipped+=("$r (target $rdest not empty; leftovers there would look like part of the restore, -y to accept)"); continue
        fi
        rplan=()
        if (( prompt_mode )); then
            # One borg list per drive is one prompt here, so keep to the first
            # reachable copy and accept that a stale one is not demoted.
            if rfirst=$(_archive_on_drive "$r" "${rcand[0]}" ""); then
                rplan=("$rfirst"$'\t'"${rcand[0]}")
            fi
        else
            mapfile -t rplan < <(_extract_plan "$r" "" "${rcand[@]}")
        fi
        if (( ${#rplan[@]} == 0 )); then skipped+=("$r (no usable archive on any copy; see the borg errors above)"); continue; fi
        prc=0
        _restore_from_plan "$r" "$in_place" "$rdest" "${rplan[@]}" -- || prc=$?
        if (( prc == 0 )); then restored+=("$r"); else rc=1; failed+=("$r"); fi
    done
    say ""
    local where="in place"; (( in_place )) || where="under $RESTORE_PATH"
    say "restored $where: ${restored[*]:-none}"
    (( ${#failed[@]} == 0 ))  || warn "failed: ${failed[*]} (see output above)"
    (( ${#skipped[@]} == 0 )) || warn "skipped: ${skipped[*]}"
    # Said once for the whole run rather than per repo: every repo shares one
    # MOUNT_BASE, so a config carried over from another machine misses every
    # drive at once, and the per-repo skip line has no room to say where it
    # looked. Only when a drive really was absent: saying it for a drive that is
    # mounted sends you to edit a setting that was never wrong.
    (( no_drive == 0 )) || warn "drives are looked for under $MOUNT_BASE; if they are mounted somewhere else, that is MOUNT_BASE in $CONFIG_FILE"
    if (( ${#restored[@]} == 0 && rc == 0 )); then
        die "nothing was restored; every repo was skipped for the reason listed against it above"
    fi
    exit "$rc"
}

# Path mode: the repo is named by path, borg prompts since nothing is stored, and
# the extract lands in the current directory. Args: <assume_yes> <args...> --
# <paths...>. This is the rescue route, so it never reads the config.
_extract_by_path() {
    local assume_yes="$1"; shift
    local -a args=() paths=()
    local a seen=0
    for a in "$@"; do
        if   (( seen ));       then paths+=("$a")
        elif [[ "$a" == -- ]]; then seen=1
        else                        args+=("$a"); fi
    done
    local repo="${args[0]}" spec=""
    if (( ${#args[@]} >= 2 )); then
        [[ "${args[1]}" == */* ]] && die_usage "path mode extracts one repo at a time; '${args[1]}' looks like a second path"
        spec="${args[1]}"
    fi
    (( ${#args[@]} <= 2 )) || die_usage "path mode takes one repo path and at most one archive spec; got ${#args[@]}"
    # With --path the collisions are known without asking borg; without it they
    # are not, and finding out would cost another listing and so another
    # passphrase prompt, so "is the directory empty" stands in for it.
    local rc=0 ppresent=() pp
    if (( ${#paths[@]} > 0 )); then
        for pp in "${paths[@]}"; do
            [[ -e "$PWD/$pp" ]] && ppresent+=("$pp")
        done
        (( ${#ppresent[@]} == 0 )) || _confirm "$assume_yes" "$PWD already contains: $(_csv "${ppresent[@]}")"
    elif _dir_nonempty "$PWD"; then
        _confirm "$assume_yes" "$PWD is not empty, and extracting here overwrites whatever the archive also holds"
    fi
    _borgex_path "$repo" "$spec" ${paths[@]+"${paths[@]}"} || rc=1
    exit "$rc"
}

# Label mode: a configured (or adopted) repo, restored into RESTORE_PATH
# or, with -i, back to its source folders. Args: <in_place> <assume_yes>
# <args...> -- <paths...>. The caller has already run parse_config.
_extract_by_label() {
    local in_place="$1" assume_yes="$2"; shift 2
    local -a args=() paths=()
    local a seen=0
    for a in "$@"; do
        if   (( seen ));       then paths+=("$a")
        elif [[ "$a" == -- ]]; then seen=1
        else                        args+=("$a"); fi
    done
    _require_repos_valid

    local repo="${args[0]}" spec="" drive_arg="" tok
    # Everything after the repo is classified by what it is, not by where it sits,
    # so 'extract docs -2 d1' and 'extract docs d1 -2' are the same command. Drive
    # labels are a closed set the config declares and archive names come from
    # borg's {now}, so the two cannot collide in practice; the drive test runs
    # first regardless, because it is a set lookup rather than a guess.
    for tok in ${args[@]+"${args[@]:1}"}; do
        if [[ "$tok" =~ ^-[0-9]+$ ]]; then
            [[ -z "$spec" ]] || die_usage "extract takes one archive spec; got '$spec' and '$tok'"
            spec="$tok"
        elif _repo_has_drive "$repo" "$tok"; then
            [[ -z "$drive_arg" ]] || die_usage "extract takes one drive; got '$drive_arg' and '$tok'"
            drive_arg="$tok"
        else
            case " $ALL_DRIVES " in
                *" $tok "*) _require_repo_drive "$repo" "$tok" ;;
            esac
            [[ -z "$spec" ]] || die_usage "extract takes one archive spec; got '$spec' and '$tok'"
            spec="$tok"
        fi
    done

    # Counting back is drive-relative: two copies hold different numbers of
    # archives whenever a drive was away for a run, so '-3' left to the ranker
    # would mean the third-newest on whichever copy it chose, which need not be
    # the one 'list <repo> <drive>' just showed. Counting back therefore names its
    # drive, and the restore is pinned to that copy with no fallthrough. -1 and a
    # literal archive name mean the same thing on every copy and need no drive.
    if [[ "$spec" =~ ^-([0-9]+)$ ]] && (( BASH_REMATCH[1] >= 2 )); then
        [[ -n "$drive_arg" ]] \
            || die_usage "'$spec' counts back from the newest, and each copy counts separately; name the drive, as in '$CMD extract $repo $spec <drive>' ($repo has: $(_repo_drives_csv "$repo"))"
    fi

    # Every mounted drive that has this repo. One copy failing does not end the
    # restore: the copies are ordered and tried in turn, unless a drive pinned one.
    local -a cand=()
    mapfile -t cand < <(_extract_candidates "$repo")
    if [[ -n "$drive_arg" ]]; then
        local c pinned=""
        for c in ${cand[@]+"${cand[@]}"}; do
            [[ "$c" == "$drive_arg" ]] && pinned="$c"
        done
        if [[ -z "$pinned" ]]; then
            mountpoint -q "$MOUNT_BASE/$drive_arg" \
                || die "$(_not_mounted "$drive_arg"); plug it in and re-run"
            die "no $repo repo on $drive_arg ($(_repo_path "$drive_arg" "$repo"))"
        fi
        cand=("$pinned")
    fi
    if (( ${#cand[@]} == 0 )); then
        warn "no mounted drive has the $repo repo"
        local why
        while IFS= read -r why; do warn "  $why"; done < <(_extract_candidates "$repo" reasons)
        exit 1
    fi

    # Ordering the copies costs one borg list per drive, and each list is one use
    # of the passphrase. Stored, that is silent; unstored, borg would prompt once
    # per drive before anything is extracted, so keep to the first copy and say so.
    local -a plan=()
    # shellcheck disable=SC2031  # reads the caller's environment, not the per-drive exports in the helpers
    if [[ -n "${BORG_PASSPHRASE:-}${BORG_PASSPHRASE_FD:-}${BORG_PASSCOMMAND:-}" ]] || _repo_has_pass "$repo"; then
        mapfile -t plan < <(_extract_plan "$repo" "$spec" "${cand[@]}")
        (( ${#plan[@]} > 0 )) || die "no copy of $repo holds a usable archive; run: $CMD check $repo"
    else
        # Two situations reach here and only one is "no entry for this repo": a
        # passphrase file that is present but could not be read leaves PASS_NAMES
        # empty for every repo, and calling that a missing entry sends you looking
        # for a set_pass line that is already there.
        local pass_why="no stored passphrase"
        (( _PASS_NAMES_UNREADABLE )) && pass_why="could not read $PASSPHRASE_PATH, so no passphrase is available"
        (( ${#cand[@]} == 1 )) \
            || warn "$pass_why for $repo; keeping to the first copy (${cand[0]}) rather than prompting once per drive"
        local how=once
        _spec_lists_first "$spec" && how=twice
        _prompt_notice "$pass_why" "$repo" "$how"
        local first_archive
        first_archive=$(_archive_on_drive "$repo" "${cand[0]}" "$spec") || exit 1
        plan=("$first_archive"$'\t'"${cand[0]}")
    fi

    local dest="$RESTORE_PATH"
    if (( in_place )); then
        (( ${#paths[@]} == 0 )) || die_usage "--in-place restores whole folders to their origins; drop --path"
        # Before the confirm, not after: there is nothing to overwrite, so asking
        # would be asking about an action that cannot happen.
        [[ -n "${REPO_SRC[$repo]:-}" ]] \
            || die "$repo has no backup_data folders, so there is nothing to put back; restore it under $RESTORE_PATH instead by dropping -i"
        _confirm "$assume_yes" "$IN_PLACE_WARNING"
    else
        _require_restore_path
        # A named repo restores flat into RESTORE_PATH, so its folders land
        # under their own names exactly as the archive holds them. That directory
        # is shared with whatever else lives there, so the only thing worth
        # gating on is a name this restore would overwrite. The whole-suite form
        # keeps its per-repo directory and its stricter non-empty gate, because
        # there the directory belongs to the restore and leftovers in it really
        # do read as part of it.
        local sp present=() b
        if (( ${#paths[@]} > 0 )); then
            for sp in "${paths[@]}"; do
                [[ -e "$dest/$sp" ]] && present+=("$sp")
            done
        else
            while IFS= read -r sp; do
                [[ -n "$sp" ]] || continue
                b=$(basename "$sp")
                [[ -e "$dest/$b" ]] && present+=("$b")
            done <<< "${REPO_SRC[$repo]:-}"
        fi
        if (( ${#present[@]} > 0 )); then
            _confirm "$assume_yes" "$dest already contains: $(_csv "${present[@]}"); these are overwritten, and anything else of that name left there stays"
        fi
    fi

    local rc=0
    _restore_from_plan "$repo" "$in_place" "$dest" "${plan[@]}" -- ${paths[@]+"${paths[@]}"} || rc=$?
    (( rc == 0 )) || warn "no copy of $repo could be restored"
    exit "$rc"
}

cmd_extract() {
    ensure_borg
    local paths=() args=() in_place=0 assume_yes=0
    while (( $# > 0 )); do
        case "$1" in
            -i|--in-place) in_place=1; shift ;;
            -y|--yes) assume_yes=1; shift ;;
            --path)
                (( $# >= 2 )) || die_usage "--path needs a value"
                [[ -n "$2" ]]  || die_usage "--path value must not be empty"
                paths+=("$2"); shift 2 ;;
            --path=*) die_usage "use '--path <value>'; '--path=value' is not supported" ;;
            -[0-9]*)                                         # -N archive spec (e.g. -3)
                [[ "$1" =~ ^-[0-9]+$ ]] || die_usage "unknown flag: $1"
                args+=("$1"); shift ;;
            -*) die_usage "unknown flag: $1" ;;
            *) args+=("$1"); shift ;;
        esac
    done

    # An in-place restore writes over the very folders an archive run reads, so
    # it takes the same lock those runs hold. A restore into RESTORE_PATH cannot
    # disturb a concurrent backup, so it is left unlocked.
    (( in_place )) && _acquire_lock

    # No repo named is the whole-suite restore. The test is on the first argument
    # rather than on the presence of a bare word anywhere, because everything is
    # classified relative to the repo and the repo comes first: '-1 docs' is a
    # mis-ordered command either way, and reading the repo out of the middle
    # would start accepting it.
    if (( ${#args[@]} == 0 )) || [[ "${args[0]}" =~ ^-[0-9]+$ ]]; then
        (( ${#paths[@]} == 0 )) || die_usage "with no repo named, extract restores whole repos; name the repo to use --path"
        _extract_all_repos "$in_place" "$assume_yes" ${args[@]+"${args[@]}"}
    fi

    # Routing. An argument containing a slash is a repo path and takes that route
    # before anything is read, so a restore from an explicit path still works
    # when the config is absent or malformed; that is the rescue property. A bare
    # word is decided after parsing, because _adopt_drive_repo needs the
    # MOUNT_BASE and ALL_DRIVES parse_config resolves. A declared repo beats a
    # same-named directory in the current folder.
    if [[ "${args[0]}" == */* ]]; then
        (( ! in_place )) || die_usage "--in-place needs a configured repo, not a repo path"
        _extract_by_path "$assume_yes" "${args[@]}" -- ${paths[@]+"${paths[@]}"}
    fi
    need mountpoint
    parse_config
    if _repo_declared "${args[0]}"; then
        :
    elif [[ -d "${args[0]}" ]]; then
        (( ! in_place )) || die_usage "--in-place needs a configured repo, not a repo path"
        _extract_by_path "$assume_yes" "${args[@]}" -- ${paths[@]+"${paths[@]}"}
    elif _adopt_drive_repo "${args[0]}"; then
        : # adopted: a repo directory of that name is on a mounted drive
    elif [[ " $ALL_DRIVES " == *" ${args[0]} "* ]]; then
        # Nothing else in this chain would name a drive as a drive: it is not a
        # repo, not a directory here, and not adoptable, so without this it
        # reports as a misspelt repo.
        die_usage "'${args[0]}' is a drive, not a repo; a drive says which copy to read, not what to restore (try: $CMD extract <repo> ${args[0]})"
    elif (( ${#REPOS[@]} == 0 )); then
        die "no repos are configured, and there is no directory at '${args[0]}' (relative to the current directory, $PWD); add repo_name blocks to $CONFIG_FILE, or give the repo's path"
    else
        die "'${args[0]}' is not a configured repo (check $CONFIG_FILE), and there is no directory of that name here either (relative to the current directory, $PWD)"
    fi
    _extract_by_label "$in_place" "$assume_yes" "${args[@]}" -- ${paths[@]+"${paths[@]}"}
}

## ─── list: one drive's copy, printed by borg ─────────────────────────
# borg is the thing that knows what is in a repo, so the wrapper's whole job here
# is to find the path, supply the passphrase and step aside. Everything after the
# drive label is handed to borg untouched, which is why --last, --first,
# --sort-by, --glob-archives, --format and --json all work without this script
# growing an opinion about any of them. Nothing is reformatted on the way out.
# local -x scopes the borg environment to this call, so no subshell is needed.
_list_one() {
    local repo="$1" dir="$2"; shift 2
    local -x BORG_REPO="$dir"
    _set_read_lock "$dir"
    local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"
    borg list ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} "$@"
}

cmd_list() {
    (( $# >= 2 )) || die_usage "usage: $CMD list <repo> <drive> [borg list flags...]"
    # The repo and the drive come first, so a leading flag is an ordering mistake
    # rather than a repo called --last.
    [[ "$1" != -* ]] || die_usage "$CMD list takes the repo and the drive first: $CMD list <repo> <drive> [borg list flags...]"
    local repo="$1" drive="$2"; shift 2
    ensure_borg; need mountpoint
    parse_config
    _require_repos_valid

    _require_declared_repo "$repo"
    _require_repo_drive "$repo" "$drive"
    mountpoint -q "$MOUNT_BASE/$drive" || die "$(_not_mounted "$drive"); plug it in and re-run"
    local dir; dir=$(_repo_path "$drive" "$repo")
    [[ -d "$dir" ]] || die "no $repo repo on $drive ($dir)"

    # shellcheck disable=SC2031  # reads the caller's environment, not the per-drive exports in the helpers
    if ! _repo_has_pass "$repo" \
        && [[ -z "${BORG_PASSPHRASE:-}${BORG_PASSPHRASE_FD:-}${BORG_PASSCOMMAND:-}" ]]; then
        _prompt_notice "no stored passphrase" "$repo"
    fi

    # borg's own exit status, not one of this script's. It is borg's answer.
    local rc=0
    _list_one "$repo" "$dir" "$@" || rc=$?
    exit "$rc"
}

## ─── check: deep-verify a repo on every drive ────────────────────────
# Fresh function scope per call so BORG_REPO and BORG_PASSCOMMAND never leak
# between repos.
_check_one() {
    local repo="$1" dir="$2"
    local -x BORG_REPO="$dir"
    local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"
    _set_read_lock "$dir"
    borg check ${RO_LOCKOPT[@]+"${RO_LOCKOPT[@]}"} "$dir"
}

cmd_check() {
    (( $# <= 2 )) || die_usage "usage: $CMD check [repo [drive]]"
    ensure_borg; need mountpoint
    parse_config
    _require_repos_valid
    # Load the names here, in the parent, for the reason _load_pass_names gives.
    # check is read-only and borg's own prompt is a legitimate fallback, so this
    # warns where archive and a whole-suite restore refuse.
    if [[ "$(_pass_source_kind)" != none ]]; then
        _load_pass_names
        if (( _PASS_NAMES_UNREADABLE )); then
            _prompt_notice "no passphrase is available from $PASSPHRASE_PATH" "every repo below"
        fi
    fi
    local -a todo=()
    local want_drive=""
    if (( $# >= 1 )); then
        _require_declared_repo "$1"
        todo=("$1")
    else
        todo=("${REPOS[@]}")
    fi
    # The drive is optional here and required by list, deliberately: losing the
    # all-drives view costs list a convenience, and would cost check the operation
    # itself, since deep-verifying every copy is the whole point of check.
    if (( $# == 2 )); then
        want_drive="$2"
        _require_repo_drive "${todo[0]}" "$want_drive"
    fi
    local repo drive dir rc=0 checked
    for repo in "${todo[@]}"; do
        checked=0
        # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
        for drive in ${REPO_DRIVES[$repo]}; do
            [[ -z "$want_drive" || "$drive" == "$want_drive" ]] || continue
            if ! mountpoint -q "$MOUNT_BASE/$drive"; then warn "$(_not_mounted "$drive")"; continue; fi
            dir=$(_repo_path "$drive" "$repo")
            if [[ ! -d "$dir" ]]; then warn "no $repo repo on $drive"; continue; fi
            say "checking $repo on $drive (reads the whole repo, may be slow)..."
            if _check_one "$repo" "$dir"; then say "$repo on $drive: ok"; checked=1
            else warn "$repo on $drive: FAILED check"; rc=1; checked=1; fi
        done
        (( checked )) || { warn "no mounted drive had the $repo repo to check"; rc=1; }
    done
    exit "$rc"
}

## ─── pass-change: rotate a repo's passphrase ─────────────────────────
_rotate_one() {
    local repo="$1" dir="$2"
    local -x BORG_REPO="$dir"
    local pc; pc=$(_repo_passcommand "$repo"); [[ -n "$pc" ]] && local -x BORG_PASSCOMMAND="$pc"
    borg key change-passphrase "$dir"
}

cmd_pass_change() {
    (( $# == 1 )) || die_usage "usage: $CMD pass-change <repo>"
    local repo="$1"
    ensure_borg; need mountpoint
    parse_config
    _require_repos_valid
    _require_declared_repo "$repo"
    [[ -t 0 ]] || die "pass-change prompts for the new passphrase; no terminal to prompt, so run it interactively"

    _acquire_lock

    local ready=() unmounted=() no_repo=() pairs=() drive dir
    # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
    for drive in ${REPO_DRIVES[$repo]}; do
        dir=$(_repo_path "$drive" "$repo")
        if ! mountpoint -q "$MOUNT_BASE/$drive"; then unmounted+=("$drive")
        elif [[ ! -d "$dir" ]]; then no_repo+=("$drive")
        else ready+=("$drive"); pairs+=("$drive"$'\t'"$dir"); fi
    done
    (( ${#unmounted[@]} == 0 )) || die "all drives for $repo must be mounted; missing: ${unmounted[*]}"
    (( ${#no_repo[@]} == 0 )) || warn "drives without a $repo repo (skipped): ${no_repo[*]}"
    (( ${#ready[@]} > 0 )) || die "no drives have a $repo repo; nothing to rotate"
    # borg key change-passphrase rewrites the repo's config and takes a lock, so
    # every copy must be writable before any of them is touched: a rotation that
    # lands on some drives and not others leaves the repo on two passphrases.
    _require_writable "rotating the passphrase for $repo" "${pairs[@]}"
    say "will rotate passphrase for $repo on: ${ready[*]}"

    _require_pass "$repo"

    local new_pass new_pass2
    read -rsp "new passphrase: " new_pass; printf '\n' >&2
    read -rsp "confirm new passphrase: " new_pass2; printf '\n' >&2
    [[ -n "$new_pass" ]] || die "passphrase must not be empty"
    [[ "$new_pass" == "$new_pass2" ]] || die "passphrases do not match"
    unset new_pass2

    local -x BORG_NEW_PASSPHRASE="$new_pass"
    local rotated=() failed=()
    for drive in "${ready[@]}"; do
        dir=$(_repo_path "$drive" "$repo")
        if _rotate_one "$repo" "$dir"; then say "rotated on $drive"; rotated+=("$drive")
        else warn "rotation failed on $drive"; failed+=("$drive"); fi
    done
    unset BORG_NEW_PASSPHRASE

    if (( ${#failed[@]} > 0 )); then
        warn "drives now on new passphrase: ${rotated[*]:-none}"
        warn "drives still on old passphrase: ${failed[*]}"
        warn "passphrase file NOT updated; fix the failed drives and re-run, or revert with:"
        warn "  BORG_PASSPHRASE='<new>' BORG_NEW_PASSPHRASE='<old>' borg key change-passphrase <repo-dir>"
        unset new_pass
        exit 1
    fi
    _update_pass_value "$repo" "$new_pass"
    unset new_pass
    say "done; $repo is on the new passphrase on ${#rotated[@]} drive(s), passphrase file updated"
}

## ─── rename: rename a repo across drives ─────────────────────────────
cmd_rename() {
    (( $# == 2 )) || die_usage "usage: $CMD rename <old-repo> <new-name>"
    local old="$1" new="$2"
    [[ "$old" != "$new" ]] || die_usage "old and new names are the same: $old"
    _validate_segment "new repo name" "$new"
    _reject_reserved_name "$new"
    need mountpoint
    parse_config
    _require_repos_valid
    _require_declared_repo "$old"
    ! _repo_declared "$new" || die "$new is already a configured repo; pick a different new name"

    _acquire_lock

    local ready=() unmounted=() no_repo=() collide=() pairs=() drive od nd parent
    # shellcheck disable=SC2086  # space-joined validated drive tokens; intentional split
    for drive in ${REPO_DRIVES[$old]}; do
        od=$(_repo_path "$drive" "$old"); nd=$(_repo_path "$drive" "$new")
        if ! mountpoint -q "$MOUNT_BASE/$drive"; then unmounted+=("$drive")
        elif [[ ! -d "$od" ]]; then no_repo+=("$drive")
        elif [[ -e "$nd" ]]; then collide+=("$drive")
        else ready+=("$drive"); pairs+=("$drive"$'\t'"$(_repo_parent "$drive")"); fi
    done
    (( ${#unmounted[@]} == 0 )) || die "all drives for $old must be mounted; missing: ${unmounted[*]}"
    (( ${#collide[@]} == 0 )) || die "new name $new already exists on: ${collide[*]}; pick another"
    (( ${#no_repo[@]} == 0 )) || warn "drives without a $old repo (skipped): ${no_repo[*]}"
    # A rename needs write permission on the directory holding the repo, not on
    # the repo itself, and it needs it on every drive before the first mv: a
    # rename that succeeds on d1 and is refused on d2 leaves the two drives on
    # different names with the config matching neither, to be repaired by hand.
    if (( ${#ready[@]} > 0 )); then
        _require_writable "renaming $old to $new" "${pairs[@]}"
        say "will rename $old -> $new on: ${ready[*]}"
    else
        say "no on-disk archives for $old; updating config and passphrase only"
    fi

    local renamed=() failed=()
    for drive in "${ready[@]}"; do
        od=$(_repo_path "$drive" "$old"); nd=$(_repo_path "$drive" "$new")
        if mv -T "$od" "$nd"; then say "renamed on $drive"; renamed+=("$drive")
        else warn "rename failed on $drive"; failed+=("$drive"); fi
    done
    if (( ${#failed[@]} > 0 )); then
        parent=$(_repo_parent "<drive>")
        warn "drives renamed: ${renamed[*]:-none}"
        warn "drives still on old name: ${failed[*]}"
        warn "config and passphrase file NOT updated; rename those back with:"
        warn "  mv $parent/$new $parent/$old"
        exit 1
    fi
    # The config first, then the passphrase label: _update_repo_name dies when its
    # line is missing while _update_pass_label only warns, so doing the fatal one
    # first means a failure leaves both files untouched rather than the passphrase
    # file already relabelled against a config that still says the old name.
    _update_repo_name "$old" "$new"
    _update_pass_label "$old" "$new"
    say "done; renamed on ${#renamed[@]} drive(s); config and passphrase file updated"
}

## ─── help and dispatch ───────────────────────────────────────────────
# The identity line is line 2 of this file, so the filename, the header comment
# and `version` output cannot drift from each other the way a separate VERSION
# constant would. Read from $SELF, the resolved path to this script.
_identity() {
    local line=""
    [[ -f "$SELF" ]] && line=$(sed -n '2p' -- "$SELF")
    line="${line#\#}"
    line="${line# }"
    [[ -n "$line" ]] || line="borg-simple, version unknown (could not read line 2 of $SELF)"
    printf '%s\n' "$line"
}

cmd_version() { _identity; }

usage() {
    # Unquoted heredoc so the command word is the name you invoked, not one baked
    # in here. Only two values expand: $CMD and $cfg. Every line in the commands
    # block opens with the same "  $CMD " prefix so the description column stays
    # aligned whatever the name is. $cfg is the config path with the home prefix
    # folded back to a tilde, which stops it drifting from CONFIG_FILE.
    local cfg="$CONFIG_FILE"
    [[ "$cfg" == "$HOME"/* ]] && cfg="~${cfg#"$HOME"}"
    cat <<EOF
$CMD: encrypted, versioned borg backups to USB drives.
Each repo is written to every one of its drives, so any single drive on its
own can restore it. What to back up, and to which drives, is set in the
config file, $cfg.

commands
  $CMD                      back up every repo
  $CMD <repo>               back up that one repo
  $CMD init [...]           set up a repo, or create configured ones
  $CMD extract [repo] [...] restore folders from a repo, or all
  $CMD list <repo> <drive>  show one copy's archives
  $CMD check [repo [drive]] verify copies against their hashes
  $CMD claim <drive>        take ownership of a drive's repo directories
  $CMD pass-change <repo>   change a repo's passphrase everywhere
  $CMD rename <old> <new>   rename a repo everywhere
  $CMD version              name and version of this script  (--version)
  $CMD help                 print this text                  (--help, -h)

init is the only command that writes to the config file.
    $CMD init <repo> <path>...  write the repo's block, store a passphrase,
                                then create it on each of its drives
    $CMD init <repo>            create an already-configured repo
    $CMD init                   the same, for every configured repo
    --drives <label>...         which drives a new repo lives on, default
                                every drive in ALL_DRIVES; every word after
                                it is read as a drive label, so put it last
    Creating a repo needs all of its drives mounted and writable, so every
    copy starts together; completing an existing repo does not.

extract puts a repo's folders under RESTORE_PATH, or back where they
came from with -i. Everything after the repo comes in any order. With no repo
named it restores every repo that has a mounted drive and a passphrase, whole,
and asks first; that form always uses the newest archive.
    -N                     which archive: -1 newest, -2 the one before it,
                           default -1
    <drive>                which copy to read; needed for -2 and lower,
                           since each drive counts back separately
    <archive>              an archive name from list, instead of -N
    --path <path>          restore only that path, a folder or a file;
                           repeat for more
    -i, --in-place         put folders back where they came from
    -y, --yes              overwrite without asking first
    <repo-path>            a repo directory instead of a repo name: no
                           config needed, borg prompts, files land here

claim is for a drive whose files belong to another user, which borg cannot
write to. It takes the drive's mountpoint and its repo directories, never
the whole drive, and shows what it will take before asking. It needs a
terminal, so it never runs under cron.

list hands borg everything after the drive, so --last 20, --sort-by,
--json and the rest work exactly as borg documents them.

examples
    $CMD init docs ~/Documents ~/Notes
        set up a repo named docs holding those two folders
    $CMD
        back up every repo now
    $CMD list docs d1
        what the docs copy on d1 holds
    $CMD extract docs -2 d1 --path Notes
        restore just Notes into RESTORE_PATH, taking the archive one
        back from newest on d1; count on the same drive you listed
    $CMD extract docs -i
        put every folder of docs back where it came from
    $CMD claim d2
        take ownership of d2's repo directories after a permission error

config, $cfg
    Per repo: repo_name, backup_data, backup_drives, keep, exclude,
    include_only, include_only_in, restore_to, compression, archive.
    Above the blocks: MOUNT_BASE, RESTORE_PATH, REPO_SUBDIR,
    PASSPHRASE_PATH, ALL_DRIVES, SRC_HOME.
    To remove a repo, delete its block; its archives and its set_pass
    line stay. The guide has the format and cron use.
EOF
}

# Hidden self-test, run before treating a version final (alongside shellcheck).
# The subcommand list, the dispatch case and the help banner are three hand-kept
# things, so assert every SUBCOMMANDS entry has a cmd_ handler, a branch in the
# dispatch case, and a line in the banner's commands block.
_selfcheck() {
    local w fn rc=0 help_text dispatch
    help_text=$(usage)
    dispatch=$(declare -f main)
    for w in "${SUBCOMMANDS[@]}"; do
        fn="cmd_${w//-/_}"
        declare -F "$fn" >/dev/null || { warn "no handler $fn for subcommand '$w'"; rc=1; }
        # Anchored to a case pattern, so the word occurring in a message or in
        # another function's name cannot stand in for a missing branch.
        grep -qE "(^|[[:space:]|])${w}[[:space:]]*[|)]" <<<"$dispatch" \
            || { warn "subcommand '$w' has no branch in main's dispatch case"; rc=1; }
        # Anchored to a commands-block line, not to the word anywhere in the text:
        # 'list' and 'version' both occur in the banner's prose, so a loose match
        # would pass with the command's own line deleted.
        grep -q "^  $CMD $w\b" <<<"$help_text" || { warn "subcommand '$w' missing from the help banner's commands block"; rc=1; }
    done
    (( rc == 0 )) && say "selfcheck ok (${#SUBCOMMANDS[@]} subcommands)"
    return "$rc"
}

main() {
    [[ $- == *x* ]] && die "refusing to run under bash -x: it would trace passphrases; debug with a targeted set -x around a non-secret section"
    # Personal tool: bare invocation runs the archive directly, no confirm, under
    # the spec's CLI carve-out; an unknown bare token is treated as a repo name.
    # An argument that is the empty string (an unset "$REPO" in a cron line) is a
    # mistake, not a full run, so it is rejected below rather than escalated.
    # cmd_archive exits on every path rather than returning, so the exit here is
    # a net for the day that stops being true, not a live line.
    # shellcheck disable=SC2317  # deliberately unreachable while cmd_archive always exits
    (( $# == 0 )) && { cmd_archive; exit; }
    case "${1:-}" in
        _emit-pass)  shift; _emit_pass "$@" ;;   # hidden: borg's passcommand re-invokes this to print one repo's passphrase
        _pass-names) shift; _pass_names "$@" ;;  # hidden: prints repo names that have a stored passphrase
        _selfcheck)  _selfcheck ;;               # hidden: assert the dispatch table is internally consistent
        -h|--help|help) usage ;;
        version|--version) cmd_version ;;
        init)        shift; cmd_init "$@" ;;
        extract)     shift; cmd_extract "$@" ;;
        list)        shift; cmd_list "$@" ;;
        check)       shift; cmd_check "$@" ;;
        claim)       shift; cmd_claim "$@" ;;
        pass-change) shift; cmd_pass_change "$@" ;;
        rename)      shift; cmd_rename "$@" ;;
        "")          die_usage "empty repo name; run '$CMD' with no arguments to back up every repo, or name a repo" ;;
        -*)          die_usage "unknown option: $1 (try: $CMD --help)" ;;
        *)           cmd_archive "$@" ;;
    esac
}

if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
    main "$@"
fi

Borg-simple core

This script is a smaller, bare-case version of the Borg-simple script. It has only two functions: archive one folder to one drive, or extract one repo from one drive. The main motivation is that experts can review quickly the core functions. It has no repo list, no retention, no in-place restore, no config file.

#!/bin/bash
# borg-super-simple, v9

set -euo pipefail
case $- in *x*) printf 'refusing to run under set -x\n' >&2; exit 1 ;; esac

CONFIG_FILE="$HOME/.borg-config"
MOUNT_BASE="/media/$(id -un)"
REPO_SUBDIR=""
PASSPHRASE_PATH=""
ALL_DRIVES=""
SELF=$(readlink -f -- "$0")

declare -A SRC=() DRIVES=() EXCL=() ALLOW=() ARCH=() PASS=()
declare -a REPOS=()
CUR=""

say()  { printf '%s\n' "$*"; }
warn() { printf '%s\n' "$*" >&2; }
die()  { printf '%s\n' "$*" >&2; exit 1; }

# shellcheck disable=SC2088  # '~/' is a literal pattern here, expanded by hand
_abs() { case "$1" in /) printf '/\n' ;; /*) printf '%s\n' "${1%/}" ;;
                      '~') printf '%s\n' "$HOME" ;;
                      '~/'*) printf '%s\n' "$HOME/${1#'~/'}" ;;
                      *) printf '%s\n' "$HOME/${1%/}" ;; esac; }

repo_name()       { CUR="$1"; REPOS+=("$1"); ARCH["$1"]=1; }
backup_data()     { local p; for p in "$@"; do SRC[$CUR]+="$(_abs "$p")"$'\n'; done; }
backup_drives()   { DRIVES[$CUR]="$*"; }
exclude()         { local g; for g in "$@"; do EXCL[$CUR]+="$g"$'\n'; done; }
include_only()    { ALLOW[$CUR]=1; }
include_only_in() { ALLOW[$CUR]=1; }
archive()         { [[ "${1:-}" == no ]] && ARCH[$CUR]=0; return 0; }
set_pass()        { PASS["$1"]="$2"; }
keep() { :; }; restore_to() { :; }; compression() { :; }

require_private() {
    local mode
    [[ -r "$1" ]] || die "$1 not found or not readable"
    mode=$(stat -c '%a' "$1") || die "cannot stat $1"
    (( (8#$mode & 8#77) == 0 )) || die "$1 is reachable by group or other (mode $mode); run: chmod 600 $1"
}

# The version gate the full tool has, for the same reason and with more at stake
# here: this is the tool you reach for on the machine where things have gone
# wrong, so it must not be the one that quietly writes archives of a shape the
# full tool cannot restore. Below 1.4 the /./ anchor is ignored without a word,
# and every folder is stored under its full path instead of its own name, so
# `backup extract -i` later asks borg for 'Documents' and the archive holds
# 'home/john/Documents'. 2.x changed the archive layout and the command surface.
_require_borg() {
    command -v borg >/dev/null 2>&1 || die "borg not found; install it first"
    local out ver major minor
    out=$(borg --version 2>/dev/null) || die "borg is installed but 'borg --version' failed"
    ver="${out##* }"
    major="${ver%%.*}"; minor="${ver#*.}"; minor="${minor%%.*}"
    case "$major$minor" in
        ''|*[!0-9]*) die "cannot read a borg version out of '$out'; expected something like 'borg 1.4.0'" ;;
    esac
    if (( major >= 2 )); then
        die "borg $ver is 2.x, which changed the archive layout; install borg 1.4 or newer and below 2.0"
    fi
    if (( major < 1 || minor < 4 )); then
        die "borg $ver is too old: 1.4 is the first version that stores a folder at the top of the archive under its own name, and older borg ignores that silently, so these archives would hold full paths and 'backup extract -i' would not find the folders. Install borg 1.4 or newer (Debian trixie and Devuan Excalibur ship it; bookworm needs bookworm-backports)"
    fi
    return 0
}

load_config() {
    _require_borg
    require_private "$CONFIG_FILE"
    # An unimplemented directive must not abort the source, but a mistyped one
    # must not vanish either: it would silently drop a folder from the backup.
    # shellcheck disable=SC2317  # bash calls this indirectly while sourcing
    command_not_found_handle() { warn "ignoring unknown config directive '$1'"; return 0; }
    # shellcheck source=/dev/null
    source "$CONFIG_FILE" || die "failed to load $CONFIG_FILE"
    unset -f command_not_found_handle
    MOUNT_BASE=$(_abs "$MOUNT_BASE")
    [[ -d "$MOUNT_BASE" ]] || die "MOUNT_BASE '$MOUNT_BASE' is not a directory"
    [[ -z "$PASSPHRASE_PATH" ]] || PASSPHRASE_PATH=$(_abs "$PASSPHRASE_PATH")
}

# borg re-invokes _emit-pass through this, so the cleartext goes from gpg or cat
# straight into borg and never enters this process. Empty when there is no
# passphrase file, and borg then prompts.
_passcommand() {
    [[ -n "$PASSPHRASE_PATH" && -e "$PASSPHRASE_PATH" ]] || return 0
    case "$BASH$SELF$PASSPHRASE_PATH" in
        *[[:space:]\'\"\\]*) die "borg splits the passcommand without a shell, so these paths must have no whitespace, quotes or backslashes: '$BASH' '$SELF' '$PASSPHRASE_PATH'" ;;
    esac
    printf '%s %s _emit-pass %s %s\n' "$BASH" "$SELF" "$PASSPHRASE_PATH" "$1"
}

# Plain or GPG is decided by content, not by filename, so the same file the full
# tool reads works here whatever it is called.
_emit_pass() {
    (( $# == 2 )) || { printf '_emit-pass needs <file> <repo>\n' >&2; exit 2; }
    local ct
    require_private "$1"
    if LC_ALL=C grep -qaE '^[[:space:]]*set_pass[[:space:]]' -- "$1"; then
        ct=$(cat -- "$1") || exit 1
    elif LC_ALL=C grep -qa '[^[:print:][:space:]]' -- "$1" \
      || LC_ALL=C grep -qa '^-----BEGIN PGP MESSAGE-----' -- "$1"; then
        ct=$(gpg --quiet --decrypt "$1") || exit 1
    else
        ct=$(cat -- "$1") || exit 1
    fi
    eval "$ct"; unset ct
    [[ -n "${PASS[$2]:-}" ]] || { printf 'no passphrase for %s in %s\n' "$2" "$1" >&2; exit 1; }
    printf '%s' "${PASS[$2]}"
}

do_backup() {
    (( $# == 0 )) || die "usage: borg-super-simple backup"
    local repo drive src g root pc rc=0 crc
    local -a srcs pats roots
    for repo in ${REPOS[@]+"${REPOS[@]}"}; do
        (( ${ARCH[$repo]} )) && [[ -n "${SRC[$repo]:-}" ]] || continue
        [[ -z "${ALLOW[$repo]:-}" ]] || { warn "$repo uses include_only; this tool cannot apply allowlists, so it is skipped (use the full tool)"; rc=1; continue; }
        srcs=(); roots=(); pats=()
        while IFS= read -r src; do
            [[ -n "$src" ]] || continue
            srcs+=( "$(dirname "$src")/./$(basename "$src")" )
            roots+=( "${src#/}" )
        done <<< "${SRC[$repo]}"
        # borg matches patterns against the source path with its leading slash
        # off, never against the name the item is stored under, so anchor every
        # glob on each source root. '**/' also matches zero directories.
        while IFS= read -r g; do
            [[ -n "$g" ]] || continue
            if [[ "$g" == /* ]]; then pats+=( --pattern "- sh:${g#/}" )
            else for root in "${roots[@]}"; do pats+=( --pattern "- sh:$root/**/$g" ); done; fi
        done <<< "${EXCL[$repo]:-}"
        pc=$(_passcommand "$repo")
        for drive in ${DRIVES[$repo]:-$ALL_DRIVES}; do
            mountpoint -q "$MOUNT_BASE/$drive" || { warn "$drive not mounted; skipping $repo there"; rc=1; continue; }
            say "archiving $repo to $drive..."
            crc=0
            (
                export BORG_REPO="$MOUNT_BASE/$drive${REPO_SUBDIR:+/$REPO_SUBDIR}/$repo"
                [[ -z "$pc" ]] || export BORG_PASSCOMMAND="$pc"
                [[ -d "$BORG_REPO" ]] || borg init --encryption=repokey || exit 1
                borg create ${pats[@]+"${pats[@]}"} "::{now}" "${srcs[@]}"
            ) || crc=$?
            (( crc <= 1 )) || { warn "$repo on $drive failed (borg exited $crc)"; rc=1; }
        done
    done
    return "$rc"
}

do_extract() {
    (( $# == 2 )) || die "usage: borg-super-simple extract <repo> <drive>"
    local path latest pc
    local -a lock=()
    path="$MOUNT_BASE/$2${REPO_SUBDIR:+/$REPO_SUBDIR}/$1"
    [[ -d "$path" ]] || die "no repo at $path"
    [[ -w "$path" ]] || { lock=(--bypass-lock); warn "$path is not writable; reading with borg's lock bypassed"; }
    pc=$(_passcommand "$1")
    export BORG_REPO="$path"
    [[ -z "$pc" ]] || export BORG_PASSCOMMAND="$pc"
    latest=$(borg list ${lock[@]+"${lock[@]}"} --last 1 --short) || die "could not list $path"
    [[ -n "$latest" ]] || die "no archives in $path"
    say "extracting $1 from $2 (archive: $latest) into $PWD..."
    borg extract ${lock[@]+"${lock[@]}"} "::$latest"
}

case "${1:-}" in
    _emit-pass) shift; _emit_pass "$@" ;;
    backup)     shift; load_config; do_backup "$@" ;;
    extract)    shift; load_config; do_extract "$@" ;;
    *)          die "usage: borg-super-simple backup | borg-super-simple extract <repo> <drive>" ;;
esac

Manual Borg and GPG

Part 1 and Part 2 hand your data to a shell script somebody else wrote. This page explains the same job without the script, for those who prefer to minimize trust.

It is not a hand-written copy of the script. Most of what the script does is convenience you will not miss with two pendrives and a bit of patience. What you will miss is the handful of things borg does not do unless you ask Those are what this page is about.

It targets borg 1.2 through 1.4, which is what Debian-family systems ship today. Check yours with borg --version.

What this page assumes

Getting started with Borg covers installing borg, creating a repository, making an archive, listing what is in one, and extracting it again. Read that first.

Once you can do those four things, you can back up. What you cannot yet do is trust the result, keep it from filling the drive, or run it without typing a passphrase over and over.

Several drives are several repositories

This is the idea everything else on this page depends on, and it is the one people get wrong.

Two pendrives holding a repo of the same name are not a mirrored pair. They are two independent repositories that happen to share a name, each with its own archives, its own key, and its own passphrase. Nothing keeps them in step except you running the same commands against both.

Three consequences, all of which bite eventually.

Changing a passphrase changes it on one repository. Do the second drive in the same sitting or you will have two passphrases for one name and no record of which drive holds which.

Renaming is mv on each drive, one at a time.

And a drive that was not plugged in last month is a month behind, silently, because nothing anywhere is tracking that. The only way to know is to look.

This is also why repokey is the right encryption mode for drives you carry. It keeps the key inside the repository, protected by your passphrase, so the drive is openable on any machine with nothing but the passphrase in your head. keyfile keeps the key in ~/.config/borg/keys instead, which means a stolen drive is useless without a file from your laptop, and a dead laptop leaves you with an unopenable drive.

Passphrases

Three rungs. Start on the first, move up when the typing annoys you more than the risk does.

Rung one, type it

Borg asks, you type, nothing is stored anywhere. This is the most secure arrangement there is and for two pendrives it is entirely workable.

One thing to know before you decide it is unworkable, and one before you decide it is fine.

It is not one prompt per backup. Borg asks once per operation that needs the repository key, and a careful run of one repo is three of those: the archive, the prune, and the check. Two drives makes six. That is the number people find out about on their second evening, not their first.

And every one of those prompts is a moment you cannot walk away from, which rules out running anything on a timer.

Rung two, a file borg reads for you

Put the passphrase in a file, tell borg how to fetch it:

(umask 077; printf '%s' 'your long passphrase' > ~/.borg-pass-documents)
export BORG_PASSCOMMAND='cat /home/john/.borg-pass-documents'

Borg runs that command whenever it needs the passphrase and reads the answer from its output. The secret exists only inside that short-lived cat, never in borg’s environment.

The umask 077 matters as much as the rest. The file’s permissions are the only thing protecting it, so it is chmod 600 and stays that way.

What this buys: no typing, and a run you can leave alone. What it costs: anyone with root on the machine can read the passphrase, and so can anyone holding an unencrypted backup of your home directory.

This is the rung that works from cron, and it is the only one that does.

Rung three, a gpg-encrypted file

gpg --encrypt --recipient you@example.com --output ~/.borg-pass-documents.gpg ~/.borg-pass-documents
chmod 600 ~/.borg-pass-documents.gpg
gpg --decrypt ~/.borg-pass-documents.gpg

That last line is not optional. Check the file decrypts to what you expect before you delete the plaintext one.

Then:

export BORG_PASSCOMMAND='gpg --quiet --decrypt /home/john/.borg-pass-documents.gpg'

Be clear about what this does and does not buy you.

It buys protection at rest. The passphrase is no longer readable text on your disk, so a stolen drive of your home directory, or a stray copy of an old backup, no longer hands it over.

It does not buy unattended running. Borg re-runs that command for every operation that needs the key, and each run needs gpg able to decrypt right then. That means the agent primed, or you answering a prompt, or a hardware token touched, per operation. On a machine you are sitting at, the agent’s cache hides all of it. On a timer at four in the morning it fails, or worse, hangs.

So rung three is for the machine you use, and rung two is for the machine that backs itself up while you sleep. They are not an upgrade path; they are two different jobs. Choosing rung three and then wondering why cron stopped working is the single most common way to end up with a month of backups that did not happen.

The one to avoid

BORG_PASSPHRASE puts the passphrase directly in the environment.

Written inline, as BORG_PASSPHRASE=hunter2 borg create ..., it can show up in the process list where every user on the machine can read it. Borg’s own FAQ warns about exactly this. Written as an export, it sits in your shell history and in the environment of every process you start for the rest of the session.

There is a fourth option, BORG_PASSPHRASE_FD, which has borg read from a file descriptor you open and is the safest of all of them, at the cost of being awkward to type. Worth knowing it exists. Note also that BORG_PASSPHRASE overrides BORG_PASSCOMMAND, which overrides BORG_PASSPHRASE_FD, so a forgotten export from an experiment will quietly win over the arrangement you think you are using.

Three things about the passcommand

Borg runs it without a shell. Environment variables like $HOME are expanded, but ~ is not, so write the path out in full.

It is split the way a shell splits a command line, so a quoted path with a space in it does survive. What does not survive is anything that needs a shell to mean anything: no pipes, no redirection, no command substitution. Put those inside sh -c 'the whole thing' and make that the passcommand.

And by hand, one small file per repository is easier than one file holding them all. The script keeps a combined file because it has parsing code; you do not, and cat or gpg -d on a single-line file needs none.

Filtering: order is the whole thing

Borg evaluates --pattern arguments in the order you give them, first match wins, and anything unmatched is kept.

That one rule is the only part worth memorising, because it decides something that matters. Put your drops before your keeps. An exclude listed first cannot be undone by an allowlist after it, which is what you want for anything you are deliberately keeping out of a backup. The other way round, an allowlist can pull back in the very thing you were excluding.

Check a pattern set before you trust it:

borg create --list --dry-run "::test" /home/john/Documents --pattern '- sh:**/cache/**'

An allowlist that matches nothing produces a perfectly valid, completely empty archive, and no error. --dry-run is how you find that out on the day you write it rather than the day you need it.

Retention: prune, then compact

Left alone, a repository grows until the drive is full. Pruning deletes archives by rule:

borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly -1

A negative number means no limit, so -1 keeps that tier forever. Rules are applied shortest interval first, and an archive kept by one rule does not count towards the next. Since borg 1.2 the oldest archive is kept whenever a rule could not otherwise meet its target, so your first archive keeps ageing until a newer one qualifies to replace it.

There is no undo, so look before you leap:

borg prune -v --list --dry-run --keep-daily 7 --keep-weekly 4

Then the step that is easy to miss and makes the whole exercise pointless if you do:

borg compact

prune marks archives as deleted. compact is what rewrites the repository and hands the space back to the filesystem. Prune without compact frees nothing at all, and the drive fills anyway while you are looking at a shorter archive list and assuming otherwise.

Two smaller things.

Prune also clears out checkpoint archives, the partial ones borg leaves behind when a run is interrupted, so if you never prune those accumulate.

And when you run borg create --stats, the number worth reading is the deduplicated size. That is what this archive actually cost you on the drive. The original and compressed sizes describe the files, not the growth.

Verification: two kinds of check

Prune is about the drive filling up. Check is about the drive lying to you, which is the failure that matters and the one nothing announces.

Fast, straight after writing an archive:

borg check --archives-only --last 1

That reads the newest archive’s metadata and takes seconds. It catches the write that did not land, which is the common case: a drive pulled early, a cable knocked, a filesystem that went read-only halfway through. Run it every time. It costs nothing.

Slow, occasionally:

borg check /media/john/d1/documents

That reads the whole repository and verifies every chunk. On a large repo over USB it can take hours, so this is a thing you start and walk away from. It is what finds bit rot on an ageing pendrive, and pendrives do rot.

Run the slow one before you rely on a drive you have not touched in a year, and after any incident that involved the drive being unplugged mid-write. A backup you have never verified is a belief.

Restoring: mount, not just extract

Getting started covers borg extract, and extract is the right tool for putting a whole folder back.

It is the wrong tool for the far more common case, which is that you want one file and you are not sure which archive has the version you mean. For that, mount the archive:

mkdir -p ~/mnt
borg mount /media/john/d1/documents::2026-07-28T20:30:00 ~/mnt

It appears as a read-only filesystem. Open it in a file manager, look at the file, copy out the one you want, then:

borg umount ~/mnt

Leave off the ::archive and mount the whole repository instead, and every archive appears as its own directory. That is the quickest way to answer “which version of this do I actually want”, and it beats extracting three candidates into three directories.

Two caveats. It needs FUSE and enough memory and temp space for the archive’s metadata. And it does not reproduce everything, some filesystem flags and ACLs among them, so use mount to find things and extract to restore them properly.

Whichever you use, open a few of the restored files afterwards. An extract that exits zero has told you the archive was readable, not that its contents are what you remember.

If you took rung three, get the key out of the machine

This is the part with no recovery, so it goes before anything else you do with gpg.

A key whose only copy is inside an encrypted repository cannot be reached when you need it. Opening the repository needs the passphrase, the passphrase is encrypted to the key, and the key is in the repository.

gpg --export-secret-keys --armor you@example.com > secret-key.asc
gpg --export --armor you@example.com > public-key.asc
gpg --gen-revoke you@example.com > revoke.asc

Export the public key as well as the secret one. It is not sensitive, and you need it for the paper copy below.

For paper, paperkey strips out everything reconstructible from the public key, leaving a much smaller amount to print:

gpg --export-secret-keys you@example.com | paperkey --output secret-key-paper.txt

The default output is a hex dump with line numbers and a checksum on each line, meant to be printed and typed back in, and to tell you which line you mistyped when you do. --output-type raw also exists, but that is binary, for feeding a barcode or QR generator, and is not something you can print and read.

Rebuilding needs the printout and the public key together:

paperkey --pubring public-key.asc --secrets secret-key-paper.txt --output secret-key.asc

Two things people assume about a paper copy that are not true.

It does not remove the passphrase. If the secret key was passphrase-protected, the rebuilt one is too, so paper rescues you from a dead disk and not from a forgotten passphrase.

And it is not self-contained. Without the public key there is nothing to rebuild against, so keep a copy of public-key.asc with the printout.

Store all of it away from the backup drives.

Where this ends up

Doing this by hand for two drives and one folder is perfectly reasonable, and you now know the parts borg will not do for you.

Doing it for four drives and five folders means running the same commands with different arguments, in the right order, remembering the compact after the prune and the check after the create, and noticing the drive that was not plugged in. Which is to say you will write it down, and then you will write a loop, and at that point you have a backup script.

The only question left is whose, and whether you have read it.

Borg 2.0

Borg 2.0 changes the surface enough that the commands here will not run unchanged.

borg init becomes borg repo-create. borg list splits into borg repo-list for archives and borg list for contents. The repo::archive syntax is replaced by -r <repo> plus a separate archive name. borg prune acts on an archive series rather than defaulting to the whole repository.

Debian-family systems still ship the 1.x line at the time of writing. Check borg --version rather than assuming either way.

Creating passphrases

A few passphrases in your life must not reside in a password manager: the passphrase that unlocks your GPG key and with it your password store, and the passphrase on a backup repository, the passphrase that decrypts your disk at boot. Each must exist before the tooling that depends on it, each must be reproducible by you from paper or memory, and each is unrecoverable by design if lost. Every other password you ever need should come from a generator inside your manager; this document is only for the few that cannot. This is the home for creating such a passphrase and keeping it; the tools that consume them live in their own documents, using-pass.md for the password store, choosing-backup-tools.md for backups, devuan-secure-workstation.md for the disk.

The stakes, stated plainly

That passphrase is not a small detail; it is the key to everything behind it. If you lose it, what it protects is as inaccessible as if it did not exist. No vendor can reset it and no support line can recover it, and that absence of a recovery path is precisely what makes the encryption worth having.

Write it on paper

Not in a password manager, not in a note on your phone, not in a document on your computer. Write it on paper. Make more than one copy. Store each copy somewhere it would not be found by a casual intruder but would be found by someone who needed it urgently: a sealed envelope in a filing cabinet, a safe-deposit box, a trusted family member’s home. Do not photograph it. Do not type it into any electronic device for the purpose of storage.

This is exactly how hardware Bitcoin wallets work. When you set up a hardware wallet, it generates a list of 12 or 24 random words, your seed phrase, and asks you to write them down on paper. That piece of paper is the only backup of your funds. It has no remote attack surface. The security model is entirely physical, which is a trade-off: you are immune to remote attackers and you are vulnerable to fire, flood, and theft. People who take this seriously make multiple copies and store them in separate locations. The same logic applies here.

Do not invent the passphrase yourself

We humans reach for patterns without realising it. Use Diceware, a method developed in 1995 that lets genuine randomness pick words from a curated list and string them together. The result looks like this: verso droopy flair unmasked shrimp oncoming. Strange, memorable, and effectively impossible to guess. There are three ways to run it, in descending order of strength.

The easy way: let software roll for you

The simplest route is a generator that picks the words for you from the EFF list using your operating system’s randomness. On Devuan or any Debian-based Linux, install it and run it:

sudo apt install diceware && diceware

On Windows or macOS, install Python and then the same tool:

pip install diceware && diceware

KeePassXC’s built-in password generator also has a passphrase mode that does the same job through a graphical interface on all three systems. Six words is the default. This is fine for almost everyone, with one caveat worth stating plainly: the phrase is generated inside your computer, so you are trusting that the machine’s random number generator is sound and that nothing on the machine is watching.

The strong way: roll the dice yourself

If you would rather trust nothing but physics, generate the phrase by hand and let the computer see only the finished result. You need two things: one balanced six-sided die, and the EFF long wordlist printed on paper, which has exactly 7,776 entries, that is 6 to the power of 5, and can be printed from the EFF’s site. That list size is the reason five rolls produce exactly one word with nothing wasted. Roll the die five times and read the results as a five-digit number, for example 4-2-6-1-3. Find that number, 42613, in the printed list, and the word beside it is your first word. Repeat five rolls for each further word. Six words means thirty rolls and about 77 bits, the same strength the software gives, except now no random number generator and no software stood between the dice and your paper. One rule does the heavy lifting: write down the word you land on every time, even a dull or awkward one, and never re-roll for a nicer result, because a re-roll is your own preference climbing back in and quietly draining the randomness you are there to collect. Keep the rolls in order, because the sequence is the number. This is the method to use if you are treating the passphrase as long-term and high-value, and it is the one to default to.

The weak fallback: pointing into a dictionary

You may be tempted by an offline method that needs neither dice nor an install: hold a thick dictionary closed, open it blind, and drop a finger on the page, taking the nearest whole word. This is the weakest of the three and is dominated by the dice method, which is also offline but countable. The same discipline applies, keep the word you land on rather than flipping to a nicer one, but discipline alone does not save it. A blind finger lands evenly across the area of the page, not evenly across the list of words, so it leans towards long words on crowded pages and silently skips anything you do not recognise, both of which pull the result back towards the familiar. The deeper cost is that you cannot measure it. With dice and a fixed list you can state your strength exactly, six words is about 77 bits, because you know how many results the method could have produced; pointing into a dictionary gives you no such number, because your real strength is set by how your finger behaves, not by how many words the book holds. A phrase that looks as though it came from a quarter of a million words may carry far less, the way a lottery ticket printed with eight digits is not one in a hundred million if only a hundred thousand were ever sold. So you are left with no provable floor under your password, which is the one thing dice give you and pointing cannot. If you use it anyway, draw more words than six, eight or more, and treat the strength as a rough margin rather than a measured one; even then it beats any password you would invent unaided. If the larger vocabulary is what tempts you, the way to actually collect it is not a finger on a page but a longer numbered wordlist with more dice per word, which is simply Diceware widened, still mechanical and still countable.

On length and numbers

Six words is genuinely enough for the uses this document serves, and it is worth understanding why rather than taking it on faith. None of these tools feeds your passphrase to the cipher directly. Borg derives the actual key with PBKDF2-HMAC-SHA256 over a random per-repository salt, LUKS runs your passphrase through PBKDF2 or Argon2id before a single block of the disk opens, and GnuPG keeps the private key on disk behind its own iterated, salted derivation. A deliberately slow, salted step like that makes every guess expensive and rules out precomputed tables. Seventy-seven bits behind a slowdown like that is past the reach of any classical attacker, including one who has stolen the drive and is grinding away offline. The twelve-words-and-numbers instinct comes from a different world: a Bitcoin seed phrase is a raw key with no such slow step in front of it, guarding a more exposed and higher-value asset, so it has to carry the full 128 bits in the entropy itself. That is a different scheme, the BIP39 2,048-word list, and its word counts do not transfer here. As for digits and symbols, skip them: a digit in a position an attacker already expects adds only about three bits, where another whole word adds nearly thirteen, so if you want more margin, add a word. The one reason to go beyond six is if you keep a copy on a network where it could be harvested and you want insurance against a future quantum attacker, who could roughly halve the effective strength; eight to ten words covers that, and an offline drive already blunts most of that concern.

One more layer, optional

You can add a word or two of your own that you never type anywhere and that exist only in your memory, laid on top of the generated phrase. Even someone who obtained the generated words would still be missing the ones only you know. Then write the whole phrase on paper, as above.

Terminal basics

If you’ve never used a terminal, here’s what you need to know.

The terminal

The terminal or CLI (Command Line Interface) is just a text window where you type commands and the computer runs them. You type one line, press enter, see what happens, then type the next. That’s the whole loop. It’s an alternate way of interacting with your computer, instead of clicking buttons in a graphical interface.

Opening the terminal

On macOS, press Cmd+Space to open Spotlight, type “Terminal”, press Return.

On Windows, you’ll need Git Bash, which provides a Linux-like terminal. Download the Git for Windows installer from git-scm.com and run it. After installation, search for “Git Bash” in the Start menu. Don’t use the regular Windows Command Prompt or PowerShell for this guide; they handle these commands differently and some will fail.

On Linux (Ubuntu, Debian, Devuan, most distributions), it’s Ctrl+Alt+T, or find “Terminal” in your applications menu. But if you’re on Linux, you likely know that already.

Getting around

Once the terminal is open, a few commands are enough to get around:
pwd prints the current directory (where you are right now in the file system).
ls lists the files in it.
cd Downloads moves into the Downloads folder.
cd .. moves up one level.
cd ~ takes you to your home directory, if you ever get lost.

Note: Commands are case-sensitive on Linux and macOS, so Ls won’t work where ls does.

Working with files

Once you can move around, a handful more commands let you create files, look at them, and tidy up:
mkdir wallets makes a new folder called wallets.
cat notes.txt prints a file’s contents to the screen, which is how you check that a file looks right after you create or edit it.
cp a.txt b.txt copies a file; to copy a whole folder and everything in it, add -r, as in cp -r olddir newdir.
mv notes.txt Downloads/ moves a file into the Downloads folder, and mv old.txt new.txt renames it; either way mv overwrites the destination without asking, so check the name before you press Enter.
echo hello prints whatever text you give it, and in guides you’ll usually see it used to write a line into a file, which the guide will spell out when it happens.
rm a.txt deletes a file. There is no recycle bin in the terminal, so rm is permanent and the file is gone the instant you press enter; never run rm on something you don’t recognize, and be especially careful when it appears next to sudo or with -rf. The -r removes all nested files and subfolders; without it, rm can only delete files, not folders. And the f means do not ask for confirmation and override any warnings.

Keys that get you unstuck

A few keystrokes make the terminal far less tedious and pull you out of trouble. The Tab key autocompletes a file or folder name once you’ve typed enough of it to be unique, and pressing it twice lists the choices when more than one matches. The Up arrow brings back your previous command so you don’t have to retype it, and pressing it again walks further back through your history; the Down arrow takes you the other way. Ctrl+C stops a command that’s running or stuck and hands you back the prompt. The clear command wipes the visible text from your terminal screen, though nothing is actually deleted; your command history is still there (you can scroll up or use arrow keys).

Running commands as administrator

Some commands change system-wide settings and need administrator rights, which you get by putting sudo in front of the command. The terminal will then ask for your password. The password stays completely invisible as you type it, with no dots or stars to show progress; that’s normal and not a frozen screen, so just keep typing and press enter.

Reading multi-line commands

When a command in this knowledge base spans multiple lines connected by \ at the end of each line, that’s one long command; paste the whole block at once. When multiple lines look separate, run them one at a time.

Why version control

This article introduces and explains important concepts of Git that every knowledge worker must know. How did you get by without it even, really?

Its companion guide, Git reference, tells you what to type or remember.

Git, What it is and why it matters

Imagine you are writing a long essay. You finish a draft on Monday, revise it on Wednesday, rewrite the introduction on Friday, and by the following Tuesday you realize the Wednesday version of the second paragraph was actually better than what you have now. But it’s gone. You typed over it. The only version that exists is the one on your screen.

This is the primary problem Git solves.

Git is the best version-control tool invented so far. It that takes snapshots of your files whenever you tell it to. Each snapshot is called a commit which, essentially, is just a version of the file or files you chose to save. You can make indefinite number of versions of a file and record why the changes were made.

The power of saving versions

There is a difference between a backup and a version history. A backup saves a file. A good VCS allows you to save an indefinite number of versions of a file and information on why a change was made, thus showing you both how the file changed over time and why. A backup says: “This is what the file looked like on 3 March 2027.” A version history says: “This is what the file looked like at 10 am on 3 March 2027, and here’s what it looked like at 5 pm, and here is what you changed from the previous versions, and here is the note you wrote to yourself at those times explaining why you made those changes.”

A backup protects you against loss. A version history gives you a record of your own thinking.

Consider what it would mean to maintain a single file, say, your personal notes on a subject you care about, across ten years of committed revisions. You would be able to open that file at any point in its history and see not just what you believed then, but what you thought was worth changing, what you added, what you removed, and what you said about each change. Every commit message is a small act of self-examination: you are forced to articulate, even in a sentence, what you did and why.

Over a decade, this accumulates into something genuinely valuable. You could trace the moment you abandoned a position. You could see when a new reading reshaped your understanding. You could watch a paragraph appear, survive three rounds of revision, and then quietly vanish from the document a year later, and read your own note explaining why you cut it. This is intellectual autobiography at the level of the sentence.

And because identical content across versions is stored only once, with delta compression underneath for the rest, the storage cost is negligible. A decade of daily commits on a text file might occupy a few megabytes. So you don’t have to choose between versions; you can keep all of them, permanently. So edit away.

Four core concepts

There are only four ideas you need to understand before using Git. Everything else is detail.

A repository, or “repo,” is a folder that Git is tracking. It’s a normal folder on your computer: your files plus a hidden .git subfolder where the history lives. You never need to open or touch that subfolder. You interact with Git either through commands in a terminal or through a visual application.

A commit is a snapshot, a saved state of all tracked files at a moment in time. Every commit has a unique identifier (a long string of letters and numbers), a timestamp, your name, and a message you wrote.

Staging is the act of telling Git which changes you want to include in your next commit. You might have changed three files but only want to snapshot two of them. Staging lets you choose. You stage changes first, then commit them.

A diff is a comparison between two versions of a file. It shows you exactly which lines were added, removed, or changed. In any visual diff tool, additions are highlighted in green and deletions in red. This is the core of what makes version history useful: not just seeing what the file looked like, but seeing precisely what changed.

Benefits of Git

Recovering a lost version is only the first application. The same history gives you several more: Every tracked file carries its full past, so you can watch the evolution of your ideas and notes over time, down to the individual line. It allows you to compare any two versions of a file, see exactly what differs, and take the parts you prefer from either side, line by line if you want. This allows for easy collaboration. Hand a copy to anyone, let them edit freely, and see precisely how their version differs from yours before deciding what to keep. This is handy when you’re working with an AI and want it to review or change something for you. You can easily tell what changed. Imagine the pain of reviewing and comparing without Git.

As a bonus, you get tamper resistence. You control what change you allow in. And because very committed change is sealed to its contents and ancestry, a change to any previous commit requires every commit after it to be changed.

Each commit carries a summary and, if you do it right, a message describing why the file changed; so when others and your future self look back, they aren’t baffled by why the change was made. Commit summary and commit messages are embeded with every commit along with the name and email you choose to provide (it doesn’t have to be your real ones).

Git concepts

This post explains important concepts of Git that every knowledge worker must know. How did you get by without it even, really?

Four core concepts

There are only four ideas you need to understand before using Git. Everything else is detail.

A repository, or “repo,” is a folder that Git is tracking. It’s a normal folder on your computer: your files plus a hidden .git subfolder where the history lives. You never need to open or touch that subfolder. You interact with Git either through commands in a terminal or through a visual application.

A commit is a snapshot, a saved state of all tracked files at a moment in time. Every commit has a unique identifier (a long string of letters and numbers), a timestamp, your name, and a message you wrote.

Staging is the act of telling Git which changes you want to include in your next commit. You might have changed three files but only want to snapshot two of them. Staging lets you choose. You stage changes first, then commit them.

A diff is a comparison between two versions of a file. It shows you exactly which lines were added, removed, or changed. In any visual diff tool, additions are highlighted in green and deletions in red. This is the core of what makes version history useful: not just seeing what the file looked like, but seeing precisely what changed.

How commits actually work

The object model is worth understanding once, early. It will make every later operation make sense rather than feel arbitrary.

When you commit, Git creates a small number of objects in its internal storage. For a repo containing three files that you just committed for the first time, Git creates five objects:

1. Three blobs. A blob is the raw content of a file, compressed. One blob per unique file content: the blob carries no filename and no permissions (those live in the tree entry that points at it), so two identical files share one blob, and a file that doesn’t change between commits keeps pointing at the same blob. Any edit, however small, produces a whole new blob; Git snapshots content, it does not store diffs at this level.
2. One tree. A tree is a list of filenames paired with the blob (or sub-tree) that holds that file’s contents. It’s essentially Git’s version of a directory listing.
3. One commit object. The commit object points to the root tree (representing the full project state at that moment), carries the metadata (author, date, message), and points to the parent commit or commits. A normal commit has one parent. The initial commit has no parent. A merge commit has two or more parents, one for each branch being merged.

The commit object is the anchor. Given a commit, you can reconstruct the entire state of the repo at that moment by walking from the commit to its tree, from the tree to its blobs, and from sub-trees to their nested blobs. Sub-trees are how directories work: a tree entry pairs a name with a hash, and that hash points either at a blob (a file) or at another tree (a subdirectory). A repo containing notes/ch1.md has a root tree with an entry notes pointing at a second tree, and that second tree has an entry ch1.md pointing at the blob holding the chapter’s text. Reconstruction is a recursive walk: start at the commit’s root tree and descend until every entry bottoms out in a blob. One pleasant consequence: a subdirectory you didn’t touch keeps the same sub-tree hash from one commit to the next, so deduplication happens at directory granularity too, with a single reused pointer covering any number of unchanged files. The commit also points backward to its parent, which points to its parent, and so on, all the way back to the initial commit, which doesn’t point to a parent commit because it doesn’t have one. That chain of parent pointers is the history: git log is just a walk along it.

Two consequences fall out of this model.

First, Git’s deduplication is structural rather than engineered. Because an object’s address is the hash of its own content, identical content cannot be stored twice: if you don’t change a file between two commits, both commits’ trees point to the same blob. Systems whose storage isn’t content-addressed have to build duplicate-avoidance as machinery (Subversion stores file changes as deltas between revisions; Mercurial’s revlogs are per-file delta chains; backup tools like borg and restic implement explicit chunk-level deduplication); in Git it falls out of the addressing scheme. The one engineered layer Git does add is delta compression inside packfiles, which squeezes similar-but-not-identical blobs during garbage collection; that runs underneath the model and changes nothing about it. This is why ten years of commits on a slowly-evolving document occupy negligible space.

Second, every commit is immutable. A commit object is identified by a cryptographic hash of its own contents. Change anything about it (the message, the author, the parent, even a timestamp) and the hash changes, which means it’s now a different commit. This is why “rewriting history”, the family of operations that replace commits with edited copies (amending the last commit, rebasing a series; both covered in § Rewriting history below), is technically a misnomer: you don’t edit old commits, you create new commits that replace them. Old commits still exist in the database until garbage collection eventually removes them, which is why the reflog can still find them weeks later. One corollary, stated directly: two commits can carry an identical title and still be distinct commits, because the message is only one of the hashed inputs, and their trees, parents, or timestamps differ. A busy repo accordingly accumulates many commits titled Merge branch 'main'.

Branches as pointers

A branch in Git is a lightweight movable pointer to a commit. That’s it. The default branch name in Git is main (older repos use master).

When you make a commit, Git creates a new commit object whose parent is the commit you were previously on. Then it moves the current branch pointer forward to the new commit. Every time you commit, the branch pointer advances automatically.

One more pointer completes the picture: HEAD. HEAD is Git’s you-are-here marker; it normally points at a branch, which points at a commit, and committing moves the branch while HEAD rides along. HEAD~1, HEAD~2, and so on count backward from wherever HEAD is, which is why the reference doc uses them as the standard way to name recent commits. HEAD can also point directly at a commit instead of at a branch, the state Git calls detached HEAD, which you enter when you check out an old commit to look around. Detached HEAD is a read-mostly state, not an error: looking is free, but a commit made there belongs to no branch, so it becomes unreachable the moment you switch away (recoverable from the reflog for ninety days, like everything else). If you decide to build on an old commit, create a branch right there first; the warning Git prints on detaching says exactly this.

Creating a new branch is just writing a new pointer. It costs nothing. Deleting a branch is just removing a pointer, it doesn’t delete the commits the branch pointed at (those remain reachable through other branches, or through the reflog).

This is why Git culture encourages using branches liberally. They are not expensive objects. They are file names containing a commit hash. Create one every time you want to try something. If it works, merge it; if it doesn’t, delete it and the experiment vanishes cleanly from your workflow while the commits themselves remain in the reflog for ninety days as a safety net.

The mental model: your main branch (usually called main) should always be in a known-good state. Every piece of in-progress work happens on its own branch, isolated from everything else, and only gets integrated into main when it’s ready. This feels like overhead on day one and feels like oxygen by month six.

Staging, explained

One distinction comes before staging: tracked versus untracked. A file Git has never been told about is untracked; it shows up in git status, but it belongs to no snapshot, and nothing in Git protects or records it until the first git add. That first add is what turns a file from invisible to tracked, and only tracked files participate in anything this document describes. The classic first-week surprise follows directly: git commit -a stages and commits modified tracked files only, so a brand-new file silently stays out of the commit until it has been added once. When a commit seems to be missing a file, check git status for it under “Untracked files” before suspecting anything deeper.

The staging area was designed for a workflow that most solo users, and especially beginners keeping personal version history, don’t actually have. Git was originally built for Linux kernel development, where a single developer might be working on several unrelated changes simultaneously and needs to package them into separate, clean commits before sharing them with other people.

Imagine you sit down to fix a bug, and while you’re in there you also notice a typo in a comment, and you decide to rename a variable for clarity. You’ve now made three logically unrelated changes to your working files. If you commit them all together, the history becomes muddy; someone reading it later sees “fixed bug” but the commit also contains the rename and the typo fix, which makes it harder to understand what actually fixed the bug, and much harder to undo just one of those changes later.

The staging area lets you say: “Of all the changes sitting in my files right now, include only these specific lines in the next commit.” You stage the bug fix and commit it with a clean message. Then you stage the typo fix and commit that separately. Then the rename. Three clean commits, each doing one thing, each reversible on its own. That’s the point of staging: it’s a workbench where you assemble a commit before finalizing it, rather than being forced to commit everything that’s currently different from the last commit.

This matters enormously on collaborative projects where other people will read your history, review your commits, and potentially revert individual ones. It matters less when you’re keeping a personal save-point history of your own work.

Solo workflows are usually “I’m working, I want to save this version, let me commit, now I can keep going, then commit again.” You can absolutely use Git that way. The command git commit -a skips the staging step entirely and commits every tracked change in one go. In GUIs, most of them have a “stage all and commit” button or a checkbox that selects all changes at once.

There are four levels of staging granularity, from least to most precise.

File-level. You’ve modified three files and want to commit only two. Stage the two with git add file1 file2, commit; the third file stays in your working directory, unstaged, waiting for a future commit.

Hunk-level. A single file contains two unrelated changes; you want to commit one, not the other. Use git add -p file, which walks you through each “hunk” (a contiguous region of changed lines) and asks y/n/q/a/d/s/e/?:

1. y stages this hunk.
2. n skips it.
3. q quits.
4. a stages this and all remaining hunks in the file.
5. d skips this and all remaining hunks.
6. s splits the current hunk into smaller hunks when Git has grouped unrelated changes together.
7. e opens the hunk in your editor for line-by-line selection.
8. ? shows help.

Line-level. Inside the e option of git add -p, you can hand-pick individual lines. In the editor, deleting an added line (one starting with +) excludes it from staging. Changing a removed line’s - to a space keeps the deletion in your working directory without applying it to this commit. Fiddly the first few times, surgical once you’re used to it. VSCodium’s “Stage Selected Ranges” (§ Seeing file changes in VSCodium, below) is the same mechanism with a better interface.

Stage-everything. git commit -a -m "message" grabs every tracked modified file, stages it all, and commits in one shot. The workflow for people who don’t want to think about staging.

Staging is not a one-way door. git restore --staged file unstages without changing the file’s contents. The staging area is a scratch pad, not a commitment.

When staging actually matters for solo work

Honest answer: for a personal project, line-by-line staging is useful rarely. Most of the time, on solo work, you’ll commit everything and move on. The elaborate machinery exists, but the situations that actually demand it are uncommon in solo workflows.

Here’s where it earns its keep, even alone. The scenario is almost always the same shape: you sat down intending to do one thing, got distracted or curious along the way, and now your working directory contains two or three logically separate changes mixed together. You fixed a bug, but while you were in that file you also cleaned up some formatting, and you started sketching a new feature that isn’t working yet. If you commit all of that as one blob with a message like “bug fix and other stuff,” you’ve lost the ability to do a few things later. You can’t revert just the bug fix without also reverting the formatting and the half-done feature. You can’t look back at the history and understand cleanly when or why any one of those changes happened. And if the half-done feature turns out to be a dead end, you can’t throw it away without losing the bug fix that’s tangled up with it.

Staging lets you untangle that mess at commit time. You stage and commit the bug fix with a clear message. You stage and commit the formatting cleanup with its own message. The half-done feature stays in your working directory, uncommitted but not lost, and you keep working on it or discard it later.

The second real use case is more specific: the “oh no” moment. You’ve been working for two or three hours. Things are partly working and partly broken. You realize you want to save the parts that work before you break them further, but you don’t want to commit the broken parts because committing broken code even to your own history is annoying. Staging lets you pick out the working parts, commit them as a solid checkpoint, and keep fiddling with the broken parts without fear.

A realistic usage pattern on a personal project: ninety percent of the time, commit everything together because your changes are coherent. Maybe eight percent of the time, use file-level staging because two files are changed for unrelated reasons. Maybe two percent of the time, reach for git add -p because you actually need to split changes within a single file. The line-by-line editing option gets used a handful of times per year. The tool exists for when you need it.

Don’t try to learn staging up front as an abstract concept. Commit everything together with git commit -a -m "message" or the equivalent GUI button, and get comfortable with the basic rhythm. Eventually you’ll hit one of the scenarios above, and at that point staging will stop feeling like arbitrary complexity and start feeling like the right tool. Learn it then. The understanding sticks much better when it’s attached to an actual problem you’re trying to solve.

Seeing file changes in VSCodium

If a visual diff is easier for you to read than terminal output, VSCodium gives you a Git workflow that makes the diff, the staging step, and the commit itself legible at a glance. The Source Control panel built into VSCodium covers the daily cycle on its own; the GitLens extension adds blame, history, and visualization on top.

Built into VSCodium

Open the folder that contains your repo. VSCodium auto-detects the .git folder inside and enables the Source Control panel without any extension installed.

The Source Control icon sits in the left sidebar: a branching-line icon. Click it. The panel has two main sections: “Changes” (files you’ve modified but haven’t staged) and “Staged Changes” (files you’ve added with git add). Below them, a text box for the commit message, and a checkmark button to commit.

Click a changed file to see the diff. The diff opens in the main editor, showing two columns: the last committed version on the left, your current version on the right. Green highlights are additions; red highlights are deletions. Lines with no highlight are unchanged context.

To stage a whole file, hover over its name in the Changes list and click the + that appears. The file moves down to Staged Changes.

To stage a specific hunk within a file (when you only want to commit part of the file’s changes), hover over the hunk in the diff view. A small + appears beside it. Click it. That hunk stages; the rest remains unstaged.

To stage individual lines, select them in the diff, right-click, and choose “Stage Selected Ranges.” This is the visual equivalent of git add -p with the e (edit) option covered in § Staging, explained above. Surgical control, done by mouse selection instead of editor surgery.

Once you have what you want staged, type a message into the text box at the top of the Source Control panel, then either click the checkmark or press Ctrl+Enter. That’s a commit.

What GitLens adds

Install GitLens from the Extensions panel (search for “GitLens, Git supercharged”; the publisher is eamodio on Open VSX, maintained by GitKraken).

GitLens adds several further capabilities worth knowing:

1. Inline blame annotations: the editor shows, for the line your cursor is on, who last changed it, when, and with what commit message. If it becomes distracting you can toggle it off: GitLens: Toggle Line Blame.
2. Gutter indicators: thin colored bars in the left margin of the editor mark which lines have been added, modified, or deleted since the last commit. A quick visual of what you’ve changed in the open file.
3. File history: right-click any file and choose “Open File History” to see every commit that touched it, with previews of each version.
4. Commit graph: a full visualization of your branch structure. Useful when history gets tangled. Free for local and public repos; private-repo access requires a GitLens Pro subscription, as do Worktrees, Visual File History, and the AI features.
5. Compare: you can compare any two branches, or any two commits, and see a combined diff.

This much is enough to replace most command-line usage for a solo writer or developer. You still drop to the terminal for complex history surgery (filter-repo, reflog recovery, ad-hoc shell tooling), but for the daily edit-stage-commit cycle the GUI is faster and more legible than the terminal.

Seeing file changes in vim

If you edit in vim, the diff view should also be in vim. A terminal pager like git-delta is a step backward when your editor is already a better diff viewer than any pager: pagers are read-only, they don’t honor your keybindings or colorscheme, and you can’t act on what you see.

The dominant tool in vim culture is Tim Pope’s vim-fugitive plugin. With your repo open in vim, run :Gdiffsplit for a horizontal split or :Gvdiffsplit for a vertical one. You get the indexed version of the current file alongside the working-tree version, both as real vim buffers, with synced scrolling, automatic folding of unchanged regions, and full editability on either side. Vim’s diff-mode keybindings worth memorizing:

1. ]c jumps to the next changed hunk; [c jumps to the previous.
2. do (diff obtain) pulls the change at the cursor from the other buffer into the current one.
3. dp (diff put) pushes your version into the other buffer.
4. :Gwrite (fugitive) stages the current buffer (equivalent to git add on that file).

The combined effect: read the diff, decide hunk by hunk what stays and what reverts, stage the result, and commit, all without leaving vim.

If you don’t want plugins, use vimdiff directly. Configure it as Git’s diff tool once:

git config --global diff.tool vimdiff
git config --global difftool.prompt false

Then git difftool <file> opens that file’s diff in vim, and git difftool with no argument walks every changed file in turn. Same ]c, do, dp keybindings; you just don’t get fugitive’s tighter integration with git add, git blame, and the rest. For comparing arbitrary files outside Git, plain vimdiff file1 file2 works with no setup.

Which tool when

The principle: use the diff tool tied to whichever editor you actually live in. The argument is the same regardless of editor: if you can read the diff and edit either side without leaving the tool you already use all day, the friction of context-switching disappears and the keybindings carry over. This doc covers vim and VSCodium concretely; other editors (Sublime, JetBrains, Helix, emacs with magit, Zed) have analogous integrations and the principle generalizes.

If you edit in vim, fugitive’s :Gdiffsplit is the strongest combination of viewer and editor available anywhere. Both sides are buffers in your editor, you can edit either, you can stage from inside, and the keybindings are the same ones you use all day for everything else. Nothing else in this list matches it for someone fluent in vim.

If you edit in VSCodium, the built-in Source Control panel plus GitLens is the strongest editor-integrated option available there. The diff view shows committed and working versions side by side, you can edit the working side directly, and hunk-level staging is one click.

GitHub’s web diff is a different category. It’s optimized for reviewing other people’s pull requests, with line-level comments and “suggest changes” boxes. You can’t see your own uncommitted local work there at all (you have to push first), and the editing affordances are minimal. Use it for code review of other people’s work, not for your own daily edit-stage-commit cycle.

git-delta and similar terminal pagers (diff-so-fancy, diffr) sit below all of these. They prettify the read-only diff output you see when you run git diff or git show in a terminal. Useful when you’re already in the terminal and want a more legible pager than less. Not a workflow tool; it’s a nicer view, nothing more.

Plain git diff piped through default less is the floor: zero setup, ugly but unambiguous, and what you’ll see when you ssh into a server you haven’t configured. Worth being comfortable with for that reason.

A two-line decision rule. If you’re editing the file, use the tool tied to your editor. If you’re just reading, use whichever pager is in front of you.

Commits as communication

Here is the principle that underlies nearly every piece of specific advice that follows: Git is not a backup tool, it’s a communication tool. It happens to also back up your work, but that’s a side effect. What it’s really doing is creating a record that communicates your thinking to other people, and the most important “other person” you will ever communicate with through Git is your own future self. Every decision about how to use Git flows from taking that seriously.

Linus Torvalds, the Bitcoin Core maintainers, the Linux kernel community: what makes their use of Git exemplary isn’t technical sophistication, it’s that they treat every commit as a message to future readers. Once that principle is internalized, most of the specific practices become obvious.

Commit message discipline

From that principle, the first concrete habit: write commit messages that explain why, not what. The diff already shows what changed. Anyone can read that. What the diff can’t tell them is why you made the change.

“Fixed bug” is useless. “Fixed off-by-one error in pagination that caused the last item to be skipped when total count was exactly divisible by page size” is a gift to the future.

The canonical format used by the Linux kernel and adopted by most serious projects:

1. Summary line of fifty characters or less.
2. Blank line.
3. Longer explanation in paragraphs, explaining the context, the reasoning, and any caveats.

Three further conventions go with that format:

1. Imperative mood for the summary. “Add login redirect,” not “Added” or “Adds.” It reads as completing the sentence “If applied, this commit will…”. This is what git itself uses for its own commits (“Merge branch”, “Revert”, “Update”).
2. No trailing period on the summary. It’s a one-line label, not a sentence. The period adds visual noise without information.
3. Body wrapped at seventy-two characters. Pairs with the fifty-character summary rule. git log indents the body by four spaces (4+72=76, fits an 80-column terminal); email patch workflows quote-prefix replies, and longer lines break across wraps; git format-patch and similar tools assume it. Without the wrap, git log and any email-based review render ragged.

A full example that obeys all of the above, for code:

Fix dropped last page on exact-multiple counts

The page count used integer division of total by page_size, which
dropped the final page whenever total was an exact multiple of
page_size. A request for the last page returned empty and the item
was never rendered.

Round up so a full final page is always emitted, and add a
regression test covering the exact-multiple boundary.

Closes #214.

The summary is imperative, under fifty characters, and has no trailing period. A blank line separates it from the body. The body explains why the bug existed and what the fix does, wrapped near seventy-two, and a trailer points at the issue.

The same shape for prose drops the issue trailer and records intent rather than mechanism:

Cut the second paragraph of the intro

It restated the thesis the opening already carried and slowed the
entry into section 1. The argument reads tighter without it.

You don’t need to follow this rigidly on personal projects, but you should absorb its spirit. Every commit message should answer the question: “if someone finds this commit in a git blame three years from now while debugging, what do they need to know?”

Several short pieces are worth reading once and internalizing.

Chris Beams, “How to Write a Git Commit Message,” https://cbea.ms/git-commit/. The seven rules with worked before-and-after examples; the canonical modern how-to.

David Thompson, “My favourite Git commit,” https://dhwthompson.com/2019/my-favourite-git-commit. A one-character whitespace fix whose message documented the error, the investigation, and the fix, later found by others running git log --grep who learned who had hit it before and how they solved it.

Michael Lynch, “No Longer My Favorite Git Commit,” https://mtlynch.io/no-longer-my-favorite-git-commit/. The rebuttal to the piece above: the same commit buries its key point too deep, which is the case for putting the load-bearing line first.

Tim Pope, “A Note About Git Commit Messages” (2008), https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html. The origin of the fifty/seventy-two/imperative conventions above; nearly universal in serious projects.

A common temptation is to skip the body because the reasoning already lives somewhere else: a changelog, a versioning file, an issue, a design note. The message and an external file are not interchangeable, for three reasons.

Locality: git blame and git log -L land you on a specific line tied to a specific hash, with the message right there (both tools are walked through in § Reading history: log and blame, below); an external file forces a second lookup, and assumes it still matches the change and was not reorganized since.

Binding: the message is hashed into the commit and cannot drift, whereas an external file is mutable and over years drifts away from the diffs it once described.

Audience: a changelog is reader- or release-facing, whereas a commit message is maintainer-facing at the grain of one change.

The resolution is a pointer, not duplication. If the deep rationale genuinely belongs in another file, let the body reference it (an issue number, a doc path, a changelog anchor) rather than restating it. What stays non-negotiable is the summary line, because that is what blame and log surface regardless. A body that is only a durable pointer is fine when the summary is load-bearing; a dead summary like “update” or “fix” is not, because the external file will not be in front of you when the tool puts you on the line.

A related question is which whys belong in the document’s own text rather than a commit at all. The split that holds is current state versus transition. Rationale about the current state, the thing every future reader of the document needs in front of them, belongs in the prose: it should be visible without running a single Git command. Rationale about a transition, why you changed something from what it was, belongs in the commit: it is tied to that specific change, and surfacing it in the document would clutter the living text with the history of decisions already made. A why that you judge not worth surfacing in the document is usually a transition why, and the commit is its proper home.

A further question: does a change so obvious it explains itself need a message at all? The subject line, always: Git refuses an empty message, and a dead subject like “update” defeats every tool that surfaces it. The body, only when it carries a why the diff cannot show; “Fix typo in § Hooks” is complete with no body, because the diff is the explanation. The trap is judging “obvious” by diff size. The most celebrated message in the reading list above, David Thompson’s favourite, decorates a one-character whitespace diff, and the body was the entire value. Test by recoverable why, not by size: if a stranger reading the diff in three years would still ask why, write the clause. When in doubt, one sentence of why costs less than the doubt.

A related scoping question: when the repo holds many files and a commit touches one, should the summary name the file? As a rule, no: the summary names the change, not the file list. Every tool that surfaces the summary sits next to a tool that surfaces the files (git show, git log --stat, git log -- <path>), so a filename in the summary spends scarce characters on what the diff already carries, and the fifty-character budget is better spent on intent. The earned exception is the area prefix, the subsystem: summary convention from large repos: the Linux kernel writes net: sched: fix refcount leak because in a tree that size a summary is unreadable without locating context, and the prose analog is ch2: tighten opening in a book-sized repo. Use the prefix when the change is meaningless without its location; never use the summary as a file inventory. For session-grain commits the dated enumeration already plays this role, and the path-prefixed body line shown in the session-grain subsection below scopes a why to one file inside the bundle.

For the day-to-day mechanics — the aliases (git c, git ca, git last), the commit.verbose setting, the commit message template, and the editor configuration that makes all this comfortable — see the Git reference doc.

One commit, one logical change

Each commit does one thing. This is where the discipline of staging becomes valuable, not because you have to use it on every commit, but because the principle of “one commit, one logical change” is what makes history valuable decades later.

A commit that mixes a bug fix, a refactor, and a new feature is nearly impossible to reason about in isolation. You can’t revert part of it. You can’t understand it at a glance. You can’t cherry-pick it to another branch.

Small, focused commits are the atoms of a useful history.

Rule of thumb: if you find yourself writing “and” in a commit message, you probably should have made two commits.

Concretely: one sitting produces a bug fix and a rename, and the single message would have been “Fix dropped last page and rename PageHelper”. Split, it becomes two commits, “Fix dropped last page on exact-multiple counts” (the commit shown in full in the previous section) and “Rename PageHelper to Paginator”, each revertable, readable, and cherry-pickable on its own.

Commit often, curate before sharing

Two practices for handling the gap between “save often” and “history should be legible.” Which fits depends on the repo. For shared code, multi-author projects, anything that gets reviewed before merging: commit messily while working, then curate before pushing (the workflow below). For solo writing, single-author docs, exploratory personal projects with no shared history to clean up for: commit at session grain with dated messages (the subsection at the end of this section). Mixed-author writing projects sit in between — session-grain locally is fine, but commits that ship as releases or get pointed at externally are worth curating. Pick the model that fits the repo’s audience; don’t try to apply both.

This resolves what otherwise feels like a contradiction. “Small focused commits” sounds incompatible with “just save your work as you go.” The answer is that your local in-progress history doesn’t have to be clean. Commit messily while you’re working, whenever you hit a checkpoint, with throwaway messages like “wip” or “trying this.” Then, before you push or share, use interactive rebase (git rebase -i) to reshape that messy sequence into clean, logical commits with real messages.

Squash the typo fixes into the commits they belong to. Split large commits that do too many things. Write proper messages. This is the workflow experienced maintainers use, and it’s what lets their public history look so crisp even though their actual working process is as chaotic as anyone’s. The private draft is scruffy; the published version is polished.

Session-grain commits

Not every project needs the curate-before-share workflow. Solo writing, exploratory personal notes, long-running single-author docs — there’s no shared history to clean up for, so the curation step adds friction without much payoff. For these workflows the defensible practice is the dated work-session commit, used as the primary commit grain rather than as scratch to be rewritten later.

The format is a date plus a brief enumeration of areas touched:

2026-05-28 — ch3 revisions, footnotes, intro tightening

The date alone is redundant with what git log shows; the value is in the enumeration after the em dash, which makes git log --grep=footnote or git log --grep=ch3 find the sessions where you touched a given area. The commit isn’t atomic — it bundles whatever a writing session produced — and the message names the bundle honestly without pretending otherwise.

Two things this is not. It is not a substitute for one-logical-change commits when the work is shareable; for code or collaborative projects, curate. It is not an excuse to skip git diff --staged before committing — even a session-grain commit benefits from a glance at what’s actually in it, especially to catch accidentally-staged secrets or junk files.

Scoping a why to one file inside a bundled session commit. A session commit gathers several files, but a commit still carries exactly one message; there is no per-file message field. There are two ways to attach a why to a single file without unbundling the rest. Put the why in the body on a line prefixed with the file’s path, so git log -- path and git blame surface it against that file later. Or split that one file into its own commit, so its message is its why, and commit the remaining files under the session line separately. Which to use is a discoverability call: history views and inline blame show only the subject line, so a why in the body is found by git log --grep or blame, not by scanning git log --oneline. A light note rides fine in the body; a why heavy enough that a future reader must not miss it is the signal to give that file its own commit, where the message is bound to the change and shows up on its own.

Concretely, the two shapes. The why riding in the session body, prefixed with the file’s path:

2026-06-08 — ch2 pass, refs cleanup, typo sweep

notes/ch2.md: cut the Hodge block quote; it was secondhand via
Berkhof, replaced with a direct citation.

And the same why promoted to its own commit, with the session commit shrinking around it:

Cut the Hodge block quote from ch2

It was secondhand via Berkhof; replaced with a direct citation so
the chapter quotes Hodge at first hand.

2026-06-08 — refs cleanup, typo sweep

The commands for both, multiple -m paragraphs and a path-scoped partial commit, are in the Git reference doc.

The hierarchy that matters: curated atomic commits are best when there’s shared history, session-grain dated commits are a legitimate practice for solo work where curation buys nothing, and unlabeled wip or update commits across years are the failure mode this section exists to prevent. Pick the grain that fits the work; commit at that grain deliberately.

Rewriting history

Git lets you rewrite history, with caveats. The mechanisms:

For the most recent commit:

git commit --amend -m "new message"

This replaces the message on the last commit you made. Without -m, git commit --amend opens your editor to write a longer message. You can also amend to add forgotten changes: stage the changes, then git commit --amend --no-edit folds them into the previous commit without touching the message.

For older commits:

git rebase -i HEAD~N

-i is interactive. HEAD~N means “go back N commits.” Git opens an editor showing those commits as a list, each prefixed with pick. Change pick to:

1. reword to change the message.
2. squash to fold the commit into the one above it (combining both messages).
3. fixup to fold in without keeping the message.
4. edit to stop at this commit and amend it.
5. drop to delete the commit entirely.

Reorder commits by moving lines. Save and close, and Git walks through your instructions.

The interactive rebase above is the general tool. For the specific case of “commit C is missing a small change,” there’s a cleaner shortcut: the fixup + autosquash workflow. Stage the change, then git commit --fixup=<C> records a new commit with the message fixup! <subject of C>. When you later run git rebase -i --autosquash <C>^, Git pre-positions the fixup next to C and pre-marks it for squashing — you confirm by saving the editor. This is the dominant workflow on rebase-based projects (Bitcoin Core, the Linux kernel, CPython) because it skips the manual reorder-and-relabel step that plain interactive rebase requires.

Note that --amend itself only operates on HEAD. There’s no syntax to target an older commit; for anything beyond HEAD you need rebase, and the fixup pair is the path of least friction.

For merges specifically, interactive rebase across them requires --rebase-merges:

git rebase -i --rebase-merges HEAD~5

Without that flag, rebase flattens out merges, which you probably don’t want.

The safety rule

Rewriting is safe when the commits only exist locally. The moment you’ve pushed to a shared remote, rewriting causes problems. Other people’s copies of history diverge from yours, and reconciling it requires force-pushing, which can destroy their work if they’ve built on top of the commits you rewrote.

For a purely solo project, or where you’re the only one pulling, rewrite freely. For anything shared, the rule of thumb: rewrite only commits that haven’t been pushed, and leave pushed history alone.

When you must force-push (amended a commit and need the remote to accept the new version), use --force-with-lease:

git push --force-with-lease

This refuses the push if the remote has changed in a way you didn’t expect, preventing you from overwriting someone else’s work you didn’t know had been pushed. Use it instead of plain --force by default.

There’s a lighter-weight alternative: git notes. Notes let you attach additional commentary to a commit without rewriting it. The commit itself is untouched; you add a note alongside it with git notes add -m "additional context" <sha>. Safe on pushed commits because it doesn’t alter the commit. Downsides: notes don’t show up in most views by default, and they don’t travel between repositories automatically without extra configuration. A niche tool, useful occasionally.

The reflog as safety net

git reflog is Git’s safety net. It’s a log of where HEAD has been, so if you ever feel like you’ve lost work through a bad rebase or a hard reset or some other destructive operation, the reflog almost certainly still has the commit you’re worried about losing.

New users panic about “losing” weeks of work, and it’s almost always recoverable through the reflog. Knowing this exists lets you experiment fearlessly, because Git very rarely actually loses anything: it just moves references around. The reflog keeps a record of every move for ninety days by default.

The typical recovery pattern:

git reflog                                # find the sha you want
git reset --hard <sha>                    # go back to it
# or:
git branch recovery <sha>                 # save it as a new branch

If you ever find yourself in a state you don’t understand after a destructive operation, stop before running more commands. Read the reflog. The panic-repair cycle is where people actually lose work, not the original mistake.

Reading history: log and blame

Good commit hygiene pays off only if you actually read history. git log and git blame are the tools that make it worth your while.

git log --oneline --graph --all --decorate is the one flag combination worth memorizing. Compact visual history of every branch. Aliasing it as git lg (see the configuration section below) saves typing. A sample of what it shows, output illustrative:

$ git lg
* 3f2a91c (HEAD -> main) Tighten §2 opening
* 8c41d07 Add covenant-of-works subsection
| * a19be4f (experiment/first-person) Recast ch1 in first person
|/
* 5d20e83 Cut second intro paragraph
* 2b7f410 Initial commit

One line per commit, branch and HEAD labels inline, and the graph column showing where the experiment branch forked from main.

git log -S "string" (pickaxe) searches history for commits that added or removed a specific string. This is how you find “when did this text first appear in the document?” or “when did I delete that paragraph?”

git log --author="name" and git log --since="2 weeks ago" filter the log by who and when.

git blame <file> annotates every line with the commit that last touched it. Combined with a good commit message, the blame tells you not only when a line was written but why. This is where the investment in commit messages compounds. Again illustrative:

$ git blame -L 12,14 notes/ch2.md
8c41d07 (jdoe 2026-05-30 14:02:11 +0530 12) The covenant of works frames
8c41d07 (jdoe 2026-05-30 14:02:11 +0530 13) the argument: obedience as the
3f2a91c (jdoe 2026-06-02 09:41:38 +0530 14) condition, life as the promise.

Three lines, two commits: the first two last moved in the subsection commit, the third in a later tightening pass, and git show 3f2a91c surfaces the why behind it.

For the full option surface, the manual pages are authoritative: git-scm.com/docs/git-log and git-scm.com/docs/git-blame. Pro Git’s chapter on viewing history (git-scm.com/book/en/v2/Git-Basics-Viewing-the-Commit-History) is the best guided tour of git log’s filtering flags.

The habit of looking at history when debugging, “when did this break? what changed around then? why?” is where Git transforms from a save-point tool into something closer to a time machine.

The commit log as changelog

Before version control, projects maintained changelogs by hand. A file called CHANGELOG, CHANGES, or NEWS sat at the root of the source tree, and the maintainer added an entry to the top of it at each release: version number, date, a few bullets of what changed. GNU projects have used NEWS files for this since the 1980s; many still do. The “Keep A Changelog” convention (2014) codified a Markdown format for the same practice.

The problem with hand-written changelogs is drift. The changelog lives parallel to the actual history. Maintainers forget to update it. Entries get written at release time from memory, which is unreliable and selective. The changelog says one thing, the commits say another, and the commits are right.

Git subsumes the raw version of this work. The commit log is the changelog, generated automatically from the commits you’ve been making all along. For any two points in history, git log v1.0.0..v1.1.0 --oneline gives you every commit between them in order, with no effort. git shortlog -sn v1.0.0..v1.1.0 groups them by author. git log --since="1 month ago" gives a month’s worth. These generate the changelog; they do not require you to maintain a changelog file.

What Git doesn’t replace is the curated release note. A commit log is for an engineer investigating history. A release note is for a user deciding whether to upgrade. Converting the former into the latter is still a writing task: grouping by category (features, bug fixes, breaking changes), discarding commits users don’t care about (typo fixes, internal refactors), and translating developer language into user-facing language. The difference is you’re editing a generated draft rather than reconstructing from memory. You can’t miss a commit because the tool is listing all of them.

Tools that bridge the two levels. git log v1.0.0..v1.1.0 --oneline as the starting draft. On GitHub, gh release create v1.1.0 --generate-notes auto-drafts release notes from commits and PRs since the last tag, which you then edit. Conventional Commits (a convention that prefixes commit messages with feat:, fix:, breaking:) lets tools like git-cliff or release-please auto-group entries by category; worthwhile for projects that ship often, overkill for solo writing.

For a personal writing repo, the commit log is the changelog, full stop. You don’t need a CHANGELOG.md file; readers who care about change history read git log or click “History” on the host’s web view. If you want a human-readable summary when you ship a revision, generate it from git log at that moment, don’t maintain it in parallel.

Per-file history is the same idea at finer grain. git log --follow <path> shows every commit that touched a specific file, following renames across its lifetime. On GitHub, https://github.com/<user>/<repo>/commits/<branch>/<path> is the web rendering of that log. A blog or reference site can expose this URL as a “History” link in the footer of every page; Simon Willison’s til.simonwillison.net is the reference implementation.

The decision space behind that link is wider than it looks. Two questions organize it. First, who decides what counts as a change: git, which makes no distinction between a typo fix and a paragraph rewrite, or a human curator, which is editorial but requires discipline you have to maintain forever. Second, who renders the diff: the host (GitHub, Codeberg, GitLab all render commit diffs in their web UI), or you (your build script generates diff pages on your own site).

Three families fall out. Host-linked: computer decides, host renders. The pattern above. For markdown files on GitHub specifically, the commit page has a “Display the rich diff” toggle that renders the markdown and highlights word-level changes inline, close to the prose-diff quality of purpose-built tools like NewsDiffs. Self-rendered: computer decides, you render. Same git walking, but your build generates per-page diff pages on your own site. Worth the work when the repo can’t be public or you want full control of the reader’s experience; otherwise overkill. Hand-written: human decides. A CHANGELOG.md in Keep a Changelog format, or a per-post “Revisions” block at the foot of each post. Same drift problem as the release-note case.

For a personal writing repo, host-linked is almost always the right pick. Five lines of template, the host does the rendering, and the reader gets a real history view; on GitHub specifically, the rich-diff toggle on each commit gives a rendered prose view (worth a one-line awareness hint in the template).

A note on what “version” means for a blog post. Git has no version concept for an individual file; every commit is a potential version. Three habits keep the history view manageable: squash related micro-commits before pushing so each row is one substantive change; write commit messages that read as reader-facing changelog entries, because they are; and prefix non-content commits (typo fixes, formatting tweaks, metadata edits) with a token like [trivial] so a build script can filter them when computing the displayed “Last updated” date. Without that filter, a typo fix advances the date on a two-year-old post whose actual content hasn’t moved.

In a log, the filter’s input looks like this (output illustrative):

$ git log --oneline -- content/covenant.md
9b3c1e2 [trivial] Fix two typos in covenant post
4f08a77 Add section on Kline's critique
e1d92c0 Publish covenant post

The build script skips the [trivial] row when computing the date, so “Last updated” stays on the Kline addition.

The lightweight alternative is to skip per-page history machinery and put corrections inline. A bracketed [Edit YYYY-MM-DD: ...] aside placed where a now-wrong claim sits catches the reader at the sentence the correction applies to, no clicking required. For posts you rarely revise, inline notes are sufficient on their own; the host-linked approach earns its place only when revisions are frequent enough that readers want to inspect them systematically.

See the GitHub reference for the Zola template snippet, including the build-time git log script with [trivial] filtering.

Versioning prose documents

The previous section concerns commit-level history: every commit is a potential version, and the log is the changelog. A separate question is whether documents (essays, reference notes, theological writing) should carry an explicit version number on top of that.

Code projects answer this with semantic versioning, usually called semver: a three-number scheme MAJOR.MINOR.PATCH where MAJOR signals a breaking change, MINOR signals a backward-compatible feature, and PATCH signals a bug fix. For documents the breaking-change concept doesn’t translate literally (prose doesn’t compile), but it translates cleanly with one substitution.

The mapping for prose

Translate “breaking change” to “thesis shift.”

1. MAJOR: the claim changed. Your view of the subject has moved. Someone who cited the previous version to support an argument may find their citation no longer supports it. This is the prose equivalent of a breaking API change, because readers who built on your earlier conclusion now have work to do.
2. MINOR: the content changed but the claim didn’t. You added a section, cut a digression, expanded an example, brought in a new source. The thesis is intact. Readers who remember what you argued don’t need to re-read; they only need to look at the new version if they want the fuller treatment.
3. PATCH: the copy changed. Typo fixes, clarified wording, reformatted a list, corrected a date. No change in meaning, no change in content. Readers don’t need to do anything.

The mnemonic is three Cs: claim, content, copy. Claim is the argumentative term; content is the body of substance; copy is the publishing-industry term for the text itself.

Pre-1.0 translates cleanly. 0.x.y is a draft you’re not yet willing to commit to publicly. 1.0.0 is the point at which you’re standing behind the thesis. After that, every revision records at what level you’ve shifted.

Worked example

A theology essay on covenant theology, first published as 1.0.0.

1. You catch a typo in a footnote, fix a quotation that had the wrong translation, and tighten an awkward sentence. Bump to 1.0.1. Copy only.
2. A month later you add a section on a minor critic you hadn’t addressed, cut a digression on an adjacent topic, and expand one example. The thesis hasn’t moved. Bump to 1.1.0. Content.
3. A year later you read a Reformed author who convinces you that one of the load-bearing claims was wrong, and you now argue a modified position. Bump to 2.0.0. Claim.

A single revision can cross categories. If you fix typos and also add a new paragraph in the same pass, the bump is the highest level involved: 1.1.0, not 1.0.1 and then 1.1.0. Skip MINOR resets on MAJOR bumps too; 2.0.0 starts fresh at .0.0, not carrying forward the prior minor count.

When to use it

Version numbers are useful only when readers come back. A blog post nobody returns to doesn’t need versioning; a reference doc people cite does. The break-even is whether someone would need to know whether what they remembered is still what’s there. If yes, version. If no, don’t.

For personal writing in a Git repo, the commit history already records every change, and filename suffixes (_v2, _v3) mark coarse milestones for workflow reasons. Semver on top of that is a public-facing contract for readers who don’t read the commit log. The two schemes don’t conflict; they operate at different layers.

Honest gotchas

The claim/content/copy line isn’t always clean. A new section you add might implicitly shift the claim by strengthening one side of an argument you didn’t mean to weight more heavily. If the effect on the thesis is material, bump MAJOR even though mechanically you only added content. The question is always what the reader would conclude, not what you technically changed.

“Breaking change” for prose is softer than for code. Code either compiles or it doesn’t; a reader’s citation either supports their argument or it doesn’t, but the line is judgment-bound. Default to the stricter call: if you’re unsure whether the change is MAJOR or MINOR, call it MAJOR. Over-signaling is cheap; under-signaling breaks trust.

Most prose doesn’t accumulate enough revisions to make fine-grained versioning worth the overhead. The minimum worthwhile use case is something you expect to revise at least a few times a year, across a few years, for an audience that might refer back.

Recommendation

For any document you plan to maintain and expect to have return readers, use semver with the claim/content/copy mapping. Start pre-1.0 while drafting; release at 1.0.0 when you’re willing to stand behind the thesis publicly; bump per the rules above on every revision. State the scheme once in a footer or about-page; don’t re-explain it per document.

For ephemeral writing (blog posts, one-off essays, anything readers won’t revisit), skip versioning entirely. Filename suffixes in your own repo are enough for you; readers don’t need to see them.

Configuring Git well on day one

A dotfile of accumulated wisdom. The commands below compound enormously over years. Set them once; later sections refer to this as the day-one block.

git config --global user.name "Your Name"
git config --global user.email "you@example.com"
git config --global init.defaultBranch main
git config --global core.editor "codium --wait"
git config --global diff.tool vimdiff
git config --global difftool.prompt false
git config --global pull.rebase true
git config --global push.autoSetupRemote true
git config --global fetch.prune true
git config --global rerere.enabled true
git config --global commit.verbose true
git config --global commit.template ~/.gitmessage
git config --global core.hooksPath ~/.git-hooks

What these do:

1. user.name and user.email: stamped on every commit. The recommended pattern is to omit both from the global config, set user.useConfigOnly = true, and bind the identity to a directory via includeIf gitdir:, so a repo created outside that directory fails loud rather than committing under whatever was lying around in global config. See § “Identity layers and isolation” for the threat-model reasoning.
2. init.defaultBranch main: new repos use main instead of master.
3. core.editor: what opens for commit messages and interactive rebase. VSCodium example; substitute your editor.
4. diff.tool vimdiff plus difftool.prompt false: sets the diff tool for git difftool and skips its per-file confirmation. Substitute another tool if you don’t edit in vim.
5. pull.rebase true: pulls rebase onto the remote instead of creating merge commits. Linear history, less clutter. The deeper treatment is in § Pull behavior below.
6. push.autoSetupRemote true: the first push of a new branch creates the remote branch and sets the upstream automatically (Git 2.37+), instead of refusing until you run git push -u origin <branch> once.
7. fetch.prune true: every fetch drops remote-tracking branches whose remote branch was deleted, so git branch -a reflects reality instead of accumulating ghosts.
8. rerere.enabled true: “reuse recorded resolution.” Git records merge-conflict resolutions and replays them automatically when the same conflict recurs. The deeper treatment is in § rerere below.
9. commit.verbose true: when you commit, the staged diff appears in the editor below your message (stripped on save). Lets you describe what’s actually changing rather than what you remember changing.
10. commit.template ~/.gitmessage: pre-fills the commit editor with reminders of the 50/72/imperative rules as comment lines (stripped on save). The reference doc has the template content.
11. core.hooksPath ~/.git-hooks: redirects Git’s hook discovery from the per-repo .git/hooks/ to a user-managed directory, so the same hooks apply to every repo on your machine. The hooks themselves are covered in the Hooks section below.

Useful aliases, added under [alias] in your global ~/.gitconfig. The block below is a representative subset; the Git reference doc has the full convergent set used by long-term experts, grouped by purpose, with reliability notes.

[alias]
    c = commit -m
    ca = !git add -A && git commit -m
    last = show --compact-summary HEAD
    lg = log --oneline --graph --all --decorate
    st = status -s
    co = checkout
    br = branch
    unstage = restore --staged
    caas = commit --amend --no-edit
    pf = push --force-with-lease --force-if-includes

The first three are the everyday workflow: git c "msg" commits staged changes, git ca "msg" stages everything and commits, git last shows what you just committed (commit metadata, message, and a compact summary of changed files including new/deleted/mode-changed annotations). The pf alias uses --force-with-lease --force-if-includes, which refuses to overwrite the remote if someone else has pushed in the meantime; never alias plain push --force to a short name. For the fuller setup that goes with these — commit.verbose, the commit message template, the editor configuration — see the Git reference doc.

Most experienced users have a dotfile of Git configuration that’s grown over years and represents accumulated wisdom about friction points. Start yours now; add to it whenever you find yourself typing the same flag combination twice. The reference doc is the single source of truth for the alias inventory; come back here only for the principle, not the list.

Remotes and the distributed model

Everything so far happens inside one folder on one machine, and that is the complete system: Git is distributed, meaning every repo is a full standalone copy carrying the entire history, not a client of some server. A remote is just a bookmark: a name in your repo’s config, conventionally origin, pointing at the URL of another copy, whether on a forge like GitHub or Codeberg, on a server of yours, or in another directory on the same disk. Nothing you do locally (commits, branches, rebases, the lot) leaves your machine until you explicitly push. For a writer that is worth saying plainly: a Git repo is private by default, and version control does not imply publishing.

Four verbs cover all the traffic. clone copies a remote repo to your machine, history and all, and wires up origin for you. fetch downloads what the remote has that you don’t and updates your remote-tracking pointers (origin/main), touching none of your own work; it is the look-don’t-touch verb. pull is fetch plus integrate: it folds the fetched commits into your current branch, and how it folds them is the subject of the next section. push uploads your commits and moves the remote’s branch pointer forward, which the remote accepts only when your history contains its current tip, so you cannot silently overwrite work you haven’t seen.

Your branch and the remote’s copy of it are separate pointers that drift apart whenever either side moves. Git tracks the pairing (your main against origin/main, called the upstream), and git status reports the drift as ahead, behind, or diverged: ahead means you have commits to push, behind means commits to pull, diverged means both.

Diverged is where merge conflicts can appear. When both sides changed the same lines and Git is asked to integrate them, it stops, marks the overlapping region in the file, and waits for you to choose; nothing is lost, both versions are right there in the marked block. A conflict is Git declining to guess between two valid versions, not an error, and on solo single-machine work you may not meet one for months. The mechanics, markers and all, are in the reference; the integration choice that decides how often you meet them is next.

Pull behavior: rebase vs merge

The day-one block sets pull.rebase true. The setting changes how git pull integrates remote work with yours, and the choice has consequences worth understanding.

What git pull does without pull.rebase true

By default, git pull is git fetch followed by git merge. If you have local commits that the remote doesn’t, and the remote has commits that you don’t, the merge step produces a merge commit joining the two lines: a commit with two parents, a message like “Merge branch ‘main’ of “, and no actual content change of its own. Run this often enough and the log becomes a forest of these no-content merge bubbles. They’re real history (Git is faithfully recording that two divergent lines were joined) but they’re noise: nobody wrote those messages, nothing about them is worth reading later, and they obscure the actual commits.

What pull.rebase true does instead

It changes the second step from merge to rebase. After fetching the new remote commits, Git takes your local commits, sets them aside, fast-forwards your branch to the new remote tip, and replays your local commits one by one on top. The result is a linear history: remote commits first, then your local commits, no merge bubble. The log reads as if you’d worked sequentially on the latest version of main all along.

The cost: your local commits get new hashes. The replay produces new commit objects, because each new parent changes the hash. This is fine for unpushed work (the old hashes only existed locally), and it’s the operational form of the rule “rewrite local history freely, never rewrite pushed history without coordination.” If you’ve already pushed those local commits to a shared branch, the rebase will diverge from the remote and the next push will need force, which is a different kind of problem.

When rebase-on-pull is the wrong default

On a feature branch you share with several people who all commit and pull constantly, rebase-on-pull can rewrite commits other people have already based work on. The merge-on-pull default avoids that by keeping each person’s commits stable. Most modern workflows don’t have this problem (feature branches typically belong to one author), but if you find yourself on a shared mutable branch, switch the setting per-repo with git config pull.rebase false for that repo.

What neither setting protects you from

A worry that sounds related but isn’t: what if the remote commits themselves are ones you don’t want? Neither pull variant helps, because both integrate them; after either kind of pull, your branch contains the remote’s work, and merge versus rebase only chooses the topology of the join. The guard against suspect remote commits is inspection before integration: git fetch, then git log HEAD..origin/main to list what arrived (or git diff HEAD...origin/main to read it as one diff), then decide what to do. No alias is needed for the deciding, because git pull already takes per-invocation flags that override the configured default for that one pull: git pull --rebase, git pull --no-rebase, and git pull --ff-only.

There is also a third configuration stance worth naming: git config --global pull.ff only. With it, a pull succeeds only when your branch can fast-forward; any divergence makes the pull refuse, and you then choose --rebase or --no-rebase explicitly for that case. This is the fail-loud setting, the same philosophy as the identity setup in § Identity layers and isolation: nothing is integrated silently, and every divergence becomes a deliberate decision. Its cost is friction in the common case, where the divergence is your own work on two machines and rebase was always going to be the answer. The recommendation here stays pull.rebase true for that reason; if you would rather decide every time, ff-only is the principled way to get it.

The pull-rebase debate, in one paragraph

This is one of the contested edges flagged in the closing caveats. The “always rebase” camp values clean linear history; the “always merge” camp values not rewriting commits already shared with anyone. Both are defensible. The recommendation in this document — pull.rebase true as a global default — fits the most common modern pattern (feature branches owned by one person, merged to main via pull request), and is what most contemporary practitioners run. Adjust per-repo if your situation is different.

rerere

rerere stands for “reuse recorded resolution.” The day-one block enables it. It’s quiet and useful — quiet enough that you may use Git for years without realizing it’s working.

What rerere records

When you resolve a merge conflict, Git saves the pre-resolution conflict text and your post-resolution result in .git/rr-cache/. If the same conflict (same context lines, same conflicting hunks) appears later, Git applies your earlier resolution automatically.

When rerere actually fires

It triggers only on identical conflict hunks. If the surrounding context drifts even slightly, the cache doesn’t match and you resolve manually again. In practice this means rerere pays off when:

1. You rebase a long-lived feature branch against a moving main and the same conflict re-emerges each rebase.
2. You’re a maintainer integrating recurring patches and seeing the same merges play out.
3. You’re cherry-picking the same commit across multiple branches.

For solo personal work with occasional conflicts on stable files, rerere will almost never fire. It costs nothing to leave enabled and pays off invisibly when a workflow gets repetitive enough to need it.

Knowing when rerere helped

The replay is silent. After a rebase or merge, git rerere status shows which paths had recorded resolutions applied, and git rerere diff shows the resolution itself. If a recorded resolution turned out wrong (the surrounding code changed in a way that makes the old resolution incorrect), forget the cache entry with git rerere forget <path> and resolve manually.

Hooks

A Git hook is a script Git runs automatically at a specific point in the workflow. Hooks let you enforce policy locally: warn before committing under conditions you specify, lint files before push, format code on staging, abort a commit that violates a rule.

Hooks live as executable files in a directory Git inspects when relevant operations run. The default location is .git/hooks/ inside each repo. Files there with the right name (pre-commit, prepare-commit-msg, pre-push, post-merge, and others) get executed at their respective stages.

Two facts about hooks worth absorbing early:

1. Hooks are per-machine. The .git/ folder isn’t tracked by the repo itself. Cloning a repo does not give you its hooks. Each developer sets up their own hooks on their own machine. This is by design: hooks run arbitrary code, and pulling a repo would otherwise mean executing whatever the repo’s author put in their hooks. Tools like pre-commit (the framework) layer a tracked configuration over this so hooks can be shared, but the raw mechanism is local-only.

2. Hooks are opt-in. A fresh repo’s .git/hooks/ directory contains only .sample files, none of them executable. Git runs nothing unless you create executable hook scripts yourself.

core.hooksPath

To share hooks across all your repos rather than copying them into each one, set core.hooksPath to a user-managed directory (the day-one block sets this to ~/.git-hooks). Every repo on the machine then reads hooks from that single directory instead of its own .git/hooks/. Add an executable hook there once and it applies everywhere. The git-setup_v12.sh script uses this pattern to install a rename-safety hook that warns when a rename commit also contains substantial content changes (the failure mode covered in § “What Git silently discards / Renames”).

The trade-off: per-repo override is lost. If one repo genuinely needs different hooks, drop a .git/hooks/<name> in that repo and unset core.hooksPath for that repo with git config --unset core.hooksPath. For most personal workflows, the global pattern is right; the rare per-repo override is the exception.

.gitignore hygiene

A .gitignore file at the repo root tells Git which files never to track. Build outputs, dependencies, credentials, OS-generated clutter (.DS_Store), editor config, log files.

GitHub maintains a repository of language-specific templates at github.com/github/gitignore. For any new project, grab the relevant template as a starting point.

Global ignore for OS and editor clutter that should never be tracked, regardless of repo:

git config --global core.excludesfile ~/.gitignore_global

Example global ignore:

.DS_Store
Thumbs.db
*.swp
.idea/
.vscode/

If you’ve already committed a file that should be ignored, adding it to .gitignore won’t untrack it. You need:

git rm --cached path/to/file

This removes it from Git’s tracking without deleting it from disk. Commit the removal plus the .gitignore update together.

Committing things that don’t belong in version control (compiled binaries, node_modules, API keys) is one of those mistakes that’s annoying at best and catastrophic at worst. The .gitignore habit protects you from the whole class of problems.

Line endings

Line endings are an invisible footgun that fires the moment a repo crosses operating systems. Windows uses CRLF (\r\n) to terminate lines; Linux and macOS use LF (\n). Git stores whatever’s in the file. If one collaborator edits on Windows and another on Linux, every line of every text file will show as changed on the next diff, because the invisible \r characters appear and disappear with each save. The diff becomes useless: the actual change you made is buried under thousands of phantom line modifications.

Even solo work is exposed if you ever clone a repo that was edited cross-platform, or if you switch machines, or if some collaborator opens a file in a Windows editor and saves it back.

The fix is a .gitattributes file at the repo root. The core line:

* text=auto eol=lf

text=auto lets Git detect text vs binary files. eol=lf declares that text files should be stored and checked out with LF endings. The same file also handles per-pattern CRLF overrides for Windows-specific scripts, binary markers, and prose diff settings; see the Git reference for the combined template.

.gitattributes is committed to the repo, so every clone gets the same rules. This is different from .gitconfig settings (core.autocrlf, core.eol), which are per-machine and don’t travel with the repo. Prefer .gitattributes for anything that should apply consistently across collaborators.

If you’ve already accumulated mixed line endings in a repo, fixing them is a one-time renormalization:

git add --renormalize .
git commit -m "Normalize line endings"

This rewrites every text file’s stored form to match the .gitattributes rules. After this commit, future diffs are clean.

The cost of setting this up on day one is two lines in a .gitattributes file. The cost of not setting it up and discovering the problem six months in is debugging phantom diffs across hundreds of files. Add the .gitattributes before the first commit.

Never commit secrets

API keys, passwords, private keys, tokens. Once committed, they exist in the history forever. Even if you remove them in a later commit, anyone with access to the repo can find them in the history.

If you accidentally commit a secret:

1. Assume it’s compromised. Rotate the key, password, or token immediately at the issuing service.
2. Do the work to purge it from history. This is genuinely difficult; see the Git reference for git filter-repo and the gitdel_v2.sh script.
3. Force-push the rewritten history.
4. Tell any collaborators to re-clone.

The discipline of storing secrets in environment variables or untracked config files, never in code, needs to be a reflex from day one. The safest configuration is to never have the secret in a file that’s in a Git repo in the first place.

A useful pattern for the case where the application genuinely needs a config file: track config.example.json (or .env.example) in the repo with empty or dummy values, and gitignore the real config.json (or .env). New collaborators clone the repo, copy the example to the real name, fill in their own values. The schema of required config is version-controlled; the actual values never are. The example file doubles as documentation — anyone reading the repo can see what config the application expects without ever seeing real credentials.

Large binaries are permanent

Commit a 50MB PDF or a video file once and every clone of the repo carries that blob forever, even if you delete the file in the next commit. Git stores objects by content hash; once an object exists in history, it stays there until you rewrite history to remove it. The deletion commit only records that the working tree no longer contains the file. The blob itself remains in the pack files, downloaded on every clone, forever.

For solo work this looks tolerable for a while. Then the repo hits 2GB, clones take minutes, GitHub starts warning about repo size, and the cost compounds. For writers especially, the temptation is constant: draft PDFs, image assets, audio recordings of dictation, exported ePubs. Each one feels small in isolation. Across two years of work they add up to a repo that’s mostly historical binaries nobody will ever look at.

The default discipline: don’t track binaries in the main repo. Add patterns to .gitignore so the temptation is removed at commit time:

*.pdf
*.docx
*.zip
*.mp4
*.mov
*.mp3
*.wav

If you need to ship a binary as part of a release, attach it to the release on the forge (GitHub Releases, Codeberg Releases). The binary isn’t in the repo; it’s hosted alongside it.

When you genuinely need to version a binary (an image asset that evolves, a Word file an editor requires you to deliver), the standard tool is Git LFS (Large File Storage). LFS replaces the binary in the repo with a small pointer file; the actual binary lives on a separate LFS server and is downloaded only when checked out. The repo stays small and fast even when you’re tracking large evolving assets. Setup is a one-time git lfs install plus a .gitattributes entry:

*.psd filter=lfs diff=lfs merge=lfs -text

Not every forge supports LFS for free at scale (bandwidth quotas apply on GitHub); check before committing to it. For small repos with occasional binaries, plain Git plus a strict .gitignore is sufficient.

If you’ve already committed large binaries you wish you hadn’t, removing them requires history rewriting via git filter-repo, identical to the secrets purge described above. The reference doc covers the procedure. Treat it like a secrets incident: rare, deliberate, all-collaborators-affected. Better to prevent than to clean up.

Signed commits

To understand signed commits, start with what a normal commit’s “author” field actually is.

When you commit, Git stamps the commit with the name and email you set in git config --global user.name and user.email. Git takes your word for it. There’s no verification. Anyone with a clone of your repo can set their config to your name and your email and make commits that look exactly like yours. Push them to a server and the server has no way to know they didn’t come from you.

For most work this doesn’t matter. The threat model is small: nobody is forging your commits, because there’s nothing to gain. But consider what changes when stakes go up. You’re a contributor to Bitcoin Core. The codebase moves money. An attacker compromises your GitHub account (phishing, credential reuse, anything). They push commits as you. The commits look like they came from you because they have your name and email on them. Maintainers who trust you might merge them. Money moves. By the time anyone realizes, the damage is done.

A signed commit closes this gap. When you commit with signing turned on, Git uses a cryptographic key (GPG, or more recently SSH) that lives only on your machine to attach a signature to the commit. The signature is mathematically bound to the commit’s contents and to your private key. Anyone with your public key (which you’ve uploaded to GitHub) can verify two things: that the commit was made by someone who held your private key, and that the commit hasn’t been altered since you signed it. GitHub displays this verification as a green “Verified” badge next to the commit.

Why this matters: even if an attacker takes over your GitHub account, they don’t have your private key (which lives on your laptop, not on GitHub). They can push unsigned commits or commits signed with the wrong key, but they can’t produce signatures that match yours. Reviewers who require signed commits can reject the unsigned ones. The compromise gets caught.

This is why Bitcoin Core, the Linux kernel, and most security-critical projects either require or strongly encourage signed commits on the main branch. The Linux kernel adds another layer: a Signed-off-by: line in the commit message, which is a contributor’s legal attestation that they have the right to submit the change under the project’s license. That’s a different mechanism from cryptographic signing (it’s just text), but it serves a related purpose of making the chain of provenance explicit.

For solo work and most personal projects, signing isn’t necessary. Nobody is trying to impersonate you on your hobby blog repo. The threat model isn’t in play. Skip it; you’re not missing anything important.

For high-stakes contributions or projects where you want the assurance, set it up once and forget about it. Generate a GPG key (or use SSH signing, which is simpler if you already have an SSH key for GitHub). Tell Git to sign commits by default. Upload the public key to GitHub. From then on, every commit gets the green badge automatically. The Git reference has the exact commands.

One pragmatic note: most developers don’t sign, and the ecosystem hasn’t fully standardized. You’ll see plenty of unsigned commits from serious developers in serious projects, because the cost of setting up signing across multiple machines and keeping keys in sync isn’t trivial, and the benefit only kicks in for projects that actually verify signatures. Don’t read the absence of signatures as carelessness; it’s usually just rational triage. Read the presence of signatures as someone taking the supply-chain question seriously, which is increasingly the right disposition for anyone touching widely-used code.

Identity layers and isolation

Git and a forge like GitHub treat three things as independent:

1. The git author identity. The user.name and user.email stamped on each commit. Set locally. Git takes your word for it.
2. The forge account. The entity that owns the repo and appears in PR threads, issues, audit logs, and the contribution graph.
3. The SSH key. What authenticates a push to the forge. Each key is registered to exactly one forge account; the forge rejects the same public key uploaded to two accounts.

These three layers are independent at the protocol level. Nothing prevents you from stamping commits with one email while pushing them with a key registered to a different account; the forge attributes the push based on the key, and the commit metadata is whatever the local config wrote. What the key does and does not bind is a frequent point of confusion: an SSH key authenticates the push to an account, but the forge never checks the commit’s author name or email against the key, so there is no server-side way to lock a key to an identity. The lock is client-side, and it is the includeIf gitdir: directory binding the git reference describes: when inside the bound directory, the include file sets user.name, user.email, and core.sshCommand together, so the author identity and the key cannot drift apart for repos under that directory. user.useConfigOnly = true then makes a repo outside that directory fail loud rather than guess from whatever identity was lying around in global config.

For a single identity, the setup is to make all three layers describe the same person consistently: configured author identity, key uploaded to your one forge account, and the directory binding selecting them together by location. The git reference’s Identity setup section has the recipe.

For pseudonymous separation, keeping one identity unlinked from another where deanonymization would have real cost, config-level separation on one OS account is not enough. The protocol independence above means a single slip at any layer links the identities: an author email set wrong once is in commit history forever (unless you rewrite history); a push that goes out with the other identity’s key lands in the wrong account; a signature publicly binds a commit to a known key; a host compromise reads both ~/.gitconfig and ~/.ssh at once. The convergent pattern among people who actually need pseudonymous separation, used by Tor developers, Bitcoin Core’s pseudonymous contributors, and Qubes users, is OS-level isolation: a separate OS user account, a VM (Whonix on Qubes is the canonical setup), or a separate physical machine. Different home directory, different ~/.gitconfig, different ~/.ssh, different browser, different shell history. The “wrong terminal” class of failures disappears because the boundaries are real, not procedural.

This guide is single-identity on a single OS account by design. The git reference’s Identity setup section gives the fail-loud directory-bound setup; that’s the right tool for keeping one identity coherent, not for separating two. If you need a pseudonymous identity, run that same setup in a separate OS user account or VM and accept the friction.

Beyond passphrase keys

A passphrase-protected SSH key on disk is the baseline for any serious setup. If your threat model warrants more, three axes go further. None is required; each is the documented next step beyond the previous.

1. Hardware tokens. YubiKey or Nitrokey. The private key lives on the hardware token and never touches the filesystem. Even a fully compromised laptop cannot extract the key; an attacker can use the key only while the token is physically present and you’ve entered its PIN. Two integrations matter: GPG subkeys stored on the token (the long-standing pattern, used by Bitcoin Core maintainers and Debian developers), and FIDO2 SSH keys (ssh-keygen -t ed25519-sk, simpler, newer, narrower in scope). YubiKey-with-GPG-subkeys is the most widely-documented setup.

2. Network metadata via Tor. Pushing over SSH leaks the source IP to the destination host (which network is doing the push). Routing SSH through Tor with ProxyCommand or running git operations from inside Whonix protects this metadata. The destination sees an SSH connection from a Tor exit node; the SSH protocol itself is unchanged. Codeberg publishes an onion service for the same reason. Useful when the fact that an identity exists at a given network location is itself sensitive.

3. Commit signing as an independent layer. SSH (or GPG) signs commits cryptographically, separate from how the push happens. A signature proves a specific key authored a commit; an attacker who compromises your forge account but not your signing key can push fake commits, but they can’t sign them. Bitcoin Core requires signed merges to master; the Linux kernel uses GPG-signed tags on maintainer trees. For personal projects, signing matters mainly if the project requires it or if you want a verifiable record of which commits you actually authored.

These are orthogonal. You can use a YubiKey without Tor, Tor without signing, signing without a YubiKey. The natural progression is: passphrase keys (current), then signing (cheap to add, no hardware needed), then hardware tokens (real cost, real benefit), then Tor (situational, depends on what metadata you’re protecting).

What experts actually do depends on what they’re protecting. Bitcoin Core release maintainers sign with GPG subkeys on YubiKeys, work over SSH on regular Internet (release attribution is the threat, not network metadata). Tor developers work over Tor inside Whonix (network metadata is the threat, signing is sometimes a separate concern). For most contributors, a passphrase-protected SSH key plus commit signing (when the project requires it) is the realistic ceiling.

Tooling worth knowing about

You don’t need any of these to start. Knowing they exist means you can reach for them when you’re ready.

1. git-delta. Replaces less as Git’s pager with syntax-highlighted, optionally side-by-side output. Configure with git config --global core.pager delta. Useful when you’re reading diffs in the terminal; read-only, so for editing while diffing see the vim and VSCodium sections.
2. tig. Terminal-based Git history viewer. Faster than any GUI once you’re comfortable, and it works over SSH where a GUI can’t. Navigate commits with arrow keys, see diffs inline, stage interactively.
3. lazygit. Terminal UI for most Git operations. Many long-time command-line users find it more efficient than raw Git or a full GUI. Commit, rebase, cherry-pick, stash, branch management, all with single keystrokes.
4. gh. The official GitHub CLI. Create PRs, review them, manage issues, clone your own repos by name, all from the terminal. The GitHub reference covers this.

These are what experienced users accumulate over years. None is necessary on day one. Don’t install them defensively. Install them when you feel the specific friction they remove.

Slow down before destructive commands

The instinct when a Git command gives you an unexpected state is to run more commands trying to “fix” it. That’s how people actually lose work: not from the original problem, but from frantic recovery attempts.

Stop. Read the output. Understand what state you’re in. Look up what the recovery actually requires. Then proceed deliberately.

Ninety-nine percent of Git “disasters” are recoverable if you don’t panic. A small percentage become unrecoverable because someone force-pushed or hard-reset in a moment of frustration without reading the reflog first.

The commands that actually destroy work, in order of danger:

1. git push --force (without --force-with-lease). Overwrites remote history unconditionally.
2. git reset --hard when uncommitted work exists. Throws away the working directory without warning.
3. git clean -fdx. Deletes untracked files, including ignored ones.
4. git rebase mid-resolution when confused. Produces states that require careful manual surgery to unwind.

When in doubt, make a backup branch first. git branch backup-before-i-try-this costs nothing and buys you a trivial way to undo whatever you’re about to do.

What Git silently discards

Git’s storage model is lower-level than its surface commands suggest. Several high-level operations look like they’re recording something, but the recording happens by inference at read time, or doesn’t happen at all. The common pattern: the user assumes Git is preserving intent; Git is actually preserving snapshots, and reconstructs intent later by heuristic. Most of the footguns that catch experienced users belong to this category.

Renames

The clearest case. There is no rename operation in Git’s object model. A commit stores trees of blobs — snapshots, not operations. When you run git mv old.js new.js, Git records old.js as deleted and new.js as added, identical to rm old.js && cp something new.js. The rename intent is thrown away at commit time.

This means git log --follow and every rename-aware diff are post-hoc heuristics. Each invocation re-infers the rename from scratch by comparing content similarity across the delete/add pair. There is no stored fact to look up — just a guess from content.

Git’s default similarity threshold for rename detection is 50%. --follow uses that threshold. So --follow traces through a rename fine as long as the file’s similarity score stays above 50%; below that, it loses the thread entirely. The practical implication: keep rename commits content-pure. If you must edit a file at the same time you rename it, do the rename in one commit and the edit in a separate one. A rename-only commit has 100% similarity and is bulletproof for --follow; a rename-plus-substantial-edit commit risks dropping below 50% and silently breaking history traversal for that file forever afterward.

The reference doc has a prepare-commit-msg hook that inspects every commit as it is being prepared, normal or amend, and warns when it contains a rename below 60% similarity (10 points above the danger floor). See “One-time setup” there.

This is a known and longstanding criticism of Git’s design — renames would need to be first-class objects in the commit format to record intent reliably, not inferred from diffs. The burden falls on the user to keep renames content-pure so the heuristic can recover what the tool discarded.

Other things in the same category

Several other footguns are variants of the same pattern: Git’s model is lower-level than users expect, so what feels like a high-level operation either discards information at commit time or relies on inference that can fail silently.

1. Amending a pushed commit rewrites history for everyone else. --amend doesn’t alter the previous commit; it replaces it entirely with a new one that has a different hash. If the original was already pushed, anyone who pulled it now has a diverged history. The rule: only amend commits that haven’t been pushed.

2. git add . and git commit -a stage everything indiscriminately. Easy to commit debug code, credentials, build artifacts, or unrelated changes by accident. The discipline is git add <file> explicitly, and reviewing git diff --staged before every commit. The staging area was designed to give you control over what’s in the next commit; bypassing it gives up the safety it provides.

3. Merge vs rebase is a permanent, often-invisible choice about what history looks like. A merge commit preserves topology — you can see that work happened in parallel. A rebase rewrites commits as if they were always linear, discarding that topology. Most users pick one out of habit without realizing the choice is irreversible once pushed.

4. .gitignore doesn’t untrack files already tracked. The ignore patterns only filter the untracked set. If you committed a file and then added it to .gitignore, Git keeps tracking it. You have to git rm --cached <file> to stop tracking — and if the file contained credentials, the secret is still in history unless you also rewrite history (see the purging section of the reference).

5. git reset --hard and git clean -fd are not symmetric with git stash. Reset hard and clean destroy uncommitted working-tree changes permanently. The reflog saves committed history but cannot recover what was never committed. The default assumption that “Git always has a recovery path” only holds for committed work.

6. Submodules are not automatically updated on pull. A repo with submodules cloned without --recurse-submodules has empty submodule directories until you run git submodule update --init --recursive. People spend hours debugging missing files before discovering this.

The common thread is the same one the rename case makes vivid. Git stores snapshots and hashes, not intentions. Every high-level operation that looks like it’s recording semantics (rename, track, ignore, amend) either discards the semantics at commit time or infers them after the fact. Knowing this in advance prevents the class of mistake that comes from assuming the tool is doing more than it actually is.

For writers specifically

Everything above applies to writers as much as to programmers; Git doesn’t care what your files contain. But a writer’s relationship to version control has some specific textures that developer-oriented tutorials rarely name. This section is the pivot.

The deepest principle survives the translation: Git is a communication tool, not a backup tool. The conversation it enables is with your future self. For a writer, this is actually more evocative than for a programmer, because writers already know viscerally what it means to lose an earlier version. The paragraph you cut and wish you hadn’t. The opening you wrote six drafts ago that was somehow closer to the truth than where you ended up. Git is, fundamentally, the ability to never actually lose any version of anything while still being free to cut ruthlessly in the moment.

Commit messages as craft notes

The “why not what” rule has a different flavor for prose. The diff shows which words changed; it doesn’t show whether you changed them because the rhythm was off, because a reader said the paragraph confused them, because you realized the character wouldn’t actually speak that way, or because you were tired and second-guessing yourself at midnight. The message is where you record the intention behind the revision, and for a writer that intention is often artistic or emotional rather than technical.

Good commit messages for prose might look like:

1. “Cut the second section; felt like throat-clearing.”
2. “Rewrote opening after workshop feedback. Tessa was right, it was starting in the wrong place.”
3. “Restored earlier version of dialogue; the revised one lost the flatness I wanted.”

These are notes to yourself about craft. Years later, when you’re wondering why a piece evolved the way it did, these messages are a record of your own thinking as a writer. Worth having for its own sake, separately from the practical use of finding a specific version.

Lucia Berlin’s notebooks, Carver’s drafts, Didion’s edited manuscripts: we treasure these because they reveal the process. Git gives you, the writer, the ability to keep your own version of this record effortlessly, for your own later study or simply as a form of self-knowledge about how you actually work.

Rhythm of commits

For a developer, a commit often corresponds to a completed small task. For a writer, commit at natural pause points: end of a writing session, after a significant revision, after a workshop meeting when you’ve absorbed feedback, after you’ve “finished” a draft in the provisional way any draft is ever finished.

Don’t try to commit after every sentence. The unit of meaningful change in prose is usually the paragraph, the scene, the session.

Do commit before any major surgery. Before you cut a whole section, commit. Before you restructure, commit. Before the “what if I rewrote this in first person” experiment, commit. The freedom to experiment radically comes from knowing the previous version is safe.

The “one commit, one logical change” principle translates as: don’t mix unrelated revisions in the same commit. If you sat down to fix typos and ended up also rewriting the ending, those are two different creative acts and deserve two different commits. This matters when you want to look back and remember what you changed and why.

Branches as creative experiments

Branches change meaning for a writer, and arguably become more interesting than they are for most developers. In code, branches are a logistical tool. In writing, branches are a tool for actual creative experimentation. The ability to say “let me try rewriting this story from the sister’s point of view, fully, not just as an exercise but as an alternative version I can live inside for a while, while keeping the current version completely safe.”

Make a branch. Do the experiment. If it works, merge that version in or keep both as separate artifacts. If it doesn’t, delete the branch and return to exactly where you were.

Writers often resist radical experiments in revision because they fear losing what they have; branches dissolve that fear entirely. This is probably the single most transformative use of Git for creative work, and it’s underappreciated because most Git tutorials are aimed at developers who think of branches more prosaically.

Making prose diff well

Git’s default diff works at the line level. For code that’s fine; code is structured by lines. For prose, a line often contains an entire paragraph, and a one-word change shows the whole paragraph as removed and the whole new version as added. The actual change drowns in noise.

There are two fixes. They solve the same problem from opposite directions, and the right choice depends on whether you want to change how you write or how you read.

The write-time fix is semantic line breaks: put each sentence on its own line in the source file. Markdown renders identically whether sentences sit on separate lines or flow in a paragraph (a single newline becomes a space; a blank line becomes a paragraph break), so the rendered output is unchanged. But the source-level shape is now line-oriented in the way Git already expects. Insert a sentence and you touch one line. Reword a sentence and the diff shows that one line changed, not the whole paragraph. The default git diff does the right thing without any configuration. This is strictly better for diffs and costs nothing at render time. New users almost never know this convention exists, because it’s invisible in the rendered output; it only pays off once you start reviewing your own history. Most serious Markdown-based documentation projects on GitHub follow it for exactly this reason.

The trade is editor-side: the source looks ragged in a plain editor, and some auto-formatters will reflow it back into paragraphs (configure them not to, or disable on Markdown). For documents you’ll diff repeatedly across years, the rag is worth the legibility.

The read-time fix is --word-diff, which tells Git to highlight changes at the word level even when whole paragraphs are on one line:

git diff --word-diff
git diff --word-diff=color

For a repo-wide default on text file types, add this to a .gitattributes file at the repo root:

*.md    diff=word
*.txt   diff=word
*.tex   diff=word

GitLens in VSCodium has similar settings for word-level or character-level diffs.

The two approaches compose: a repo written with semantic line breaks and configured for word-diff gives you the cleanest possible prose diffs at both granularities. The line-level diff shows which sentences moved or changed; the word-level view shows what changed inside those sentences. If you only do one, do semantic line breaks. It’s tooling-free, travels with the file rather than the config, and works in every Git interface including forge web views.

Plain text, not Word

For serious writing work, write in plain text formats (Markdown, reStructuredText, plain text, LaTeX) and only convert to Word or PDF for delivery. Plain text is what Git is built for. Word documents technically work but you lose most of the benefit, because you can’t see what changed. Git stores the whole binary as “this file changed” without being able to show a meaningful diff.

If you must use Word (an editor insists, a contract requires it), do the editing in plain text, convert to Word at the final step, and commit the Word file at that point. Version the markdown. Deliver the Word.

Don’t let tooling become procrastination

The entire apparatus of version control, tooling, configuration, organizational scaffolding is in service of the actual writing. It’s very easy to spend a whole afternoon perfecting your Git setup as a way of not confronting a difficult scene.

Set things up well once. Establish simple habits. Let the system fade into the background. The writing is the thing. Git is the frame that keeps the writing safe and legible across time. If you find yourself thinking about Git more than writing, you’ve inverted the relationship.

Three things to keep in mind

Three framings worth holding across years of use. Not commands, not mechanics. Principles about how to learn Git, what you get out of using it, and how to evaluate advice about it.

Mental model before commands

Start by internalizing the object model (blobs, trees, commits, refs) and the branches-as-pointers idea. Once those are solid, specific commands become obvious in shape: what they do, why the flags exist, what they cost. The reverse path, memorizing commands first and hoping the model assembles itself from fragments, takes longer and produces worse intuition. It also makes recovery from unfamiliar states harder, because you’re pattern-matching to commands rather than reasoning about what’s actually in the repo. Everything in this document is structured around that priority: model first, mechanics second.

Commit messages are a prose practice

Every commit message forces a small compression: what changed, why, for whom. Over years the practice sharpens your prose in general, because compressing a thought into a summary line is exactly the skill long-form prose depends on. Writers who version their work get this benefit twice: the prose they commit improves, and the messages about the prose improve too. Treat the summary line the way a copy editor treats a headline: one line, specific, load-bearing. Treat the body the way you treat a good paragraph: context first, reasoning second, caveats last. This is work you were going to do anyway as a writer. Git just gives it a daily occasion.

AI advice on Git has a known asymmetry

AI answers about Git are usually smooth, syntactically correct, sometimes elegant, and can still miss what a practitioner who has lived with a particular failure mode for five years knows. The gap is widest on judgment calls (when to rebase vs merge, when to sign commits, when to force-push, when to rewrite history) and on edge cases that only surface after thousands of hours of real use. Weight AI advice accordingly: fine for mechanics and syntax, less reliable for “should I.” For judgment calls and high-stakes operations, cross-check against practitioner sources: Pro Git, kernel mailing list threads, Bitcoin Core maintainer posts, Tim Pope’s essay, the man page. The AI gives you a plausible answer quickly; practitioner sources give you the answer that survived contact with real projects over real time. Both are useful. Don’t confuse them.

Closing: deliberate practice

Git rewards deliberate practice and punishes casual use. The people whose repositories you admire (Torvalds, the Bitcoin Core maintainers, whoever) didn’t arrive at their practices by being naturally gifted. They arrived by making mistakes early, thinking carefully about what went wrong, adjusting their habits, and being consistent over years.

Every piece of advice in this document is really a variant of “take it seriously, even when the stakes feel low, because the habits you build on small projects are the habits you’ll have on consequential ones.”

You won’t look back in twenty years and thank anyone for any specific command. You’ll thank them, if they’ve done their job, for conveying that Git is a craft; that the craft is mostly about clarity and discipline rather than cleverness; and that the small investments in doing it well early are what make the later years pleasant rather than painful.

Consistency over cleverness, across time, is the whole game.

Honest caveats on best practice

Some edges of “best practice” in Git are contested. The pull-rebase-versus-merge debate. The squash-versus-preserve-history debate. The when-to-rewrite-history debate. Reasonable, experienced people disagree.

Don’t take any of this as dogma. Take it as a strong starting position that will serve you well, and adjust over time as you develop your own views through experience. The framework matters more than any specific rule within it.

A floor, not a ceiling

This document is deliberately not comprehensive. It covers the object model and the habits that pay off over years, and it stops there, because the goal was never to memorize every command but to push you toward seeing what Git actually is underneath. Everything here rests on one fact: a repository is a small content-addressed database of blobs, trees, and commits, and the everyday commands are a convenience layer over reading and writing those objects. Once that graph is something you can hold in your head, the commands that used to feel dangerous stop being dangerous, because you are reasoning about what is actually stored rather than pattern-matching to remembered incantations. That fluency is what makes someone the person a team runs to when history gets tangled. The single best hour you can spend toward it is Tim Berglund’s “Git From the Bits Up” (in Further reading below), which takes Git apart to the raw objects and rebuilds a commit by hand from plumbing alone. Watch it, then do it yourself once; the day it clicks is the day Git stops being a pile of commands and becomes a database you happen to drive from a command line.

Where Git came from

Git’s design makes more sense once you know what it was reacting against.

Version control before Git was overwhelmingly centralized: one authoritative copy of the history lived on a server, and your working copy was a client of it. The lineage ran through RCS, then CVS, then Subversion.

RCS (Revision Control System, Walter Tichy, around 1982) versioned one file at a time on a single machine, locking a file so only one person could edit it at once. CVS (Concurrent Versions System) grew out of RCS: Dick Grune’s 1986 shell scripts, rewritten in C by Brian Berliner around 1989. CVS added a network server and concurrent editing, and it became the de facto standard for open-source projects through the 1990s. But it inherited RCS’s core limitation: it tracked each file’s revisions separately, with no notion of a project-wide snapshot, no atomic commit (an interrupted commit could leave the repository half-written), and no real way to follow a file across a rename.

Subversion (CollabNet, begun 2000, 1.0 in February 2004) was built explicitly as “CVS done right.” It fixed the worst of it: commits became atomic and repository-wide, with a single revision number advancing across the whole tree. But it stayed centralized: the server still held the only complete history, every commit needed the network, and branching and merging remained heavyweight. At the moment Git appeared, Subversion was the rising standard and CVS the fading one.

The immediate trigger was something else. The Linux kernel had, from 2002, been using BitKeeper, a proprietary distributed system offered to kernel developers free of charge. Distributed meant each developer held a full copy of the history rather than renting access to a central one, and the kernel community valued that enough to depend on a closed tool. In April 2005 the arrangement collapsed: after Andrew Tridgell wrote a tool to interoperate with BitKeeper’s protocol, BitMover treated it as a breach of the gratis license and withdrew free access. The kernel was left without version control it was willing to use.

Linus Torvalds wrote the first version of Git that same month, April 2005, and had it self-hosting within days. The design was a direct response to the whole lineage above. Distributed, because BitKeeper had shown the value and the central-server model had shown the cost: every clone is a full, independent copy of the history (developed in Remotes and the distributed model). Snapshot-based and atomic, because CVS’s per-file, non-atomic model was the thing to escape: a commit records the entire tree at once (developed in How commits actually work). Content-addressed and cryptographically chained, because trust across thousands of contributors required history to be tamper-evident rather than merely stored (developed in the object-model sections, and in the Torvalds talk under Further reading). Fast at branching and merging, because a kernel-sized project lives or dies by how cheaply work can diverge and rejoin.

Much of what the rest of this document treats as fundamental was, originally, a deliberate correction of something a prior system did badly.

Further reading

1. Pro Git by Scott Chacon, free at git-scm.com/book. Read it once in your first year and once again after a few years of real use. Different sections will matter at different stages. For the object model specifically, the “Git Internals” chapter (git-scm.com/book/en/v2/Git-Internals-Git-Objects) walks through building a commit from git hash-object, git write-tree, and git commit-tree by hand.
2. “A Note About Git Commit Messages” by Tim Pope (2008). Short. Internalize and move on.
3. git help <command>. The manual pages are dense but accurate. The ability to read them directly, rather than relying on Stack Overflow answers of uncertain quality, is a long-term advantage.
4. “Git for Ages 4 and Up” by Michael Schwern (recording from linux.conf.au 2013; recording at youtube.com/watch?v=1ffBJ4sVUb4). Schwern builds a Git repository out of children’s construction toys while running the equivalent commands, so the talk shows what is actually happening inside Git rather than teaching the command surface. Blobs, trees, commits, branches, the staging area, remotes, and rebase all become physical objects you can see. The gentlest on-ramp to the object model; watch it first if the graph has never clicked.
5. “Version Control (Git)” by MIT’s The Missing Semester of Your CS Education (Lecture 6, 2020; ~1h25m; recording at youtube.com/watch?v=2sjqTHE0zok; notes and exercises at missing.csail.mit.edu/2020/version-control/). The lecture deliberately teaches Git’s data model before the command interface, a repository as a content-addressed store of blobs, trees, and commits forming a DAG, and then derives the everyday commands from it. The fullest free lecture in the data-model-first tradition; watch it once you want the whole model laid out end to end.
6. “Tech Talk: Linus Torvalds on git” (Google Tech Talk, 2007; recording at youtube.com/watch?v=4XpnKHJAok8). Torvalds, who wrote Git, explains at Google why it is built the way it is: distributed rather than centralized, content-addressed and cryptographically chained so history is tamper-evident, and optimized for merging and trust across a large contributor base. Not a tutorial but the design-rationale talk that explains the why behind the model the other talks teach; also a candid window into the priorities that shaped the tool.
7. “Git From the Bits Up” by Tim Berglund (JAXConf, 2013; ~55 min; recording at youtube.com/watch?v=MYP56QJpDr4). An advanced talk, explicitly not for beginners, that takes Git apart to its raw objects and rebuilds a commit by hand using low-level plumbing. The canonical “see the object graph” talk; watch it once you are comfortable with the daily workflow.
8. “git: not just for source code anymore” by Josh Triplett (linux.conf.au, January 2013; recording at youtube.com/watch?v=3-vAh9uDItY). Triplett, a longtime Linux kernel and Debian developer, makes the case that Git is a general-purpose, tamper-evident versioning engine that just happens to be famous for code: the same blobs, trees, and commits version configs, server state, datasets, and entire wikis. Worth watching once to see how much of “Git is for code” was convention rather than design.

Start now. Be consistent. The value compounds.

Git reference

Lookup by task. The concepts doc covers the mental model and the why; this one tells you the command.

Conventions in this doc:

1. Commands shown with bash tags.
2. Angle-bracket placeholders like <file> or <sha> mean substitute your own value.
3. HEAD~N means “N commits before the current one.” HEAD~1 is the previous commit.
4. Anything marked “local only” rewrites history; never run on commits that have been pushed to a shared remote.

One-time setup

Set your identity. Stamped on every commit; run once per machine.

git config --global user.name "Your Name"
git config --global user.email "you@example.com"

Recommended: don’t set a global identity. Turn on fail-loud and bind the identity to a directory instead, so a repo created outside that directory errors rather than committing under whatever was lying around in global config. See Identity setup.

Set the default branch name to main so new repos use it.

git config --global init.defaultBranch main

Set the editor Git opens for commit messages and interactive rebase.

git config --global core.editor "codium --wait"

Set vim as the diff tool for git difftool. Skips the per-file confirmation prompt. Use this if you read diffs in vim (with or without vim-fugitive); see the concepts doc for how this fits among the other diff-viewing options.

git config --global diff.tool vimdiff
git config --global difftool.prompt false

Make git pull rebase instead of merge. Keeps history linear; no noise commits.

git config --global pull.rebase true

Push new branches without the -u dance. With this set, the first push of a new branch creates the remote branch and sets the upstream automatically (Git 2.37+); without it, Git refuses until you run git push -u origin <branch> once.

git config --global push.autoSetupRemote true

Prune deleted remote branches on every fetch. Without it, branches deleted on the remote linger in git branch -a output indefinitely.

git config --global fetch.prune true

Enable rerere (reuse recorded resolution). Git remembers how you resolved a merge conflict and applies the same resolution if the conflict recurs.

git config --global rerere.enabled true

Make git commit show the staged diff in the editor (below a scissors line; the diff gets stripped on save). Reading the diff while writing the message is what most experienced users do reflexively. Available since Git 2.9.

git config --global commit.verbose true

Point Git at a commit message template. The template pre-fills the editor with reminders of the 50/72/imperative rules as comment lines (which get stripped on save).

git config --global commit.template ~/.gitmessage

Create the template file:

cat > ~/.gitmessage <<'EOF'


# Subject (line 1): imperative, under 50 chars, no period.
# Blank line between subject and body.
# Body (line 3+): wrapped at 72. Explain why, not what.
# The diff below the scissors line shows what.
EOF

The two leading blank lines are intentional: that’s where the cursor lands when the editor opens.

Codium settings for commit-message editing. Open Settings, click the JSON icon top-right, merge into settings.json:

"[git-commit]": {
    "editor.rulers": [50, 72],
    "editor.wordWrap": "wordWrapColumn",
    "editor.wordWrapColumn": 72
}

Vertical rulers at columns 50 and 72; word wrap activates at column 72.

Install a rename-safety hook. Git records renames as delete+add and re-infers them post-hoc when you run git log --follow; that inference is content-similarity based, and --follow loses the thread when the rename commit’s similarity drops below Git’s 50% default. The hook below inspects the commit being prepared, normal or amend, and warns when it contains a rename below the 60% safety margin (10 points above the floor). See the concepts doc, “What Git silently discards,” for the fuller framing.

mkdir -p ~/.git-hooks
cat > ~/.git-hooks/prepare-commit-msg << 'EOF'
#!/bin/sh
# Warn when the commit being prepared contains a rename with substantial
# content changes (similarity <60%); such commits break `git log --follow`.
# $2 is the message source; $3 is HEAD when the source is an amend.
if [ "$2" = "commit" ] && [ "$3" = "HEAD" ]; then
  base=HEAD~1   # amend: the new commit's parent is HEAD's parent
else
  base=HEAD     # normal commit: the new commit's parent is HEAD
fi
git rev-parse --verify --quiet "$base" >/dev/null || exit 0   # initial commit: nothing to compare
if git diff --cached --find-renames --name-status "$base" | grep '^R' | awk '{score=substr($1,2)+0; if(score<60) found=1} END{exit !found}'; then
  echo "Warning: this commit contains a rename with substantial content changes (similarity <60%). git log --follow may lose history. Rename in one commit; edit in a separate commit." >&2
fi
exit 0
EOF
chmod +x ~/.git-hooks/prepare-commit-msg

Point Git at that hooks directory globally:

git config --global core.hooksPath ~/.git-hooks

The hook checks the staged diff against the commit’s actual future parent: HEAD for a normal commit, HEAD~1 for an amend, whose parent is the old commit’s parent and whose content the index already holds at hook time. It covers both the fresh rename-plus-edit commit and the pure-rename commit about to receive edits via amend, skips cleanly on a repo’s first commit, and warns without ever blocking.

Useful aliases. Add to ~/.gitconfig under [alias], or set with git config --global alias.<name> '<value>'. The set below is the convergent core that shows up across long-stable expert configurations (mwhite, Haacked, Pro Git, Atlassian, oh-my-zsh’s git plugin, GitHub’s published dotfiles), grouped by purpose. Each line is what gets typed; the comment is what it does.

[alias]
    # Status
    s   = status                              # full status
    st  = status -s                           # short, one line per file

    # Stage
    a   = add
    ap  = add -p                              # interactive hunks (atomic commits)

    # Commit
    c       = commit -m                       # one-line message
    ca      = !git add -A && git commit -m    # stage everything + one-line message
    caa     = !git add -A && git commit --amend --no-edit
                                              # stage everything + fold into prev commit, keep message
    caas    = commit --amend --no-edit        # fold ALREADY-STAGED into prev commit, keep message
    caam    = "!f() { git add -A && git commit --amend -m \"$1\"; }; f"
                                              # stage everything + fold into prev commit, NEW message
    caams   = commit --amend -m               # fold ALREADY-STAGED into prev commit, NEW message
    cm      = commit                          # opens editor for substantive message
    cf      = commit --fixup                  # mark fixup; pair with ria
    fixto   = "!f() { sha=$(git rev-parse \"$1\") && git add -A && git commit --fixup \"$sha\" && if base=$(git rev-parse -q --verify \"$sha^\"); then GIT_SEQUENCE_EDITOR=: git rebase -i --autosquash \"$base\"; else GIT_SEQUENCE_EDITOR=: git rebase -i --autosquash --root; fi; }; f"
                                              # stage everything + fold into <commit>, root-aware; note 8
    fixtos  = "!f() { sha=$(git rev-parse \"$1\") && git commit --fixup \"$sha\" && if base=$(git rev-parse -q --verify \"$sha^\"); then GIT_SEQUENCE_EDITOR=: git rebase -i --autostash --autosquash \"$base\"; else GIT_SEQUENCE_EDITOR=: git rebase -i --autostash --autosquash --root; fi; }; f"
                                              # fold ALREADY-STAGED into <commit>, root-aware; note 8
    rewordto = "!f() { sha=$(git rev-parse \"$1\") && if [ -n \"${2:-}\" ]; then target_subject=$(git log -1 --format=\"%s\" \"$sha\") && git commit --allow-empty -m \"amend! $target_subject\" -m \"$2\"; else git commit --allow-empty --fixup=reword:\"$sha\"; fi && if base=$(git rev-parse -q --verify \"$sha^\"); then GIT_SEQUENCE_EDITOR=: git rebase -i --autosquash \"$base\"; else GIT_SEQUENCE_EDITOR=: git rebase -i --autosquash --root; fi; }; f"
                                              # replace message of <commit>; editor or inline; no content change; note 9
    foldinto = "!f() { git diff --quiet && git diff --cached --quiet || { echo \"foldinto: working tree not clean; commit or stash first\" >&2; return 1; }; orig=$(git rev-parse HEAD) && tgt_subj=$(git log -1 --format=%s \"$2\") && if base=$(git rev-parse -q --verify \"$2^\"); then base_arg=\"$base\"; else base_arg=\"--root\"; fi && git rewordto \"$1\" \"fixup! $tgt_subj\" && GIT_SEQUENCE_EDITOR=: git rebase -i --autosquash \"$base_arg\" || { git rebase --abort 2>/dev/null; git reset --hard \"$orig\" >/dev/null; echo \"foldinto: fold conflicts; restored to $orig. Resolve manually with: git rebase -i <target>^\" >&2; return 1; }; }; f"
                                              # fold existing <commit> into <target>; clean-tree, atomic; note 10
    cs      = commit -s                       # signoff (DCO projects)

    # Diff
    d   = diff
    dc  = diff --cached                       # the about-to-commit diff
    ds  = diff --stat                         # filenames + line counts
    dw  = diff --word-diff                    # word-level (good for prose)

    # Log
    lg  = log --oneline --graph --all --decorate    # standard pretty log
    ll  = log --oneline                              # quick scan, no graph
    lf  = log --follow --                            # file history, follows renames

    # Inspect
    last     = show --compact-summary HEAD          # what you just committed
    when     = log -1 --format=%cr --               # last touch on a file (relative)
    lastfile = log -1 --format='%cs %s' --          # last touch (date + subject)
    who      = shortlog -sne                        # contributors with counts
    aliases  = config --get-regexp '^alias\\.'      # list your own aliases

    # Branch / switch
    br  = branch
    co  = checkout
    cob = checkout -b
    sw  = switch                              # modern checkout (branches only)
    swc = switch -c                           # modern checkout -b

    # Restore
    unstage = restore --staged
    discard = restore                         # discard working-tree edits to file

    # Reset (commit rewriting)
    uncommit = reset --soft HEAD~1            # undo last commit, keep changes staged
    squash   = reset --soft                   # use as: git squash HEAD~3, then git commit
    # Rebase
    ria = rebase -i --autosquash              # auto-orders --fixup commits
    rc  = rebase --continue
    ra  = rebase --abort

    # Stash
    sl  = stash list
    sa  = stash apply
    sp  = stash pop

    # Push
    pf  = push --force-with-lease --force-if-includes    # the safe force push

Reliability notes on the alias set above.

1. pf uses --force-with-lease --force-if-includes, not plain --force. The lease form refuses to push if the remote has new commits since your last fetch; --force-if-includes (Git 2.30+) additionally verifies your local history includes everything from the remote ref before allowing the push. Together they prevent silent overwrites of someone else’s work. Never alias plain --force to a short name.
2. There are four amend-related aliases (caa, caas, caam, caams) and a deliberate one omitted (commit -a --amend). caas and caams operate on already-staged changes; you control what gets folded in, with caams also taking a new message. caa and caam both run git add -A first, which stages new files in addition to modified ones. commit -a --amend only stages modified tracked files, not new ones, which can lead to commits that omit the file you thought you were including. If you want one-step “stage everything and amend,” prefer caa/caam over commit -a --amend. All three of these rewrite the previous commit, so the local-only rule applies: don’t run on pushed commits without a force-with-lease push afterwards.
3. There is deliberately no alias for reset --hard or clean -fdx. The typing friction is the safety. Reflog can recover lost commits but cannot recover uncommitted working-tree changes a hard reset destroys.
4. sw / swc (Git 2.23+) and restore separate what checkout did into two cleaner commands. They refuse some operations that checkout would silently complete, which is desirable. The older co / cob are kept for muscle memory; both forms work indefinitely.
5. The cf + ria pair is the standard atomic-commit fixup workflow used on Bitcoin Core, the Linux kernel, CPython, and other projects with rebase-based PR review. git cf <sha> records a fixup against an earlier commit; git ria <base> runs interactive rebase with autosquash, which automatically orders and squashes the fixups into their targets.
6. Shell-form aliases (the ones starting with !) are ca, caa, caam, fixto, fixtos, rewordto, and whoami. The leading ! makes git run a shell command rather than a git subcommand. Shell aliases execute relative to the directory where git was invoked, not the repo root. For the ones that call git add -A (ca, caa, caam, fixto) this matters because git add -A always operates from the repo root regardless of where you invoke from, which is what you want for snapshot-style commits but worth knowing if you ever expected scoped staging.
7. The aliases entry uses '^alias\\.' with a doubled backslash. That’s gitconfig escaping: the value is parsed before being passed to the regexp engine, so \\ becomes \ and the pattern matches a literal dot. If you copy this pattern outside gitconfig (into a shell script, for instance), unescape back to a single backslash. Relatedly, a value containing ; or # must be wrapped in double quotes in gitconfig, since both characters otherwise begin a comment; that is why caam, fixto, and fixtos are double-quoted above.
8. fixto resolves its argument to a hash with rev-parse before committing, because creating the fixup commit shifts every HEAD~N name by one; without the resolution, fixto HEAD~1 would record the fixup against the wrong commit. It stages everything (git add -A, so the ca/caa caveat in note 2 applies), then runs the autosquash rebase non-interactively by setting GIT_SEQUENCE_EDITOR=:, which accepts the generated todo unchanged. Both fixto and the staged-only fixtos are root-aware: when the target is the root commit they rebase with --root, because the target’s parent <hash>^ does not exist and naming it aborts the rebase and strands the fixup! commit at HEAD. fixtos differs only in that it skips git add -A, folding just the staged index, and adds --autostash so the unstaged remainder it leaves does not block the rebase, which refuses to run on a dirty working tree. Either form rewrites every commit from the target forward, so the local-only rule applies; the fold conflicts when a later commit changed the lines around your change, because the patch’s context no longer matches at the target, and on that conflict the rebase stops as usual: resolve and git rc, or back out with git ra.
9. rewordto is the message-only sibling of fixto, with two interfaces selected by whether you pass a second positional argument. With no second arg (git rewordto <sha>), it uses git commit --allow-empty --fixup=reword:<sha> (Git 2.32+, June 2021, shorthand for --fixup=amend:<sha> --only), which creates an empty amend! commit and opens the editor with the target’s old message pre-filled for you to refine; save and close, and the autosquash rebase folds it in. With a second arg (git rewordto <sha> "new title"), it constructs the amend! commit manually with two -m flags (the target’s subject becomes the first, prefixed with amend!; your inline argument becomes the body, which is what autosquash installs as the target’s replacement message), bypassing the editor entirely. The two forms exist because -m and --fixup=reword: are mutually exclusive at the git level (fatal: options '-m' and '--fixup:reword' cannot be used together), so the inline form has to go through the manual amend! construction; the editor form goes through --fixup=reword: because it gives you the target’s old message as the starting point and lets commit.verbose and commit.template apply. Use the inline form for single-line subject changes (the common case); use the editor form for multi-line message rewrites where you want the editor as your composition surface. --allow-empty is defensive in both branches; the commit has the same tree as its parent. Same root-awareness as fixto. Same local-only constraint: the rewrite changes the target’s hash and every descendant hash, so don’t run on commits already pushed to a shared remote. The pre-commit hooks fire on the empty commit too; the rename-safety hook is a no-op there (no rename in an empty diff), but any custom hook that fails on empty commits will block this alias. The amend! autosquash lands on the commit named by <sha>, not on whichever commit happens to share its subject: because the rebase is based at the target’s parent, the target is the oldest commit in the range, and autosquash attaches the amend! to the oldest in-range commit with the matching subject, so a duplicate subject newer in range loses to the target and any older duplicate sits below the base untouched. The same <sha>^-windowing protects fixto, fixtos, and foldinto.
10. foldinto folds an existing commit into an earlier existing commit, the case fixto and fixtos cannot reach because their source is the working tree or index rather than a commit. It marks the source as fixup! <target subject> by calling rewordto, then runs an autosquash rebase based at the target’s parent (--root when the target is the root commit), so the operation is two rebases: the message rewrite, then the fold. The fold lands on the commit you name regardless of duplicate subjects, by the same <sha>^-windowing noted for rewordto in note 9: basing at the target’s parent makes the target the oldest commit in range, autosquash folds into the oldest in-range match, older duplicates sit below the base, and newer in-range duplicates lose. It requires a clean working tree and refuses otherwise, so its failure path can hard-reset to the starting commit without endangering uncommitted work; on a conflict it aborts the rebase, resets to the HEAD it captured on entry, and reports that you should redo the fold by hand with git rebase -i <target>^. This is the one place the fold family does not leave you inside the rebase to resolve in place: fixto and fixtos stop for git rc, but foldinto restores and hands the conflict back. Same local-only constraint as the rest of the family: it rewrites the target and every descendant hash, so don’t run it on commits already pushed to a shared remote without a coordinated force-with-lease push afterwards. It depends on rewordto being defined.

Check what’s configured.

git config --list
git config --list --show-origin   # values + which file each came from
git config --global --edit        # opens the global config file directly
git aliases                       # list just your aliases (after adding the alias above)

git config --list --show-origin is the unified view: every effective setting and the file it came from (system, global, per-repo, includeIf chains). Run inside a repo to confirm the identity is resolving correctly; run outside any identity directory to see the fail-loud state (no user.email set anywhere).

Automated setup

The script git-setup_v12.sh in this project applies everything in this section in one go. If you’ve just installed Git on a fresh machine, this is how to get to a fully configured setup in two minutes without typing each git config --global command by hand.

Quick start

1. Open git-setup_v12.sh in an editor and edit the CONFIG block at the top: set REAL_NAME and REAL_EMAIL to your name and email, and REAL_DIR to the directory your repos will live under (default ~/code/me).

2. Run the script:

   bash git-setup_v12.sh
   

3. Verify the result:

   git config --list --show-origin
   

That’s the whole flow. If the script printed Done. at the end, your setup matches what’s documented in the rest of this section.

What it does

The script does exactly what the rest of “One-time setup” above instructs, but as one command instead of thirty:

1. Sets the global config to fail-loud: user.useConfigOnly = true and no global identity (it unsets any user.name/user.email an earlier run left). Also sets default branch, editor, diff tool, pull/push/fetch behavior (pull.rebase, push.autoSetupRemote, fetch.prune), rerere, commit verbose mode, message template, and hooks path.
2. Generates ~/.ssh/id_ed25519 if missing (passphraseless, ed25519). Prints the public key so you can paste it into GitHub.
3. Writes the directory-bound identity to ~/.gitconfig-me (name, email, SSH signing with the key, and a core.sshCommand pinning that key), chmod 600s it, and registers an includeIf for REAL_DIR.
4. Sets every alias from the table above, including whoami.
5. Writes the commit message template to ~/.gitmessage.
6. Writes the rename-safety hook to ~/.git-hooks/prepare-commit-msg.

Nothing magic; you could do all of this by hand by reading the script. The script is a shortcut, not a different mechanism.

Why “idempotent” matters

The script is idempotent, meaning re-running it produces the same result as running it once. There’s no “already-set-up” state the script needs to detect or avoid. If you change REAL_EMAIL in the CONFIG block and re-run, the new email replaces the old one cleanly. If you change REAL_DIR, the script unsets the stale includeIf and registers the new one.

In practical terms: the script is the source of truth. Edit it, re-run, done. You don’t need to remember which git config commands you ran by hand and which you didn’t, because re-running the script reconciles whatever state you’re in with whatever’s currently in the CONFIG block.

What it touches and what it leaves alone

A script that writes to your home directory deserves scrutiny. The full list of paths it touches:

1. ~/.gitconfig is never edited directly. The script only invokes git config --global ..., which is Git’s own mechanism for editing this file safely. Those invocations set user.useConfigOnly, unset any global user.name/user.email, and register the includeIf entry; identity itself comes only from the per-directory include file. Any existing settings the script doesn’t touch (custom sections, other aliases you’ve added) are preserved.
2. ~/.ssh/id_ed25519 is generated if missing, never overwritten if present. The script checks for the file and only runs ssh-keygen when it’s absent. If you already have a key there from before, it’s left alone.
3. ~/.gitmessage is overwritten. If you had a custom commit message template there, it gets backed up to ~/.gitmessage.backup.<timestamp> before being rewritten.
4. ~/.git-hooks/prepare-commit-msg is overwritten. This is a managed file the script owns.
5. ~/.gitconfig-me is overwritten and chmod 600’d. This is a managed file the script owns; backed up before write.

Nothing outside these paths is changed. The script does not install packages, modify shell rc files, or touch anything system-wide.

Dotfiles repo: the convention experts use

The script alone gets you set up on one machine. The convention for multi-machine setup, used widely across the Linux and privacy-focused dev world, is a dotfiles repo: a private Git repository containing your git-setup_v12.sh, your ~/.gitmessage (and often ~/.bashrc, ~/.vimrc, anything else you customize). On a fresh machine you clone the dotfiles repo, run the script, and you’re done.

Two tools make the dotfiles repo pattern smoother but neither is required:

1. GNU Stow (sudo apt install stow) symlinks files from the dotfiles repo into ~ so the repo stays the source of truth. Lightweight, no templating, Devuan/Debian native. The most common pick.
2. chezmoi is the heavier alternative with templating (different machines can render the same dotfile differently) and built-in age encryption for secrets. Worth it once you’re managing several machines or want secrets in the repo without plaintext exposure.

If the dotfiles repo is private and on encrypted storage, plain files are fine. If it’s pushed to GitHub or similar, encrypt anything you’d rather not publish with git-crypt or chezmoi’s built-in age support.

Pseudonymous work

This script is single-identity by design. For a pseudonymous identity (separate from your real identity), use a separate OS user account, a VM (Whonix on Qubes is the canonical setup), or a separate physical machine. Run this same script there with that identity’s REAL_NAME and REAL_EMAIL.

The reason is structural. Config-level identity separation on one OS account leaves several footguns intact: a repo placed under the wrong directory, a remote pointed at the wrong account, the SSH agent offering the wrong key, a host compromise that reads both identities at once. The convergent pattern among Tor developers, Bitcoin Core’s pseudonymous contributors, and Qubes users is OS-level isolation: different home directory, different ~/.gitconfig, different ~/.ssh, different browser, different shell history. The “wrong terminal” class of failures disappears because the boundaries are real, not procedural.

See the concepts doc § “Identity layers and isolation” for the threat-model reasoning, and § “Beyond passphrase keys” for the optional next tiers (hardware tokens, Tor transport, independent commit signing).

Upgrading from v10 or v11 (had anon configured)

Earlier versions of this script supported a second “anon” identity in the same config. v12 dropped it. If you’ve previously run v10 or v11 with ANON_GH_USER set, clean up the residue once:

git config --global --unset includeIf.gitdir:~/code/anon/.path        # adjust path if ANON_DIR was different
rm -f ~/.gitconfig-anon
sed -i '/^# === managed by git-setup (multi-identity SSH) BEGIN ===$/,/^# === managed by git-setup (multi-identity SSH) END ===$/d' ~/.ssh/config
sed -i '/^# === managed by git_setup (multi-identity SSH) BEGIN ===$/,/^# === managed by git_setup (multi-identity SSH) END ===$/d' ~/.ssh/config

Strips the includeIf pointing at ~/.gitconfig-anon, deletes the file itself, and removes both the current and legacy SSH marker blocks. If you never ran with ANON_GH_USER set, none of these commands do anything; the snippet is safe to run regardless.

Starting a repo

Initialize a new repo in the current folder.

git init

Clone an existing remote repo.

git clone <url>
git clone <url> <folder-name>   # clone into a specific folder

The daily cycle

Check what has changed and what’s staged.

git status
git status -s   # short output, one line per file

Stage specific files for the next commit.

git add <file>
git add <file1> <file2>

Stage everything that has changed, including new files.

git add -A

Stage only changed tracked files (ignores new untracked files).

git add -u

Stage hunks interactively. Walks you through each change and asks y/n. Use s to split a hunk, e to hand-edit line-by-line.

git add -p <file>

Commit staged changes with an inline one-line message.

git commit -m "short summary of the change"

Commit and open the editor for a longer message (summary line, blank line, body).

git commit

Skip staging and commit all tracked changes in one step. Does not include new untracked files.

git commit -a -m "message"

Stage and commit in one interactive hunk-by-hunk pass.

git commit -p

Commit just one path (or a few) out of many changed, leaving every other change uncommitted. Useful for splitting one file’s distinct rationale into its own commit while the rest stays bundled. Takes the working-tree content of the named paths and does not touch the index for anything else, so you need not stage first.

git commit -m "Drop rebase glossary entry: duplicates concepts doc" -- glossary.md

Everyday workflow

Six commands cover most days, assuming the aliases from one-time setup are in place. The principles behind these (commits as communication, atomic commits, message discipline) are in the Git concepts doc.

Stage everything that changed and commit with a one-line message:

git ca "fix login redirect"

Commit already-staged changes with a one-line message:

git c "fix login redirect"

See what you just committed (metadata, message, and compact summary of changed files):

git last

Stage everything and fold it into the previous commit, keeping the previous commit’s message. Use this when you commit and immediately notice you missed something (a file, a typo fix, a forgotten change in the same logical unit):

git caa

Fold already-staged changes into the previous commit, keeping the previous commit’s message. Use this when you staged precisely what you want and don’t want add -A picking up unrelated working-tree changes:

git caas

Stage everything and fold it into the previous commit, but with a new message. Use this when you want to also change the previous commit’s title or message:

git caam "new commit title"

caa, caas, and caam all rewrite the previous commit (new hash). Safe locally; on already-pushed commits you’ll need git pf afterwards.

When the everyday six aren’t enough.

For a multi-paragraph commit from the command line, subject plus one or more body paragraphs, pass -m multiple times. Git concatenates the values as separate paragraphs separated by blank lines. Documented git behavior, no alias needed.

git commit -m "subject" -m "body paragraph"
git commit -m "subject" -m "first paragraph" -m "second paragraph"

The same form scopes a body note to one file in a commit that touches several: prefix the body line with the file’s path.

git commit -m "2026-06-03 — intro, glossary, bibliography" \
           -m "glossary.md: dropped the rebase entry; it duplicated the concepts doc and would drift."

git log -- glossary.md and git blame glossary.md surface that line against the file later.

Combined with staging:

git add -A && git commit -m "subject" -m "body paragraph"

To start a message with -m and finish in the editor (useful when you’ve typed the subject and want the editor for the body, with rulers and diff visible):

git commit -m "subject" -e

For a fully substantive message, anything where you want auto-wrap at 72, the diff visible while writing, and no shell quoting hassles, drop the -m entirely and let the editor open:

git add -A
git commit

The template pre-fills the editor with rule reminders, the codium rulers mark columns 50 and 72, and the staged diff is appended below a scissors line so you can read the changes while writing the message. Save and close the tab to commit.

To fix the previous commit’s message or fold forgotten changes into it:

git commit --amend          # opens editor with previous message preloaded
git commit --amend --no-edit # keeps message, folds in newly staged changes (long form of caas)

To fold staged changes into a commit older than HEAD, use the fixup + autosquash pair. Local only, rewrites every commit from the target forward.

git add <files>
git cf <hash>          # records "fixup! <subject>" against <hash>
git ria <hash>^        # interactive rebase, autosquash slots the fixup into place

For the common case of folding changes into the commit before HEAD (e.g., you committed something, then committed something else, then realized the earlier commit was incomplete):

git add <files>
git cf HEAD~1
git ria HEAD~3

Why HEAD~3 and not HEAD~2: the fixup commit itself moves HEAD, so the target that was HEAD~1 is HEAD~2 by rebase time, and the rebase base must be the target’s parent, one further back. The sha-based general form above is immune to this shift, because a hash doesn’t move when HEAD does. The rebase opens the editor with the fixup pre-positioned next to its target and pre-marked as a fixup. Save and close to apply. --amend itself only operates on HEAD; there’s no syntax to point it at an arbitrary commit.

The fixto alias from one-time setup wraps the whole sequence and resolves the target to a hash first, so the shift cannot bite. Its fixtos sibling does the same for staged-only changes:

git fixto HEAD~1       # stage everything, fold into the commit before last
git fixto <sha>        # same, into any earlier commit
git fixtos <sha>       # fold ONLY staged changes; leaves unstaged work untouched

fixto stages everything (add -A, the same caveat as ca and caa) and runs the autosquash rebase without opening the editor. fixtos skips the add -A, so only the staged index folds in, and adds --autostash to set the unstaged remainder aside during the rebase and restore it after. Both are root-aware: targeting the root commit rebases with --root rather than the nonexistent <root>^, which would otherwise abort and strand a stray fixup! commit. The fold conflicts when a later commit changed the lines around your change, since the patch’s context no longer matches at the target; on a conflict it stops like any rebase: resolve and git rc, or back out with git ra.

To replace the message of an older commit without changing its content, use rewordto. It has two forms selected by whether you pass a new message as a second positional argument.

git rewordto HEAD~1 "New subject for the prior commit"   # inline form, no editor
git rewordto <sha>  "New subject for an earlier commit"  # same, against any earlier commit
git rewordto HEAD~1                                       # editor form, opens with old message pre-filled
git rewordto <sha>                                        # same, against any earlier commit

The inline form constructs the amend! commit manually with two -m flags (target’s subject prefixed with amend! becomes the first; your new message becomes the body, which autosquash installs as the target’s replacement message), and is fastest for the common case of changing a single-line subject. The editor form goes through git commit --fixup=reword:<sha> (Git 2.32+, June 2021), opens the editor with the target’s old message pre-filled, and lets commit.verbose and commit.template apply just like any other substantive commit; preferred for multi-line message rewrites where the editor is a better composition surface than the command line. The two paths exist because -m and --fixup=reword: are mutually exclusive at the git level, so the inline form has to go through the manual amend! construction.

Either form runs the autosquash rebase non-interactively, leaving the target’s tree unchanged and its message reading as the one you wrote. Same root-awareness and same local-only constraint as fixto; the rewrite changes the target’s hash and every descendant. Don’t use this on a commit you’ve already pushed to a shared remote without a coordinated force-with-lease push afterwards.

To fold an existing commit into an earlier one, rather than folding the working tree or index, use foldinto. This reaches the case fixto and fixtos cannot: the change you want to absorb is already its own commit, not staged work.

git foldinto HEAD~1 HEAD~3      # fold the commit at HEAD~1 into the commit at HEAD~3
git foldinto <source> <target>  # by hash; source first (dissolved), target second (absorbs it)

The argument order is source then target: the first commit is dissolved, the second absorbs its diff and keeps its own message. It requires a clean working tree and refuses otherwise, so that its conflict recovery can hard-reset to the starting commit without endangering uncommitted work. Targeting is by the commit you name, not by subject text: a duplicate subject elsewhere does not misdirect the fold, because the rebase is based at the target’s parent, which makes the target the oldest commit in range (note 10). Unlike fixto and fixtos, which stop inside the rebase on a conflict for you to resolve and git rc, foldinto restores you to the exact starting state and reports it; redo the fold by hand with git rebase -i <target>^, reorder the source under the target, and mark it fixup. Same local-only constraint: it rewrites the target and every descendant, so don’t run it on commits already pushed to a shared remote without a coordinated force-with-lease push afterwards.

The aliases used above (c, ca, caa, caas, caam, caams, last, cf, ria, fixto, fixtos, rewordto, foldinto) are defined in one-time setup. The fallback editor flow uses commit.verbose, commit.template, and the codium settings, all also in one-time setup.

Note on shell quoting: every form that takes a message in quotes (the aliases and all -m variants) is subject to shell rules. Apostrophes, parens, $, !, and * need escaping or careful quoting. If a message would need heavy escaping, use editor mode instead.

Remotes

List remotes.

git remote -v

Add a remote.

git remote add origin <url>

Change a remote URL.

git remote set-url origin <new-url>

Remove a remote.

git remote remove origin

Fetch from remote without merging.

git fetch origin

Fetch and clean up references to branches deleted on the remote. Without --prune, deleted remote branches linger in git branch -a output indefinitely.

git fetch --prune
git fetch -p

Make pruning automatic for all fetches (one-time setup sets this):

git config --global fetch.prune true

Pull (fetch plus merge or rebase depending on config).

git pull

Push the current branch, setting upstream the first time. With push.autoSetupRemote from one-time setup, a plain git push does this automatically; -u is only needed where that setting is absent.

git push -u origin main
git push              # after upstream is set

Push a new branch.

git push -u origin <branch-name>

Delete a remote branch.

git push origin --delete <branch-name>

Force-push. Overwrites remote history. Use --force-with-lease --force-if-includes instead of plain --force: the lease refuses the push if the remote changed unexpectedly, and --force-if-includes (Git 2.30+) additionally verifies your local history contains everything on the remote ref. Aliased as git pf in one-time setup.

git push --force-with-lease --force-if-includes

Keep a linear history on pull instead of generating merge commits. Rebase your local commits on top of what you fetched, ad hoc or as the default.

git pull --rebase
git config --global pull.rebase true        # make rebase the default for every pull
git config --global pull.ff only            # otherwise, refuse pulls that aren't fast-forward

Setting pull.rebase true is the durable choice for solo work: it prevents the accidental merge bubbles a plain git pull creates when local and remote have both moved.

Set or retarget the upstream-tracking branch without pushing. Use when a branch was created or pushed without -u, or to point it at a different remote branch.

git branch -u origin/<branch>
git branch --set-upstream-to=origin/<branch>     # long form

Removing and renaming tracked files

Both operations stage their change immediately, so the next commit records them.

Remove a tracked file from the repo and the working tree.

git rm <file>
git rm -r <dir>                      # recursively, for a directory

Rename or move a tracked file. Git records a rename as a delete plus an add and re-infers it later by content similarity, so do the rename in its own commit and edit the file’s contents in a separate commit. A combined rename-and-edit commit drops the similarity below Git’s detection threshold and breaks git log --follow, git blame across the rename, and GitLens history.

git mv <old> <new>

To stop tracking a file while keeping it on disk (the inverse of git add), see git rm --cached under “Ignoring files”.

Ignoring files

Create a .gitignore at the repo root. Each line is a pattern.

# comments start with hash
*.log
node_modules/
.env
.DS_Store
build/
/dist          # leading slash: only at repo root
!important.log # negation: don't ignore this one

Start from a language-specific template at github.com/github/gitignore.

Global ignore for OS and editor clutter that should never be tracked.

git config --global core.excludesfile ~/.gitignore_global

Stop tracking a file that’s already tracked, without deleting it locally. Needed because .gitignore only affects untracked files.

git rm --cached <file>

Ignore a file in this repo only, without committing a change to the shared .gitignore. Patterns go in .git/info/exclude, which lives in the repo’s .git directory and is never committed. Use it for personal editor or scratch files that shouldn’t be imposed on collaborators.

# .git/info/exclude  --  same pattern syntax as .gitignore
.scratch/
notes-to-self.md

.gitattributes template

A .gitattributes file at the repo root tells Git how to handle specific file types: line endings, binary detection, diff strategy. Committed to the repo, so every clone gets the same rules. Overrides per-machine core.* settings, which is why it’s preferred for anything that should apply consistently across collaborators.

Combined template covering line endings, binary markers, and prose diff settings:

# Line endings: normalize to LF in repo, convert on checkout as needed
* text=auto eol=lf

# Files that must keep CRLF (Windows-specific scripts)
*.bat   text eol=crlf
*.cmd   text eol=crlf

# Binary files: don't normalize, don't try to diff
*.png   binary
*.jpg   binary
*.gif   binary
*.pdf   binary
*.zip   binary
*.mp4   binary
*.mov   binary
*.mp3   binary

# Prose files: word-level diff for cleaner diffs on Markdown/text
*.md    diff=word
*.txt   diff=word
*.tex   diff=word

Word-level diffs inline (one-off, without configuring as default):

git diff --word-diff
git diff --word-diff=color

If a repo already has mixed line endings, renormalize once after adding the template:

git add --renormalize .
git commit -m "Normalize line endings"

Per-machine line-ending fallback. Use only when .gitattributes isn’t an option (e.g., contributing to a repo whose maintainers haven’t adopted the file). Travels with the machine, not the repo, so it’s the inferior choice when you control the repo:

git config --global core.autocrlf input     # Linux/macOS
git config --global core.autocrlf true      # Windows

Inspecting state and history

Show the full log.

git log

Show a compact one-line-per-commit log.

git log --oneline

Visualize all branches as a graph.

git log --oneline --graph --all --decorate

Filter log by author.

git log --author="name"

Filter log by date range.

git log --since="2 weeks ago"
git log --since="2026-01-01" --until="2026-03-01"

Search commit history for a string that was added or removed (pickaxe).

git log -S "search string"
git log -G "regex pattern"   # regex variant

Show what changed in a specific commit.

git show <sha>

Show a file as it existed at a specific commit.

git show <sha>:path/to/file
git show HEAD~3:path/to/file

Diff: unstaged changes against last commit.

git diff

Diff: what’s staged for the next commit.

git diff --staged

Diff between two commits.

git diff <sha1> <sha2>
git diff HEAD~3 HEAD

Diff between two branches (any branch name works as a ref).

git diff main..feature-branch
git diff main feature-branch        # same thing, two-dot form omitted

Diff against the upstream-tracking branch. Useful for “what am I about to push” before git push.

git diff @{u}                       # current branch vs its upstream
git log @{u}..                      # local commits not yet on upstream
git log ..@{u}                      # upstream commits not yet local

Diff against an earlier state of HEAD (from the reflog). Useful for “what changed in my last reset/rebase/merge.”

git diff HEAD@{1}                   # current state vs HEAD one move ago
git diff HEAD@{1}..HEAD             # explicit form
git diff HEAD@{yesterday}           # time-based reflog reference

Diff at word level (better for prose than line level).

git diff --word-diff

Who last modified each line of a file, and in which commit.

git blame <file>
git blame -L 20,40 <file>   # limit to lines 20-40

Find which commit introduced a bug by binary search.

git bisect start
git bisect bad                    # current commit is broken
git bisect good <sha>             # known good commit
# git checks out a midpoint; you test it and run:
git bisect good   # or: git bisect bad
# repeat until git names the culprit, then:
git bisect reset

Automate the search by handing bisect a test command instead of judging each step by hand. Git runs the command at every midpoint and reads its exit code: 0 is good, 1 to 127 except 125 is bad, 125 means skip this commit as untestable. The command can be a test script, a one-liner, or a compiler invocation, and bisect names the first bad commit with no further interaction.

git bisect start HEAD <known-good-sha>
git bisect run ./test.sh
git bisect reset

Show every commit that touched a file, following it across renames. Without --follow the history stops at the rename; keeping renames content-pure is what makes --follow reliable (see the concepts doc). Aliased as git lf.

git log --follow -- <file>
git log --oneline --follow -- <file>

Show a file’s history with the actual diffs, or with per-commit change stats.

git log -p -- <file>        # full patch at each commit touching the file
git log --stat              # files changed and line counts per commit

Show the history of specific lines or a single function, with the diff at each step. The line-range form takes start and end; the function form names the symbol and lets Git find its bounds. This is the time axis to blame -L’s snapshot: blame says who last touched each line, -L says how the region got there.

git log -L 40,60:path/to/file        # history of lines 40-60
git log -L :funcName:path/to/file    # history of one function, bounds auto-detected

Search tracked content. Faster than a plain recursive grep, respects .gitignore, and can search any revision rather than just the working tree.

git grep "pattern"
git grep "pattern" <sha>            # search the tree at a past commit
git grep -n "pattern"               # show line numbers

Find which branches or tags contain a given commit. Answers “which release shipped this fix.”

git branch --contains <sha>
git tag --contains <sha>

Diff a branch against the point where it diverged, not against the other branch’s current tip. The three-dot form diffs from the merge-base, so it shows only what your branch added; the two-dot form above mixes in the other branch’s later commits.

git diff main...feature             # changes on feature since it forked from main

Compare two versions of a commit series, such as a branch before and after a rebase, or v1 against v2 of the same work. It pairs commits up by content and shows a diff of the diffs, so you can confirm a rewrite changed only what you meant it to. The reflog form feature@{1} is the quickest before-and-after right after a rebase.

git range-diff main..feature@{1} main..feature   # this branch, before vs after the last rebase
git range-diff <base> <v1-tip> <v2-tip>          # explicit three-arg form

Keep git blame readable across bulk-reformat commits. List the reformat commits’ shas in a .git-blame-ignore-revs file (one per line) and blame skips them, attributing each line to its last meaningful change instead.

git blame --ignore-rev <sha> <file>
git config blame.ignoreRevsFile .git-blame-ignore-revs    # apply the file by default

Undoing

Discard unstaged changes to a file. Destroys your edits.

git restore <file>
git restore .   # all files in current directory

Unstage a file (moves it from staged back to modified, does not change its contents).

git restore --staged <file>

Fix the most recent commit’s message. Local only.

git commit --amend -m "new message"

Add forgotten changes to the most recent commit. Local only.

git add <forgotten-file>
git commit --amend --no-edit

Undo the last amend, restoring the pre-amend commit. The amend replaced HEAD with a new commit and left the original in the reflog; this points HEAD back at it. The target is HEAD@{1} (the reflog entry of the original commit), not HEAD~1 (its parent): reset --soft HEAD~1 would discard the original commit’s own content along with the amend. --soft keeps everything the amend touched staged, so nothing is lost.

git reset --soft HEAD@{1}
git reset --hard HEAD@{1}    # discard the amend's changes too; destructive

HEAD@{1} is correct only if the amend was the most recent action that moved HEAD. If you’ve committed, checked out, or reset since, read git reflog and reset to the original commit’s explicit sha instead.

Re-stamp the author of the last commit. Use after a commit lands under the wrong identity (the bug the Identity setup section warns about). --reset-author re-reads the current user.name and user.email; the explicit form sets a specific author.

git commit --amend --reset-author --no-edit
git commit --amend --author="Name <email>" --no-edit

Rewrites the commit (new hash), so it’s local-only or force-push with git pf after.

Undo the last commit but keep the changes staged.

git reset --soft HEAD~1

Undo the last commit and unstage the changes (keep them in working directory).

git reset HEAD~1

Undo the last commit and discard the changes entirely. Destructive.

git reset --hard HEAD~1

Create a new commit that reverses a previous one. Safe on pushed history.

git revert <sha>

Revert a merge commit. Plain git revert fails on a merge because it can’t tell which parent to treat as mainline; -m 1 selects the first parent (the branch you merged into).

git revert -m 1 <merge-sha>

Reset vs revert: reset rewrites history and is local-only safe; revert adds a new commit that undoes the target and is safe to push.

Preview untracked files Git would delete (always run this before git clean -fd). The -n flag is dry-run; nothing is deleted.

git clean -nd
git clean -ndx     # also include ignored files

Actually delete untracked files. Destructive: files not tracked by Git are gone, including any new work you forgot to add. Run -nd first.

git clean -fd      # untracked files and directories
git clean -fdx     # also ignored files (.env, build artifacts)

Restore one file to its state at another commit or branch, leaving everything else untouched. Unlike git restore <file> (which discards to HEAD), --source pulls the file’s content from any ref.

git restore --source=<sha> <file>
git restore --source=main <file>     # the version on another branch

Undo a just-completed merge, rebase, or pull. Each of these sets ORIG_HEAD to the tip before the operation, so a hard reset to it backs the whole operation out.

git reset --hard ORIG_HEAD

For an operation still in progress (conflicts unresolved), use --abort instead; ORIG_HEAD is for one that already completed.

Quick recipes for common messes

Committed to the wrong branch.

git reset --soft HEAD~1         # undo the commit, keep the changes staged
git stash                        # save them
git switch <correct-branch>
git stash pop                    # changes come back unstaged
git add -A
git commit -m "message"

Accidentally committed a huge file.

git reset HEAD~1                # undo the commit
# remove or gitignore the huge file, then recommit

Works only while the commit is unpushed; once pushed, the blob is in history and needs the Purging section.

Accidentally committed a secret. Rotate the secret first (assume it’s compromised). Then purge from history with gitdel_v4.sh or git filter-repo. Force-push.

Rolled back too far with git reset --hard.

git reflog                       # find the sha from before the reset
git reset --hard <sha>

Want to throw away all local changes and match the remote exactly.

git fetch origin
git reset --hard origin/main

Want to see who changed a line and why, three years later.

git blame <file>                 # find the sha
git show <sha>                   # read the commit message

Branches

List local branches. Current branch is marked with *.

git branch

List all branches including remotes.

git branch -a

Create a branch (stays on current branch).

git branch <name>

Switch to an existing branch.

git switch <name>

Create and switch in one step.

git switch -c <name>

Look at an old commit without affecting any branch (detached HEAD). Useful for inspecting a historical state. Switch back to a real branch when done; commits made in detached HEAD without a branch become unreferenced.

git switch --detach <commit-sha>
git switch -                 # back to previous branch

Older equivalent still widely used.

git checkout <name>
git checkout -b <name>

Rename the current branch.

git branch -m <new-name>

Delete a branch that has been merged.

git branch -d <name>

Force-delete a branch that hasn’t been merged. Destroys unmerged work on that branch.

git branch -D <name>

Merge another branch into the current one.

git merge <other-branch>

Abort a merge in progress (after conflicts).

git merge --abort

Rebase current branch onto another. Local only if the current branch has been pushed.

git rebase <base-branch>
git rebase --abort        # back out mid-rebase
git rebase --continue     # after resolving conflicts

List branches by merge status, to see what’s safe to delete and what still has unmerged work.

git branch --merged main            # already merged into main; safe to delete
git branch --no-merged main         # still hold unmerged commits

Control how a merge is recorded. --no-ff forces a merge commit even when a fast-forward is possible, preserving the branch as a visible unit; --ff-only refuses to merge unless it can fast-forward, rejecting anything that would create a merge commit.

git merge --no-ff <branch>
git merge --ff-only <branch>

Drop the current commit during a rebase. Use only when the rebase stops on a commit whose changes are already present upstream and applying it is redundant. Anywhere else, --skip throws that commit’s changes out of the result; in a fixup/autosquash rebase that means the fix you queued silently vanishes. When in doubt, resolve and continue, or git rebase --abort and rethink.

git rebase --skip

Transplant a range of commits onto a new base, or drop a range. --onto takes the new base, the old base (an exclusive lower bound), and the branch to move.

git rebase --onto <newbase> <upstream> <branch>
git rebase --onto HEAD~3 HEAD~1     # replay HEAD onto HEAD~3, dropping HEAD~1 and HEAD~2

Read it as: replay the commits in <upstream>..<branch> on top of <newbase>.

Merge conflicts

When git merge, git pull, or git rebase stops with conflicts, git status lists the conflicted files. Each conflicted file contains markers:

<<<<<<< HEAD
your version
=======
their version
>>>>>>> other-branch

Edit the file to the final desired content, remove all three markers, then:

git add <file>
git commit              # if merging
git rebase --continue   # if rebasing

To abandon the attempt entirely:

git merge --abort
git rebase --abort

Take one whole side of a conflicted file instead of editing markers by hand, then stage it.

git checkout --ours <file>          # keep your side
git checkout --theirs <file>        # take the incoming side
git add <file>

The meaning inverts between merge and rebase. In a merge, --ours is your current branch and --theirs is the branch being merged in. In a rebase the sides are swapped: --ours is the branch you’re replaying onto and --theirs is your own commit being replayed. Check which operation you’re in before picking a side.

Stashing

Save uncommitted changes and revert the working directory to the last commit.

git stash push -m "what I was working on"

List stashes.

git stash list

Reapply the most recent stash and remove it from the stash list.

git stash pop

Reapply a stash but keep it in the list.

git stash apply stash@{0}

Drop a stash without applying.

git stash drop stash@{0}

Clear all stashes.

git stash clear

Promote a stash to a branch. Creates a new branch from the commit the stash was made on, then applies the stash on top. Useful when a stash turns out to be larger than expected or conflicts with the current branch.

git stash branch <new-branch-name>
git stash branch <new-branch-name> stash@{1}   # specific stash

Include untracked files in the stash. Plain git stash leaves new (untracked) files in the working directory, a common cause of “my stash didn’t save everything.”

git stash push -u -m "what I was working on"
git stash -u                         # short form, no message

Inspect a stash’s contents before applying it.

git stash show -p stash@{0}          # full diff of the stash
git stash show stash@{0}             # summary only

Rewriting history (local only)

Interactive rebase opens an editor with the last N commits listed, each prefixed with pick. Change the prefix to alter what happens to that commit:

1. reword — keep the commit, change the message.
2. squash (or s) — fold this commit into the one above, combining messages.
3. fixup (or f) — fold into the one above, discarding this commit’s own message.
4. edit — stop at this commit so you can amend it, then git rebase --continue.
5. drop (or d) — remove the commit entirely. Same effect as deleting its line.

Reorder commits by reordering lines in the editor.

git rebase -i HEAD~5

Skip the editor entirely for a straight squash-all-into-first via environment variable:

GIT_SEQUENCE_EDITOR="sed -i 's/^pick/squash/; 1s/squash/pick/'" git rebase -i HEAD~5

Non-interactive shortcut for the collapse-all-into-one case: reset --soft plus a fresh commit. Cleaner than interactive rebase when all you want is to combine the last N commits into one new commit with a new message.

git reset --soft HEAD~4
git commit -m "your message"

reset --soft moves the branch pointer back N commits but keeps all the changes staged. The single new commit captures everything. No editor. Limited to the straight-collapse case; use rebase -i for reorder, edit, drop, or selective squash. The aliased forms uncommit (for HEAD~1) and squash (general) are defined in the one-time setup.

Rebase preserving merge commits.

git rebase -i --rebase-merges HEAD~5

Apply a single commit from elsewhere onto the current branch.

git cherry-pick <sha>
git cherry-pick <sha1> <sha2> <sha3>

When a cherry-pick hits a conflict, resolve and continue, drop the conflicting commit, or back out entirely.

git cherry-pick --continue           # after staging the resolution
git cherry-pick --skip               # drop this commit, keep going
git cherry-pick --abort              # undo the whole cherry-pick

Recovery

The reflog is your safety net. It records every move of HEAD for ninety days by default, including operations that “lost” commits. Almost nothing is truly gone.

Show the reflog.

git reflog

Recover a commit by its sha from the reflog.

git checkout <sha>                               # look at it
git branch recovery <sha>                        # save it as a branch
git update-ref refs/heads/<branch> <sha>         # force a branch to point at it

Recover after a bad reset. Find the sha of the commit you were on before the reset in the reflog, then:

git reset --hard <sha>

Find dangling commits that the reflog doesn’t surface.

git fsck --lost-found

Tags

List tags.

git tag

Create a lightweight tag (just a pointer).

git tag v1.0.0

Create an annotated tag (recommended for releases; carries a message and metadata).

git tag -a v1.0.0 -m "First stable release"

Push a specific tag.

git push origin v1.0.0

Push commits plus any annotated tags reachable from them in a single operation. Useful when releases are tagged on commits you’re about to push; avoids the two-step push + push –tags.

git push --follow-tags

Push all tags.

git push --tags

Delete a local tag.

git tag -d v1.0.0

Delete a remote tag.

git push origin --delete v1.0.0

Get a human-readable name for the current commit relative to the most recent tag. Output like v1.2.0-5-gabc123 means five commits after v1.2.0, at commit abc123.

git describe --tags
git describe --tags --dirty          # append -dirty if the working tree has changes

SSH setup for GitHub

The automated path: git-setup_v12.sh generates the SSH key for you. If you’ve run that script, this section is reference for what it does manually; skip ahead to “Identity setup” for the directory-bound identity setup.

Generate a key.

ssh-keygen -t ed25519 -C "you@example.com"

Start the agent and add the key.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Copy the public key to paste into GitHub at Settings > SSH and GPG keys > New SSH key.

cat ~/.ssh/id_ed25519.pub

Verify the connection.

ssh -T git@github.com

Use SSH URLs for remotes (git@github.com:user/repo.git) rather than HTTPS URLs once SSH is set up.

Identity setup

Single-identity setup with directory binding and fail-loud safety. The global config carries no identity; the identity is bound to a directory via includeIf gitdir:. A repo created outside that directory errors instead of committing under whatever happened to be in global config.

For pseudonymous work see “Pseudonymous work” under Automated setup, above; this section is the mechanics for one identity on this OS account.

Fail-loud global

The global config carries no usable identity. user.useConfigOnly (Git 2.8+) then makes a repo outside the configured identity directory refuse to commit (“Author identity unknown”) instead of silently stamping whichever identity happened to be global.

git config --global user.useConfigOnly true
git config --global --unset user.name     # remove any existing global identity
git config --global --unset user.email

useConfigOnly only stops Git guessing or falling back to a default; on its own it does nothing if a global user.name/user.email is still set. The unset lines are what close the fallback.

Identity include file

The identity lives in its own include file that sets name, email, signing config, and the SSH key to use. The core.sshCommand line pins the key by directory, so the right key is used even when a remote URL was never rewritten to a Host alias.

~/.gitconfig-me:

[user]
    name = Your Name
    email = you@example.com
    signingkey = ~/.ssh/id_ed25519.pub
[gpg]
    format = ssh
[commit]
    gpgsign = true
[core]
    sshCommand = ssh -i ~/.ssh/id_ed25519 -o IdentitiesOnly=yes

Set the file to chmod 600 so other local users on the same machine can’t read it; the script does this for you.

Bind the identity to a directory

In ~/.gitconfig:

[includeIf "gitdir:~/code/me/"]
    path = ~/.gitconfig-me

A repo under ~/code/me/ resolves the identity, anything elsewhere has no identity and fails loud. Three caveats on the match:

1. The trailing slash matters. Without it the match is a prefix on the path string, so ~/code/me would also match ~/code/medical/ or similar.
2. gitdir: matches the canonicalized path of the repo’s .git. A repo reached through a symlinked parent can miss the match; point the condition at real paths.
3. Use gitdir/i: instead of gitdir: for a case-insensitive match on a case-insensitive filesystem.

Alternative: bind by remote URL

If your repos won’t all live under one identity directory, key the include off the remote URL instead (Git 2.36+, April 2022):

[includeIf "hasconfig:remote.*.url:git@github.com:youruser/**"]
    path = ~/.gitconfig-me

One trap: you cannot define a remote inside a file included this way. The condition needs the remote to exist before the file that would define it is read, so a [remote] block there is a circular dependency Git rejects; keep remotes in the repo’s own config. Directory binding sets identity at git init, before any remote exists; remote binding activates only once a remote is set, so a fresh repo has no identity until then. Prefer directory binding by default.

Check the active identity

Before committing in an unfamiliar repo, confirm who you are. Add an alias once:

git config --global alias.whoami '!git config user.name; git config user.email; git config user.signingkey'

Then, inside any repo:

git whoami                              # name, email, signing key for this repo
git config --show-origin user.email     # and which file set it

With the fail-loud global in place, a repo in an uncategorized directory errors here instead of resolving an identity. That error is the safety net, not a misconfiguration.

Signed commits

Sign commits so GitHub (and similar) show a “Verified” badge. Two methods: SSH (Git 2.34+, GitHub support since 2022) and GPG. SSH reuses your existing SSH key and is simpler to set up; GPG is the long-standing standard. Pick one; don’t combine.

SSH signing

git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/id_ed25519.pub
git config --global commit.gpgsign true
git config --global tag.gpgsign true

For local verification (git log --show-signature), Git needs an allowed_signers file listing trusted keys. Without it, signature verification produces an error rather than a result.

echo "$(git config --get user.email) namespaces=\"git\" $(cat ~/.ssh/id_ed25519.pub)" >> ~/.ssh/allowed_signers
git config --global gpg.ssh.allowedSignersFile ~/.ssh/allowed_signers

The namespaces="git" annotation is what tells ssh-keygen (and Git through it) that this key is authorized for Git-namespace signatures specifically, not other SSH-signed contexts.

Upload the same public key as a Signing Key in GitHub at Settings > SSH and GPG keys > New SSH key, with the key type set to Signing Key (a separate entry from the authentication key, even when the key material is identical).

The gpg.format ssh and commit.gpgsign true setting names are misleading. They control all signing, including SSH; the config keys retained gpg in their names from before SSH signing existed.

GPG signing

git config --global user.signingkey <key-id>
git config --global commit.gpgsign true
git commit -S -m "signed commit"

List your GPG keys.

gpg --list-secret-keys --keyid-format=long

Skip signing entirely for solo work unless you want the badge. Required by some security-sensitive projects.

Verify signatures rather than create them. The first shows signature status inline in the log; the others check a specific commit or tag and exit nonzero on failure.

git log --show-signature
git verify-commit <sha>
git verify-tag <tag>

SSH-signature verification needs the allowed_signers file from the SSH signing setup above; without it these report an error instead of a verdict.

Worktrees

Check out a second branch in a parallel working directory without cloning the repo again. Useful for quick context switches.

git worktree add ../other-branch-folder <branch>
git worktree list
git worktree remove ../other-branch-folder

Submodules and LFS

Both are advanced features that usually cause more friction than they solve for personal projects. See “Submodules and LFS” in the concepts doc for why. Minimal commands when you do need them.

Add a submodule (pin another repo at a specific commit inside this repo).

git submodule add <url> <path>
git submodule update --init --recursive

Clone a repo that has submodules (without this, the submodule directories will be empty).

git clone --recurse-submodules <url>

LFS install and track. Requires git-lfs installed (sudo apt install git-lfs).

git lfs install                        # one-time per user
git lfs track "*.psd"                  # tell LFS to handle this pattern
git add .gitattributes                 # the track command writes here

Files matching tracked patterns are stored in LFS instead of the regular git object store.

Purging files from history

Removing a file in a commit does not remove it from history; earlier commits still contain it. To actually purge a file from every commit in every branch, history has to be rewritten.

The tool depends on what’s being scrubbed:

1. Specific deleted files, with rename-chain awareness: gitdel_v4.sh, this project’s wrapper around git filter-repo.
2. Files matching a glob, or files over a size threshold: BFG Repo-Cleaner.
3. Secrets or other content inside files to be kept: BFG --replace-text or git filter-repo --replace-text.
4. Surgical edits to the last N commits (unpushed or coordinated): git rebase -i. See the “Rewriting history (local only)” section above for commands.
5. Never: git filter-branch. Deprecated by the git project itself.

After any rewrite: force-push, then assume platform-side caches (forks, PR pages, mirror clones, search indices) still hold the old content. Rewrite is the start of a leak response, not the end.

git filter-repo

The modern tool. The filter-branch man page now points users here. Python-based, supports path filtering, content replacement, ref renaming, and per-blob callbacks. Refuses to operate on non-fresh clones by default; --force bypasses, which is what gitdel_v4.sh uses internally because the wrapper performs its own working-tree cleanliness checks. The typical trigger on a follow-up invocation is a prior rewrite in the same repo: filter-repo’s check looks for a freshly packed repo, and once a previous rewrite has run, the repo no longer matches that shape, so the check fires even though the working tree and remotes are otherwise clean.

Install: sudo apt install git-filter-repo.

Purge a single file from all history.

git filter-repo --invert-paths --path secret.env

Purge multiple paths listed in a file, one per line.

git filter-repo --invert-paths --paths-from-file paths-to-purge.txt

Rename a path across all of history, so the file reads under the new name from the commit that first added it and no rename event appears. The argument is matched against the full repo-relative path from the root, at path-component boundaries rather than as a raw string prefix. A path matches only when it equals OLD exactly or continues with a / after it, so old/path:new/path rewrites old/path and everything under old/path/ but leaves old/path2 and old/path-archive untouched even though they share the leading text. It rewrites every commit from the add point forward, with new hashes, broken signatures, and a force-push, so the post-rewrite hygiene below applies.

git filter-repo --path-rename old/path:new/path

Operational points about this flag. A trailing slash makes the argument a directory rename bounded to that directory’s contents: old-dir/:new-dir/ moves everything under old-dir/ and does not match a file literally named old-dir. The slashes must agree on both sides; filter-repo errors on old/:new or old:new/, accepting only both-slash, neither-slash, or an empty side (old-dir/: lifts the directory’s contents to the repo root). Without slashes the argument names a single path component, file or directory: old.md:new.md renames the file old.md, or if old.md is a directory it renames the directory and everything under it, but never a sibling like old.md.bak whose name merely starts with the same text. A colon makes filter-repo error, it does not silently mis-split: the argument is split on every colon and must yield exactly two fields, so any path containing a colon aborts with “expects one colon”; rename such paths with --paths-from-file using an ==> line, which is split on ==> and tolerates colons in the path. The rename never selects paths: a bare --path-rename keeps every other path and only relabels matches, so files vanish only when a --path filter is also present, which is a whitelist that drops everything unmatched, or when a rename lands on a path that already exists in the tree, which is silently overwritten. A rename collision inside a single commit errors loudly with “File renaming caused colliding pathnames!” unless one side is a deletion or the two blobs are identical, so two distinct files folded onto one name fail fast rather than losing data quietly. filter-repo rewrites every ref by default (heads, tags, stashes), so a single --path-rename invocation propagates the rename across every branch and tag in the repo; this is what you usually want for a file-shape change but is the same blast-radius point that gitdel_v4.sh’s TRACKED_SET block already documents.

This rewrites history, clears remote-tracking refs by design, and breaks other clones of the repo. After running, re-add remotes and force-push, and tell any collaborators to re-clone from scratch.

If filter-repo refuses with “not a fresh clone”

Aborting: Refusing to destructively overwrite repo history since this does not look like a fresh clone. (expected freshly packed repo) ... Please operate on a fresh clone instead. If you want to proceed anyway, use --force.

This is filter-repo’s pre-rewrite safety check, refusing to start because the repo’s pack state doesn’t match a fresh clone. The check exists because the standard recovery from a bad rewrite is to discard the clone and re-clone, and that recovery only works if a forge copy holds the pre-rewrite state and you have no local-only commits the forge doesn’t.

Two paths.

Add --force to the same command and re-run when the repo mirrors to a forge you can re-clone from, has no local-only commits, and you accept that the throw-the-clone-away recovery posture is now provided by the forge mirror, not by the check. This is what gitdel_v4.sh does internally for exactly this reason.

git filter-repo --path-rename old/path:new/path --force

Re-clone fresh from the forge into a sibling directory, run the rewrite there, git push --mirror to every mirror, and discard the original working copy. This is the path the check is enforcing. Choose it when the repo is local-only, when you have any local commits the forge doesn’t (verify with git log @{u}..HEAD per branch), or when you’ve forgotten whether you’ve pushed.

git clone --mirror <forge-url> /tmp/repo.git
cd /tmp/repo.git
git filter-repo --path-rename old/path:new/path
git push --mirror <forge-url>

The --mirror clone preserves every ref, so the rewrite covers every branch and tag the forge holds, not just the current branch.

If filter-repo crashes after writing new history

New history written in N seconds; now repacking/cleaning... followed by a Python traceback such as FileNotFoundError: ...first-changed-commits means the rewrite finished but filter-repo died in its post-rewrite metadata step before refreshing the working tree. Refs already point at the rewritten history; the index and working tree still show the pre-rewrite snapshot, so a casual ls makes it look as if nothing changed.

This happens specifically when filter-repo finds .git/filter-repo/already_ran from an earlier rewrite older than a day, prompts Treat this run as a continuation of filtering in the previous run (Y/N)?, you answer Y, and the prior metadata is incomplete or written by a filter-repo version that didn’t yet emit first-changed-commits. The bare avoidance is to answer N to that prompt for any new rewrite (the docs explicitly tell you to answer N unless the prior rewrite is one you actually want chained to this one); the bare fix when you have already answered Y and hit the traceback is below.

First verify HEAD has the rewrite before doing anything destructive.

git ls-tree -r HEAD | grep <new-name>

If that prints a line, refs are correct. The fix is to repoint the working tree and finish the cleanup filter-repo aborted.

git reset --hard HEAD
rm -rf .git/filter-repo/
git reflog expire --expire=now --all
git gc --prune=now --aggressive

reset --hard HEAD replaces the stale index and working tree with what HEAD already points to. Removing .git/filter-repo/ clears state from the partial run; left in place and older than a day, it triggers the same continuation prompt on the next filter-repo invocation in this repo, and another Y answer reproduces the same crash. The reflog expire plus aggressive gc is the cleanup filter-repo would have done if it had reached the end; run it only after ls-tree HEAD verification, since it drops the unreachable-objects grace window that the Recovery section below relies on for restoring a botched rewrite.

If ls-tree HEAD does not show the new path, the rewrite did not match. --path-rename is literal and root-anchored, so a bare filename matches only a file at the repo root. Clean up and re-run with the full repo-relative path on both sides.

rm -rf .git/filter-repo/
git filter-repo --path-rename docs/old.md:docs/new.md --force

--force is needed on the re-run because the repo is no longer a fresh clone after the prior rewrite, but the prior rewrite was a no-op (it matched no paths), so the force-bypass is safe here.

The gitdel_v4.sh wrapper in this project pre-cleans .git/filter-repo/ before invoking filter-repo for exactly this reason; the crash mode above is only reachable when filter-repo is invoked directly without the wrapper.

gitdel_v4.sh

For a safer, validated purge that auto-detects deleted files, handles renames, protects files still live on any branch, and prints a pre-rewrite HEAD for recovery, use the gitdel_v4.sh script in this project.

bash gitdel_v4.sh <repo-path> [file-list.txt] [--dry-run] [--save-list <file>] [--yes]

Always run with --dry-run first.

bash gitdel_v4.sh ~/projects/myrepo --dry-run
bash gitdel_v4.sh ~/projects/myrepo --save-list ~/purge_log.txt

Without a file list, it scans git history for every deleted file and offers to purge them. With a file list, it purges only those paths (after validation).

BFG Repo-Cleaner

JVM tool, multi-threaded, can be significantly faster than filter-repo on large repos, especially for size-based stripping. Two distinctive features:

1. --strip-blobs-bigger-than 100M: remove every blob over a size threshold without enumerating files.
2. --replace-text rules.txt: rewrite blob contents to redact secrets in-place. The file stays in history; the secret does not.

Weaknesses: no rename awareness, HEAD-only working-tree protection by default, no concept of “files that were deleted but might be on another branch.” Best for size-driven or pattern-driven scrubs, not exact-set surgery.

git filter-branch

Deprecated. The git man page actively recommends against it: orders of magnitude slower than filter-repo, with subtle semantic bugs in path filtering. Mentioned only because old Stack Overflow answers still recommend it. Don’t.

If the only available tool is filter-branch (locked environment, no Python, no JVM), the right move is to install one of the alternatives. The cost of a one-time install is dwarfed by the cost of getting filter-branch wrong on a real repo.

git rebase -i for recent surgical edits

For dropping or amending a few specific recent commits. Works when:

1. The change is recent (within reach of an interactive rebase).
2. The commits are unpushed, or the force-push is coordinated with everyone holding the branch.
3. The scope is a few specific commits, not “every commit that touched file X.”

Beyond that, use filter-repo or BFG. See the “Rewriting history (local only)” section above for the commands.

Removing specific versions of a file from history

Use case: a tracked file has accumulated history, and you want to erase exactly two (or N) historical versions from the repo while keeping every other version of that file, every other file, and the surrounding commits intact. Common reason: a draft you don’t want preserved, accidentally-committed content that shouldn’t have been there, content you want to archive externally and then scrub from history.

Step 1, find the commits if you don’t already have the SHAs:

git log --all --follow -- path/to/file
git log --all -S 'distinctive substring from the version' -- path/to/file

--follow walks renames. -S (pickaxe) finds commits that added or removed a specific string.

Step 2, extract each version to a location outside the repo before doing anything destructive:

git show <SHA_A>:path/to/file > ~/safekeeping/file_va.txt
git show <SHA_B>:path/to/file > ~/safekeeping/file_vb.txt

git show <sha>:<path> prints the file’s exact contents as of that commit. Verify the extracts are what you wanted before continuing.

Step 3, decide which path to use.

Path A: drop the commits entirely. Use this when the target commits only changed that file, or when you accept losing everything else they did. Cleanest history afterward.

git rebase -i <parent-of-earliest-target>
# in the editor, change `pick` to `drop` on the two target lines

Path B: surgical erase of just the file content at those commits, keeping the commits themselves and any other changes they carried. Use when the target commits touched other files you want to keep.

Find the two blob hashes (the IDs Git uses for the file contents at each commit):

git ls-tree <SHA_A> path/to/file
git ls-tree <SHA_B> path/to/file
# output format: <mode> blob <hash> <path>

Rewrite to blank just those blobs:

git filter-repo --force --blob-callback '
if blob.original_id in (b"<blob_a_hash>", b"<blob_b_hash>"):
    blob.data = b""
'

The two commits remain in history with their other changes intact; the file at those commits is empty. Other versions of the file in other commits survive untouched because they point to different blobs.

Caveat on Path B: a downstream commit that “added paragraph X” relative to an erased blob will produce an unusual diff afterward, because its parent’s content at that path is now empty. If history readability at those points matters more than commit preservation, prefer Path A.

Post-rewrite (both paths):

git push --force-with-lease
git reflog expire --expire=now --all
git gc --prune=now --aggressive

If the repo is on a forge (GitHub, Codeberg), forks, PR pages, mirror clones, and search indices may retain the old blobs. If the hidden content is sensitive, assume it’s been seen and rotate whatever the content protected. See “Anti-patterns” below for the rest of the post-rewrite hygiene.

git replace

The non-rewriting alternative: keep the old commits, graft a replacement in front. Doesn’t propagate to clones unless refs/replace/* is explicitly pushed (rare). Useful for hiding history locally when force-push isn’t available. Not a real secret-scrub mechanism; the original blob still exists in the repo.

Decision tree

Start at the top. Stop at the first match.

1. Need to remove a specific known set of deleted files, with rename awareness? Use gitdel_v4.sh.
2. Need to strip blobs over a size threshold? Use BFG --strip-blobs-bigger-than.
3. Need to redact secrets inside files to keep? Use BFG or filter-repo --replace-text.
4. Need to drop or edit specific recent commits? Use git rebase -i.
5. Need to remove specific historical versions of a still-tracked file (not the whole file)? Use the “Removing specific versions of a file from history” recipe above.
6. Need to remove every file matching a glob, accepting path-literal matching with no rename chasing? Use filter-repo or BFG with the appropriate path filter directly.
7. Need to hide history locally without force-pushing? Use git replace.

Anti-patterns

1. Running the rewrite on the canonical working clone. Always use a throwaway. filter-repo’s fresh-clone check exists for a reason.
2. Force-pushing without coordinating with anyone else holding the branch. Their local history diverges silently; the next merge reintroduces what was scrubbed.
3. Assuming git gc reclaims space automatically. It doesn’t. Run git reflog expire --expire=now --all && git gc --prune=now --aggressive to actually free disk.
4. Treating “rewrite succeeded” as “secret is gone.” Platform caches, PR pages, mirror clones, search indices, IDE history, backup systems may all still hold it. Rotate the credential at its source regardless.
5. Rewriting signed commits without re-signing. Signatures break on commit-hash change; downstream verifiers see invalid signatures.
6. Running filter-branch in 2026. Stop.

Recovery

gitdel_v4.sh captures the pre-rewrite HEAD with git rev-parse HEAD and prints the SHA before filter-repo runs. Restore with git update-ref refs/heads/<branch> <sha> if the rewrite goes wrong.

For broader coverage across every branch, tag, and the top of the stash stack, capture all refs before the rewrite:

git for-each-ref --format='%(objectname) %(refname)' > /tmp/pre-rewrite-refs.txt

If the rewrite is wrong, restore each ref:

while IFS=' ' read -r sha ref; do
    git update-ref "$ref" "$sha"
done < /tmp/pre-rewrite-refs.txt

This works as long as the original objects are still reachable, i.e. until git gc --prune runs (default 90-day grace for unreachable objects). So: don’t run aggressive gc until the rewrite is verified.

Belt and braces: tarball the entire .git directory before the rewrite. Cheap, and recovers from any failure mode including gc-eaten objects.

After the rewrite

1. Force-push every remote: git push --force --all && git push --force --tags. filter-repo clears the remote config by design, so remotes need re-adding first; gitdel_v4.sh prints those commands.
2. Rotate the leaked credential at its source. Treat the rewrite as evidence of disclosure, not as a cure.
3. Notify collaborators. Anyone with a local clone needs to re-clone or carefully re-base their work; existing branches reference commit hashes that no longer exist.
4. Expect platform caches to lag. GitHub, GitLab, and Codeberg may keep old refs accessible via the API or PR pages for some window. Contact platform support for accelerated removal if the leak is severe.
5. Verify: clone fresh, git log --all --oneline -- <path> should show no trace of the purged paths.

to sort

A plain alias can’t interleave -m between your arguments — simple aliases just append args verbatim. You need a shell alias.

In your .gitconfig:

cm = "!f() { git commit -m \"$1\" -m \"$2\"; }; f"

Or set it via the command line (single quotes keep $1/$2 literal for the shell):

git config --global alias.cm '!f() { git commit -m "$1" -m "$2"; }; f'

Then git cm "subject" "body" expands to git commit -m "subject" -m "body".

The function wrapper is necessary because without it, git appends the positional args after the whole shell command string, which would tack "subject" "body" on as file arguments and break the commit.

A two-argument fixed function can’t handle that — you need a variadic loop. Replace the alias with this in .gitconfig:

cm = "!bash -c 'args=(); for m in \"$@\"; do args+=(-m \"$m\"); done; git commit \"${args[@]}\"' --"

Usage:

git cm "subject line" "first body para" "second body para"

Expands to git commit -m "subject line" -m "first body para" -m "second body para".

Two notes:

The explicit bash -c is necessary because on Devuan /bin/sh is dash, which has no arrays; the ! prefix would otherwise hand the shell command to dash and break.

The trailing -- sets $0 inside the bash -c script so that git’s appended arguments land in $1 onward and $@ stays clean — without it, the first argument would be consumed as the script name and dropped.

Edit .gitconfig directly rather than using git config --global alias.cm '...' for this one — the layered shell quoting on the command line is error-prone and the file edit is cleaner.

Add a shell alias to ~/.bashrc:

alias g=git

Then source ~/.bashrc or open a new terminal. g cm 'subject' 'body' will work because g expands to git before the shell invokes it, so git sees cm as a subcommand and looks it up in your .gitconfig aliases normally.

Nothing changes in the alias. Single and double quotes at the call site are interchangeable in bash for plain strings — the shell strips them both and passes the same bytes to git.

g cm 'subject' 'first body para' and g cm "subject" "first body para" are identical.

The only practical difference: single quotes suppress $VAR and backtick expansion, double quotes allow it. For commit messages that don’t contain shell variables, it doesn’t matter which you use.

restoring

You asked two questions about git file restoration.

The best way to restore a file to an older state is git restore --source=<commit> -- <file> followed by a normal commit. It’s non-destructive: all prior commits survive and the old state appears as a new tip. Interactive rebase is the alternative but rewrites history, making it appropriate only when you actually want the intervening commits gone, not just the file rolled back.

The second question was how the restore command differs from manually copying old content and pasting it into the file. The answer is: not at all, functionally — the resulting commit is identical. The command is faster, skips clipboard and editor surface area that can corrupt whitespace or encoding, and works on binary files where paste isn’t an option. But the manual approach isn’t wrong, just slower.

commits

git commit -F - <<‘EOF’ zola: remove publish, keep setup-only, remove nesting in home

The script now does one thing: scaffold a Zola blog and open a live preview. It no longer publishes posts or edits configuration from the command line.

What changed for you:

• Personalize every new blog by editing the settings block at the top of the script before you run it, so a fresh blog is born with your details already filled in.
• The home page lists your posts as a flat list with a link into each section; sections can nest as deep as you like and you click through to browse them.
• “previous” and “next” at the foot of a post now move within that post’s own section.
• New blogs ship with demo posts showing internal and external links, a hidden post, and a small section with breadcrumbs, all safe to delete.
• Breadcrumbs and the table of contents are off by default; turn them on in config.toml.
• Setup installs the current Zola and pins that exact version into the GitHub or Codeberg deploy workflow; run “zola-blog-setup update-zola” to upgrade later. EOF

Git history delete

To permanently purge files from a git repo, review and run:

#!/usr/bin/env bash
set -euo pipefail

# Git delete

## ─── Usage ────────────────────────────────────────────────────────────────────
## To permanently purge files from a git repo in say ~/projects/myrepo, run in terminal:
##   bash gitdel_v4.sh <repo-path> [file-list.txt] [--dry-run] [--save-list <file>] [--yes]
##   example: bash gitdel_v4.sh ~/projects/myrepo --save-list ~/Downloads/purge_log.txt
##
## Without a file list: auto-detects all deleted files in history (all refs).
## With a file list:    purges exactly those files (same validation rules apply).
## Renames: if a deleted file was renamed, all historical names back to its
##          original are purged. If its final name is still tracked, the
##          entire chain is skipped.
## --dry-run:           shows what would be purged, touches nothing.
## --save-list <file>:  saves the final purge list to a file.
## --yes / -y:          skip the interactive confirm prompt. Use only in
##                      non-interactive contexts where you have already
##                      reviewed the dry-run output.

## ─── Flow ─────────────────────────────────────────────────────────────────────
## 1. Parse arguments into REPO_PATH, INPUT, DRY_RUN, SAVE_LIST, ASSUME_YES.
## 2. Validate dependencies, repo state, and working tree cleanliness.
## 3. Capture remote URLs before filter-repo wipes them.
## 4. Detect submodule paths to skip them during purge.
## 5. Build a rename map: old_path -> new_path across all branches.
## 6. Build a lookup of all currently tracked files across every ref.
## 7. Collect files to purge: from input list or by scanning deleted files in history.
## 8. Validate each candidate: skip tracked, on-disk, submodule, or live-renamed files.
## 9. For renamed files whose final destination is also gone, resolve and purge the full chain.
## 10. Report skipped files, show purge list, optionally save it.
## 11. In dry-run mode, exit here.
## 12. Confirm with user (unless --yes), record pre-rewrite ref tips, run filter-repo, print remote re-add and force-push instructions.

## ─── Scope ────────────────────────────────────────────────────────────────────
## Deletions and renames are scanned across all refs (--all). TRACKED_SET is
## built from the union of every ref's tree, so a file live on any branch,
## tag, or stash is protected from purge.
##
## Merge-only renames (renames recorded only in merge commits) are invisible
## to the rename scan because --no-merges skips them. If you know a file was
## renamed during conflict resolution in a merge commit, pass those paths
## explicitly via --paths-from-file input.
##
## Rename-similarity threshold is -M10%, which catches low-similarity renames
## at the cost of occasional false positives on repos with many near-identical
## files (generated code, lockfiles). A false positive here results in
## under-purge, which is the safer direction for this tool.
##
## Path scans run under core.quotePath=false so that non-ASCII paths are
## emitted verbatim rather than C-quoted; without this a tracked non-ASCII
## file could be misread as untracked and purged, and the purge list handed
## to filter-repo could match nothing. One residual limitation remains: a
## path containing a literal tab or newline byte cannot be round-tripped
## through the newline-delimited scan and is not handled; such paths are
## vanishingly rare and must be purged manually.

## ─── Argument parsing ─────────────────────────────────────────────────────────

REPO_PATH=""
INPUT=""
DRY_RUN=false
SAVE_LIST=""
ASSUME_YES=false

while [[ $# -gt 0 ]]; do
    case "$1" in
        --dry-run)
            DRY_RUN=true
            shift
            ;;
        --save-list)
            [[ $# -lt 2 ]] && { echo "Error: --save-list requires an argument."; exit 1; }
            SAVE_LIST="$2"
            shift 2
            ;;
        --yes|-y)
            ASSUME_YES=true
            shift
            ;;
        -*)
            echo "Unknown option: $1"
            echo "Usage: bash gitdel_v4.sh <repo-path> [file-list.txt] [--dry-run] [--save-list <file>] [--yes]"
            exit 1
            ;;
        *)
            if [[ -z "$REPO_PATH" ]]; then
                REPO_PATH="$1"
            elif [[ -z "$INPUT" ]]; then
                INPUT="$1"
            else
                echo "Unexpected argument: $1"
                exit 1
            fi
            shift
            ;;
    esac
done

if [[ -z "$REPO_PATH" ]]; then
    echo "Usage: bash gitdel_v4.sh <repo-path> [file-list.txt] [--dry-run] [--save-list <file>] [--yes]"
    exit 1
fi

REPO_PATH="$(realpath "$REPO_PATH")"
[[ -n "$INPUT" ]] && INPUT="$(realpath "$INPUT")"

if [[ -n "$SAVE_LIST" ]]; then
    SAVE_DIR="$(dirname "$SAVE_LIST")"
    [[ ! -d "$SAVE_DIR" ]] && { echo "Error: --save-list parent directory does not exist: $SAVE_DIR"; exit 1; }
    SAVE_LIST="$(realpath "$SAVE_DIR")/$(basename "$SAVE_LIST")"
fi

## ─── Dependency check ─────────────────────────────────────────────────────────

if ! command -v git-filter-repo &>/dev/null; then
    echo "git-filter-repo not found. Install it with:"
    echo "  sudo apt install git-filter-repo"
    exit 1
fi

## ─── Repo validation ──────────────────────────────────────────────────────────

if [[ ! -d "$REPO_PATH" ]]; then
    echo "Repo path not found or not a directory: $REPO_PATH"
    exit 1
fi

cd "$REPO_PATH"

if ! git rev-parse --git-dir &>/dev/null; then
    echo "Not inside a git repository: $REPO_PATH"
    exit 1
fi

GIT_ROOT="$(git rev-parse --show-toplevel)"
if [[ "$(pwd -P)" != "$GIT_ROOT" ]]; then
    echo "Switching to repo root: $GIT_ROOT"
    cd "$GIT_ROOT"
fi

git diff --quiet          || { echo "Unstaged changes detected. Commit or stash them first."; exit 1; }
git diff --cached --quiet || { echo "Staged changes detected. Commit or stash them first."; exit 1; }

echo "Repo: $(pwd -P)"
[[ "$DRY_RUN" == true ]] && echo "(dry-run mode — no changes will be made)"
echo ""

## ─── Capture remote URLs before filter-repo removes them ─────────────────────

declare -A REMOTE_URLS=()
while IFS= read -r remote; do
    url="$(git remote get-url "$remote" 2>/dev/null || true)"
    if [[ -z "$url" ]]; then
        echo "WARNING: remote '$remote' has no URL configured; omitting it from re-add instructions."
        continue
    fi
    REMOTE_URLS["$remote"]="$url"
done < <(git remote)

if [[ ${#REMOTE_URLS[@]} -eq 0 ]]; then
    echo "WARNING: No remote detected. If something goes wrong, history cannot be recovered."
    echo "Consider pushing to a backup remote before proceeding."
    echo ""
fi

## ─── Submodule detection ──────────────────────────────────────────────────────

declare -A SUBMODULE_PATHS=()
if [[ -f ".gitmodules" ]]; then
    while IFS= read -r line; do
        if [[ "$line" =~ ^[[:space:]]*path[[:space:]]*=[[:space:]]*(.+)$ ]]; then
            SUBMODULE_PATHS["${BASH_REMATCH[1]}"]=1
        fi
    done < .gitmodules
fi

## ─── Build rename map (across all branches) ───────────────────────────────────
## RENAMED_FROM: old_path -> immediate new_path (one hop only).
## -F'\t' is required — paths can contain spaces; default awk splitting breaks them.
## -M10% lowers the similarity threshold from the default 50% so that low-similarity
## renames are still detected. This may produce occasional false positives on repos
## with many near-identical files, but missing a rename is worse than a false positive
## here because an undetected rename leaves stale history under the old name.

echo "Scanning rename history..."
declare -A RENAMED_FROM=()

while IFS=$'\t' read -r old new; do
    [[ -z "$old" || -z "$new" ]] && continue
    RENAMED_FROM["$old"]="$new"
done < <(git -c core.quotePath=false log --all --no-merges --pretty=format: --diff-filter=R -M10% --name-status \
         | awk -F'\t' '/^R[0-9]*\t/{print $2"\t"$3}')

## resolve_rename_chain <start>
##
## Walks RENAMED_FROM transitively and writes every name in the chain
## (including <start>) to stdout, one per line, in order: start … final.
##
## Requires bash 4.3+. The binding constraint is the negative array index
## ${chain[-1]} used below, which bash introduced in 4.3; the 'declare -A'
## scoping and '-v' key tests this function also relies on are older (4.0
## and 4.2 respectively). Debian and Devuan stable ship bash 5.x, so this
## is not a concern in practice, but do not run on bash < 4.3.
##
## Cycle detection: keeps a visited associative array; if a name reappears
## the chain is corrupt — emit an error to stderr and return non-zero so
## the caller can decide what to do with the partial chain.

resolve_rename_chain() {
    local current="$1"
    declare -A visited=()

    while true; do
        if [[ -v visited["$current"] ]]; then
            echo "ERROR: rename cycle detected involving: $current" >&2
            return 1
        fi
        visited["$current"]=1
        echo "$current"
        [[ -v RENAMED_FROM["$current"] ]] || break
        current="${RENAMED_FROM[$current]}"
    done
}

## ─── Build tracked-files lookup (across all refs) ─────────────────────────────
## git ls-files only lists HEAD, which is narrower than what filter-repo
## rewrites (all refs). Build TRACKED_SET from every ref's tree so a file
## live on any branch, tag, or stash is protected from purge.

echo "Building tracked-files lookup across all refs..."
declare -A TRACKED_SET=()
while IFS= read -r f; do
    [[ -n "$f" ]] && TRACKED_SET["$f"]=1
done < <(
    git for-each-ref --format='%(refname)' | while IFS= read -r ref; do
        git -c core.quotePath=false ls-tree -r --name-only "$ref" 2>/dev/null || true
    done | sort -u
)

## ─── Collect files to purge ───────────────────────────────────────────────────

declare -A PURGE_SET=()   # used for deduplication
PURGE=()

## Single associative array: skipped_renamed["old_path"]="final_path"
declare -A SKIPPED_TRACKED_SET=()
SKIPPED_TRACKED=()
declare -A SKIPPED_EXISTS_SET=()
SKIPPED_EXISTS=()
declare -A SKIPPED_SUBMODULE_SET=()
SKIPPED_SUBMODULE=()
declare -A SKIPPED_RENAMED=()   # old_path -> live final destination

add_to_purge() {
    local name="$1"
    if [[ ! -v PURGE_SET["$name"] ]]; then
        PURGE_SET["$name"]=1
        PURGE+=("$name")
    fi
}

validate_and_collect() {
    local f="$1"

    if [[ "$f" == /* ]]; then
        echo "ERROR: absolute path not supported: $f"
        exit 1
    fi

    if [[ -v TRACKED_SET["$f"] ]]; then
        if [[ ! -v SKIPPED_TRACKED_SET["$f"] ]]; then
            SKIPPED_TRACKED_SET["$f"]=1
            SKIPPED_TRACKED+=("$f")
        fi
        return
    fi

    if [[ -e "$f" ]]; then
        if [[ ! -v SKIPPED_EXISTS_SET["$f"] ]]; then
            SKIPPED_EXISTS_SET["$f"]=1
            SKIPPED_EXISTS+=("$f")
        fi
        return
    fi

    if [[ -v SUBMODULE_PATHS["$f"] ]]; then
        if [[ ! -v SKIPPED_SUBMODULE_SET["$f"] ]]; then
            SKIPPED_SUBMODULE_SET["$f"]=1
            SKIPPED_SUBMODULE+=("$f")
        fi
        return
    fi

    if [[ -v RENAMED_FROM["$f"] ]]; then
        local chain=()
        local chain_ok=true
        while IFS= read -r name; do
            chain+=("$name")
        done < <(resolve_rename_chain "$f") || chain_ok=false

        if [[ "$chain_ok" == false ]]; then
            echo "ERROR: skipping '$f' due to rename cycle in history. Inspect manually." >&2
            return
        fi

        local final="${chain[-1]}"

        if [[ -v TRACKED_SET["$final"] ]]; then
            SKIPPED_RENAMED["$f"]="$final"
        else
            for name in "${chain[@]}"; do
                if [[ -v TRACKED_SET["$name"] ]]; then
                    echo "WARNING: intermediate rename name '$name' is currently tracked; skipping that name only." >&2
                else
                    add_to_purge "$name"
                fi
            done
        fi
        return
    fi

    add_to_purge "$f"
}

if [[ -n "$INPUT" ]]; then
    echo "Reading file list from: $INPUT"
    while IFS= read -r line; do
        [[ -z "$line" || "$line" == \#* ]] && continue
        f="${line#./}"
        f="${f%$'\r'}"
        validate_and_collect "$f"
    done < "$INPUT"
else
    echo "Scanning git history for deleted files (all branches)..."
    while IFS= read -r line; do
        [[ -n "$line" ]] && validate_and_collect "$line"
    done < <(git -c core.quotePath=false log --all --no-merges --pretty=format: --name-only --diff-filter=D | sort -u)
fi

## ─── Report skipped files ─────────────────────────────────────────────────────

if (( ${#SKIPPED_TRACKED[@]} > 0 )); then
    echo "WARNING: ${#SKIPPED_TRACKED[@]} file(s) skipped — currently tracked on some ref:"
    printf '  %s\n' "${SKIPPED_TRACKED[@]}"
fi

if (( ${#SKIPPED_EXISTS[@]} > 0 )); then
    echo "WARNING: ${#SKIPPED_EXISTS[@]} file(s) skipped — exist on disk but not tracked:"
    printf '  %s\n' "${SKIPPED_EXISTS[@]}"
fi

if (( ${#SKIPPED_SUBMODULE[@]} > 0 )); then
    echo "WARNING: ${#SKIPPED_SUBMODULE[@]} submodule path(s) skipped — handle these manually:"
    printf '  %s\n' "${SKIPPED_SUBMODULE[@]}"
fi

if (( ${#SKIPPED_RENAMED[@]} > 0 )); then
    echo "INFO: ${#SKIPPED_RENAMED[@]} file(s) skipped — renamed in history, live destination is tracked:"
    for old in "${!SKIPPED_RENAMED[@]}"; do
        echo "  $old  ->  ${SKIPPED_RENAMED[$old]}"
    done
fi

if (( ${#PURGE[@]} == 0 )); then
    echo "No files to purge after validation."
    exit 0
fi

## ─── Show purge list ──────────────────────────────────────────────────────────

echo ""
echo "Files to purge from history (${#PURGE[@]} total):"
printf '  %s\n' "${PURGE[@]}"
echo ""

## ─── Optionally save the list ─────────────────────────────────────────────────

if [[ -n "$SAVE_LIST" ]]; then
    printf '%s\n' "${PURGE[@]}" > "$SAVE_LIST"
    echo "File list saved to: $SAVE_LIST"
fi

## ─── Dry-run exit ─────────────────────────────────────────────────────────────

if [[ "$DRY_RUN" == true ]]; then
    echo "Dry-run complete. No changes made."
    exit 0
fi

## ─── Confirm and execute ──────────────────────────────────────────────────────

echo ""
echo "WARNING: git filter-repo --force bypasses the fresh-clone check."
echo "  - Stashes will be discarded."
echo "  - Other worktrees pointing at this repo will break."
echo "  - Remote-tracking refs will be cleared (by design)."
echo "  - Reflog entries may be expired by subsequent gc."
echo "If this repo is not a throwaway clone dedicated to this purge, stop now."
echo ""

if [[ "$ASSUME_YES" == true ]]; then
    echo "Proceeding without prompt (--yes specified)."
else
    printf "Proceed? This rewrites history and cannot be undone. (y/n): " >/dev/tty
    read -r CONFIRM </dev/tty
    [[ "$CONFIRM" == "y" ]] || { echo "Aborted."; exit 0; }
fi

TMPFILE="$(mktemp)"
trap 'rm -f "$TMPFILE"' EXIT
printf '%s\n' "${PURGE[@]}" > "$TMPFILE"

## Recovery anchors: filter-repo rewrites every ref, so record the tip of
## each ref (not just HEAD) before force-pushing. If the rewrite is wrong,
## `git update-ref <refname> <sha>` restores any ref to its pre-rewrite
## tip without depending on reflog retention.
echo "Pre-rewrite ref tips (record these; recover any ref with: git update-ref <refname> <sha>):"
git for-each-ref --format='  %(refname) %(objectname)' refs/heads refs/tags
echo ""

## Pre-clean filter-repo metadata directory. A prior gitdel run (or any
## prior filter-repo run) leaves state under .git/filter-repo/. If that
## directory persists for more than a day and we hit filter-repo's
## continuation prompt, answering Y to it makes filter-repo try to chain
## off the prior metadata, which can crash in `_record_metadata` with
## `FileNotFoundError: ...first-changed-commits` after the rewrite has
## already been written to refs. Removing the directory here ensures no
## continuation prompt fires and no chaining attempts occur.
rm -rf "$GIT_ROOT/.git/filter-repo"

git filter-repo --invert-paths --force --paths-from-file "$TMPFILE"

echo ""
echo "Done. Git history has been rewritten."
echo "Verify a path was purged with: git log --all --oneline -- <path>  (should print nothing)."
echo "Note: remotes and remote-tracking refs are cleared by filter-repo by design."
echo ""

if [[ ${#REMOTE_URLS[@]} -gt 0 ]]; then
    echo "Re-add your remote(s) and force-push:"
    ## Sort remotes for deterministic output
    for remote in $(printf '%s\n' "${!REMOTE_URLS[@]}" | sort); do
        echo "  git remote add $remote ${REMOTE_URLS[$remote]}"
        echo "  git push --force --all $remote"
        echo "  git push --force --tags $remote"
    done
    echo ""
fi

echo "WARNING: Any automated sync scripts, CI pipelines, or backup jobs pointed"
echo "at this remote will fail until you have manually force-pushed. They will"
echo "not auto-recover — trigger them manually after the force-push completes."

Security Overview

Secure your system. The long answer for why you must secure it is explained here. The short answer is that it’s the prudent thing to do. Before the steps, the security landscape maps what you are defending and how far the climb goes. It’s a long journey, but the guides below will help you get through the steps in order. They place concepts before procedures and climb slowly, each reducing more attack surface. The early ones (OS, encryption) are the foundation the rest assume. You don’t have to reach the summit on day one, but you can start climbing today.

1. Replace the OS. Compare distros, leave Windows or Mac, pick one, migrate. The largest single cut to your attack surface, and the foundation every guide above it assumes.
2. Encrypt. Disk and file encryption. Read the GPG concepts guide alongside it for the key, signing, and identity model behind those choices; that one is concept, not procedure.
3. Separate identities, manage keys. Identity separation across users and VMs, SSH and GPG key strategy, hardware tokens, behavioral discipline.
4. Secure messaging. Signal at the base, up through federated, Nostr-rooted, P2P, off-grid, and email.
5. Sovereign transport. VPN, mesh, overlay, Tor, censorship-evasion, off-grid radio.
6. Detect compromise. Host integrity monitoring; the shift from prevention to detection.
7. Vet documents. Scanning and metadata hygiene for files you receive, before you open them.
8. Dedicated hardware. A desktop built from parts, Linux or coreboot laptops, hardware tokens; raises the floor the apex guides stand on.
9. Harden the whole workstation. The desktop apex: LUKS, VM compartmentalization, USBGuard, nftables, kernel hardening, encrypted DNS. Driven by the install script.
10. Harden mobile. The phone apex, a parallel track you can climb any time after step 1: Pixel plus GrapheneOS, flashed per the GrapheneOS install guide.

Cross-cutting concerns

Things that don’t belong cleanly to any one layer but matter to the overall stack.

Hardware tokens

A small USB device that holds cryptographic keys and performs operations with them without ever releasing the keys to the host. The key material lives inside tamper-resistant hardware; an attacker who compromises the laptop cannot extract the keys without physically having the token.

Uses:

• SSH authentication. SSH keys live in the token. ssh calls into the token via FIDO or PKCS#11 to sign authentication challenges. The host never has the private key. Lost or stolen laptop is no longer “and now they have my SSH keys”, they have a laptop with no SSH access until they steal the token too.
• GPG / OpenPGP. Same idea for GPG keys (encryption, signing, certification). YubiKey and Nitrokey both support OpenPGP keys natively; the gpg-agent integration is straightforward on Devuan.
• Sudo authentication / PAM. pam_u2f lets you require a token tap for sudo, sudo-rights operations, or login. Tightens the local-attacker threat model substantially.
• 2FA for online accounts. WebAuthn / FIDO2 for every service that supports it. Strictly better than TOTP because the token signs the origin domain, so a phishing site can’t replay the second factor.
• LUKS unlock. Niche but real: a token can hold a keyfile that unlocks LUKS, used in combination with a passphrase.

The major options:

• YubiKey. The dominant brand. YubiKey 5 series supports FIDO2, OpenPGP, PIV, OATH, OTP, basically everything. Closed-source firmware that is not updatable: if a vulnerability is found in the firmware (as in the September 2024 EUCLEAK side-channel disclosure against YubiKey 5), you replace the device rather than apply a patch. Available everywhere; works on every platform. FIDO Level 2 Certification (higher than the open-source competitors’ current certifications).
• Nitrokey. German company. The Nitrokey 3 series runs fully open-source firmware called Trussed (Rust-based, jointly developed with SoloKeys). Older Nitrokey models (Pro, Start) are partially open, the underlying smartcard chips have proprietary firmware. For users who care about firmware auditability, the Nitrokey 3 is the meaningful pick; for users who care less about auditability and more about feature breadth, YubiKey wins.
• Solo / SoloKey. Open-source hardware and firmware. First open-source FIDO2 security key, launched 2018 via Kickstarter. Solo 2 series shares the Trussed firmware framework with Nitrokey 3. FIDO2-focused, no OpenPGP or OATH; simpler scope.
• OnlyKey. Open-source firmware. Acts as a USB keyboard and types stored passwords on a button press. Six physical buttons on the device plus a PIN entered on the device itself. Stores up to 24 accounts on-device. Different model from the others (password manager on hardware) rather than just a FIDO key.

Recommendation: a Nitrokey 3 (open-source firmware) for users who care about that property; a YubiKey 5 for users who want the broadest software support. Always buy two, one daily, one in a safe deposit box, both registered to the same accounts. A single hardware token that gets lost or destroyed locks you out of everything it protected. Don’t repeat that mistake; everyone tells the same story about it.

Integration on Devuan is straightforward:

sudo apt install libpam-u2f yubikey-manager

Configuration depends on what you’re protecting; the Arch Wiki entries for pam_u2f, gpg-agent, and OpenSSH FIDO are the practical references.

Heads coreboot

Open-source firmware that replaces the proprietary UEFI/BIOS on supported hardware. Built on coreboot; uses a YubiKey or similar token to verify that the boot path hasn’t been tampered with.

What it defends against:

The evil-maid attack. An attacker with physical access to a powered-off laptop modifies the firmware or bootloader. On a normal laptop, the user has no way to detect this; they boot, type the LUKS passphrase, and the modified firmware captures it. On a Heads-equipped laptop, the firmware itself is measured at boot, the measurement is checked against a signature on the token, mismatch triggers a visible alert before the user types anything sensitive.

The cost:

Hardware-specific. Heads runs on a specific list of supported machines, mostly older ThinkPads (X230, T440, T530, X1 Carbon up to specific generations) and some Purism Librem and Insurgo PrivacyBeast models. Flashing Heads is real work; failures can brick the laptop. There are vendors (Insurgo, Mullvad’s PrivacyBeast partnership) that sell Heads-preinstalled hardware at a premium, which is the cleaner path for most users.

Heads also has trade-offs in daily use: firmware updates require re-signing the boot measurement; kernel updates require re-signing too. Recovery from a forgotten token PIN or a corrupted token is non-trivial. For users with the threat model that justifies it, the trade-off is correct; for users without that threat model, Heads is overkill and the recovery friction is worse than the protection is worth.

Where to read more: osresearch.net for the project itself, plus Insurgo and Mullvad for purchase options.

Tor versus VPN versus self-hosted mesh

A frequent question. The short answer: three different things often grouped under “VPN,” each with a different threat model.

1. Commercial VPN. Single-hop tunnel to one company that knows who you are. Replaces your ISP with one trusted company. The “VPN for privacy” frame; structurally weaker than Tor and not an anonymity tool.
2. Self-hosted or mesh VPN. WireGuard between machines you control; Tailscale-style mesh with the coordinator either centralized or self-hosted (Headscale, NetBird). Not an anonymity tool; the right answer for connecting your own devices over hostile networks.
3. Tor. Multi-hop volunteer-run onion network. The actual answer for anonymity.

For the commercial-VPN versus Tor question:

A commercial VPN encrypts your traffic between your device and the VPN provider. From the VPN’s exit, your traffic continues to its destination over normal internet, unencrypted at that hop unless the destination uses HTTPS (which it usually does in 2026). What a commercial VPN gets you:

• Your ISP sees encrypted traffic to the VPN, not your destinations.
• The VPN sees your traffic and your destinations.
• The destinations see the VPN’s IP, not yours.

The commercial-VPN threat model is “I don’t trust my ISP.” The VPN replaces the ISP with a single company that you trust more (or that you think you trust more). If the VPN is compromised, logs are subpoenaed, or the VPN itself is hostile, all your traffic is exposed. This is not a hypothetical: VPN companies have been compromised, have been compelled to log, and have been acquired by surveillance-adjacent parents.

Tor encrypts your traffic and routes it through three unrelated volunteer-run nodes, each of which only knows one hop. The entry guard knows who you are but not what you’re doing; the middle relay knows nothing useful; the exit knows what you’re doing but not who you are. For any single party to deanonymize you, they need to control or observe both your guard and your exit, which is exponentially harder than compromising a single VPN.

Tor’s threat model is “I don’t trust any single party with both who I am and what I’m doing.” It is structurally stronger than any commercial VPN can be.

For the self-hosted and mesh case:

WireGuard between machines you control gives you an encrypted tunnel with no third party. Tailscale, NetBird, and ZeroTier add convenient mesh networking on top, with a central coordinator that handles peer discovery and access control. Self-hosting that coordinator (Headscale for Tailscale clients; NetBird in self-hosted mode) closes the structural capture-risk that the hosted-coordinator products introduce. None of this is an anonymity tool; it’s an “encrypted private network between your devices” tool.

The Nostr-rooted frontier is nostr-vpn plus its underlying FIPS protocol, a sovereignty-aligned mesh where your identity is a Nostr keypair and coordination happens over public Nostr relays. Alpha-grade in 2026; the architecture is what to learn from now, deployment comes later.

The exception worth naming:

A commercial VPN may make sense as a transport for getting to Tor when your ISP blocks Tor directly. User → VPN → Tor → internet. This is the “Tor over VPN” configuration and is the only legitimate use of a VPN alongside Tor that the Whonix project documents. It is advanced, not leak-tested at the level of the base Whonix setup, and is unnecessary unless your ISP actually blocks Tor. For the more general case of accessing Tor in a censored environment, Tor’s pluggable transports (Snowflake, Meek, obfs4) and bridges are the project’s own answer.

For the typical user with no specific reason: use Tor (via Tor Browser or Whonix) when you need anonymity; use WireGuard or Headscale when you need a private network between your own devices; don’t bother with a commercial VPN unless you have a narrow specific reason (geographic bypass, hostile-ISP bypass). The whole “VPN for privacy” marketing of the 2010s and 2020s sold a weaker product to people who wanted stronger.

For the full landscape of VPN, mesh, overlay, anonymity, and off-grid networking options, see choosing-networking-tools.md.

Browser hardening

The browser is the largest attack surface most users have. Several real options.

• Tor Browser. Firefox-based, configured by the Tor Project, ships through Tor. The right choice for sensitive sessions and the only choice for genuine browser-level anonymity. Slow (Tor is slow), incompatible with sites that block Tor exits (Cloudflare-protected sites in particular). Not a daily-driver browser.
• LibreWolf. Firefox without telemetry, configured for privacy by default, with the worst Firefox-tracking-defaults flipped. Available in Devuan or via Flatpak. The right default daily-driver browser for a privacy-conscious user. Comes with uBlock Origin and reasonable defaults.
• Mullvad Browser. Firefox-based, made by the Mullvad VPN company in collaboration with the Tor Project. Designed to give Tor Browser’s anti-fingerprinting properties without Tor itself; designed to be used over a VPN for users who want Tor-level browser anonymity without Tor-level speed costs. A reasonable middle ground if you’re already on Mullvad’s VPN; less interesting otherwise.
• arkenfox user.js. Not a browser; a configuration file for stock Firefox that flips the same privacy settings LibreWolf does, plus a configurable amount more. The right answer for users who want stock Firefox’s update cadence with LibreWolf’s defaults.

Don’t use Chrome. Don’t use Edge. Don’t use Safari. The threat model these documents target includes the vendor of the browser as part of the threat.

For Devuan plus the hardening doc, the right browser stack is LibreWolf as default plus Tor Browser available for specific sessions. Add Firefox Multi-Account Containers (or arkenfox’s container patterns) to keep work, personal, and miscellaneous browsing isolated within the same browser.

This document doesn’t cover browser hardening in more detail; there are dedicated projects that do it better. PrivacyGuides.org has up-to-date recommendations and is independent of any commercial party. A future choosing-browser-hardening.md is on the roadmap.

Credential isolation across machines

The pattern: separate the machine that writes code (or composes emails, or signs documents) from the machine that holds the credentials to push code (send emails, distribute signed documents).

The threat model: an attacker who compromises the writing machine should not automatically gain credentials to act as you on the network. Compromise of the writing machine leaks code-in-progress, draft emails, unsigned documents, bad, but bounded. Without credential isolation, compromise of the writing machine leaks the credentials too, much worse, because the attacker can now act as you on the network and the bad version of every project gets pushed. The IronWorm npm worm in June 2026 was exactly this cascade: it stole developers’ npm publish credentials and republished itself into their own packages. The supply-chain section below covers the defenses; why-secure-your-system.md has the case.

The Qubes pattern is the cleanest: split GPG and split SSH. A dedicated credential-holding qube has the keys; work qubes don’t. When a work qube needs to sign something or push something, it makes a qubes-rpc call to the credential qube; the credential qube performs the operation and returns the result. The credentials never leave the credential qube. The work qube can be wiped and rebuilt without losing the credentials.

The non-Qubes version is bundle-queue.sh for the git-push case. The work machine creates a git bundle (no credentials needed); the bundle travels by sneakernet or by a controlled transport to a separate credential-holding machine; the credential machine pushes the bundle to the remote. The work machine never holds the remote’s credentials.

Similar patterns for other workflows:

• Email: write on the work machine, transport drafts (encrypted) to a separate machine that holds the SMTP credentials.
• Signing: prepare the document on the work machine; transport to an air-gapped signing machine (Tails or a Qubes offline qube); transport the signed artifact back.
• Cryptocurrency: hot wallet for small transactions and balance display; cold wallet on an air-gapped machine for the bulk; transfer to hot wallet only what you’ll spend.

When to pick into this layer: when you’ve identified a specific credential whose compromise would unacceptably cascade. A personal GitHub credential probably doesn’t justify it. A code-signing key for production software does. An SSH key for production infrastructure does. The cryptocurrency case where balance is meaningful does.

Supply chain hygiene

The code you install is an attack surface in its own right, separate from the documents you open. Every dependency you pull through apt, Flatpak, npm, cargo, pip, or a curl-piped install script is code that runs on your machine, often the moment it installs, with your privileges and your secrets in reach. The IronWorm npm worm in June 2026 is the live example: a binary that fired on install, swept the machine for cloud, npm, AI, and wallet credentials, then republished itself through the victims’ own publishing credentials. The credential-isolation pattern above is half the defense: keep the credentials that publish or sign off the machine that installs and builds, so a compromised build environment cannot act as you. The rest, disabling install hooks where the ecosystem allows it, pinning dependencies by hash, vetting what you add, sandboxing the build of code you have not reviewed, and hardening how you publish, is in choosing-supply-chain-tools.md.

Backup as the final answer

Backups aren’t a layer in the defensive stack; they’re the recovery answer when the defensive stack fails. The detection layer tells you that compromise happened; backups tell you how to come back.

The operational floor: Borg with daily automated snapshots to a local encrypted external drive, plus monthly rotation of a second external drive to an off-site location, plus a verified-working restore procedure. Choosing the backup tool and the discipline around it (3-2-1, off-site rotation, append-only, restore testing) is in choosing-backup-tools.md; the Devuan procedure that implements this floor is in devuan-secure-workstation.md Part 3.1.

The threat model where backups specifically matter: ransomware, disk failure, theft, fire, your own mistake (rm -rf to the wrong directory). For all of these, the answer is “restore from yesterday’s snapshot.” For the off-site-fire case specifically, the answer requires the off-site copy; backups in the same building as the original are one bad day from being no backups.

The hardest part of backups is not the technology; it’s the routine. A backup that’s never tested doesn’t restore. A backup whose passphrase is in the head of someone who’s now in the hospital doesn’t restore. Test the restore quarterly; document the recovery procedure somewhere a trusted person can find it.

Out of scope, deliberately

Things this project does not cover and where to find the equivalent treatment for each.

Browser hardening details. The cross-cutting section above gives the framework. The deep dive belongs to PrivacyGuides.org and the arkenfox project. A future choosing-browser-hardening.md in this project is on the roadmap.

Operational security in general. The discipline of not leaking metadata, not reusing names, not posting on schedules, not getting photographed at the same coffee shop where you do anonymous work. This is the layer no software can provide. Read the EFF SSD, Grugq’s older essays on operational security, and the Tails operational-security documentation.

Compliance, audit logging, fleet management. This project is for individual workstations. Enterprise security at scale is a different conversation.

Security Landscape: The Layered Stack

The map. The security overview is the trailhead and the reading order; why you should secure your system is the reason to bother. This is the terrain in between: what you are actually defending, the layered stack that defends it, how to size your effort, and how far the climb goes. Read it after the overview and before the first rung (the OS picker). You do not need to act on any of it yet; it is orientation, not procedure.

The six layers

Security on a single Linux workstation is a stack of six layers, not a single decision. Each layer defends against a class of threats the others don’t.

1. Foundation. Which OS. Devuan is the default this project recommends; Qubes, Tails, or Whonix for specific high-security scenarios.
2. Confidentiality. Full-disk encryption plus per-file encryption for sensitive data. LUKS at install time; Borg for backups; age for one-off encryption needs.
3. Hardening. Reducing the running system’s attack surface. CPU microcode, kernel sysctls, AppArmor, MAC randomization, encrypted DNS, USBGuard, Thunderbolt/DMA authorization, kernel lockdown.
4. Input vetting. Scanning the documents and files you receive before opening them. ClamAV plus Didier Stevens’ pdfid suite plus YARA, with Firejail as the sandboxed-open layer. The doc-malware-scan.sh script in this project is the operational wrapper. The code you install is a second input class this layer does not cover: a package that runs on install is a different vector from a document you open, and its defenses live in choosing-supply-chain-tools.md.
5. Detection. Catching tampering after it happens. AIDE plus debsums plus auditd is the standard stack.
6. Compartmentalization. Isolating credentials and identities across machines, or workloads on one machine. Qubes covers the on-one-machine case; the bundle-queue script covers the credential-isolation case.

These layers compose; none of them is a wall on its own. Each shifts the odds rather than guaranteeing anything: a hardened machine running a browser can still be exploited, the exploit just costs more, persists less easily, and is likelier to be caught. Depth is the point.

Plus cross-cutting concerns that aren’t a single layer: hardware tokens, Heads coreboot for firmware-level integrity, the Tor-versus-VPN-versus-mesh question, browser hardening, secure messaging, off-machine backups, and the operational-security habits that no software can replace, all covered in the cross-cutting section of the security overview.

Everyone should run the baseline: Linux, full-disk encryption, the hardening floor, document scanning, and integrity monitoring. It is low-friction once set and it helps no matter who is after you. From there, climb as far as your time and money allow: add anonymity (Tor, Tails) for sessions that shouldn’t trace back to you; add isolation (Qubes, identity separation) for workloads that can’t safely share a machine; combine both into the maximal posture (Qubes-Whonix on coreboot hardware) if you’re under active targeting and will keep it maintained. The detail is in “The ascent” below.

The 14 attack surfaces, ranked

First, the broader picture: what attack surfaces actually matter for a normal person’s online privacy. The main OS guide compresses this into one paragraph in “Why your OS matters.” The fuller version, in rough order of how much practical privacy loss each one accounts for:

1. The browser. The single biggest surface. Fingerprinting via canvas, WebGL, fonts, screen resolution, timezone, plugin list. Cookies, supercookies, localStorage, IndexedDB, referrer headers, and the sheer volume of JavaScript execution that can exfiltrate data. Most people spend 90% of their online time here.
2. DNS. Almost always leaks where you go even if everything else is encrypted. Your ISP sees every domain you resolve unless you’re using encrypted DNS, and even then, the resolver operator sees it all.
3. Your ISP. Sees your IP traffic metadata regardless of content encryption. Knows your real identity by contract.
4. The OS itself. Telemetry (Windows is catastrophic here), automatic connections to update servers, NTP pings, captive-portal detection. All happen before you do anything deliberately.
5. Your IP address. Ties your physical location and ISP identity to everything you connect to. VPNs shift trust to the VPN operator rather than eliminating the problem.
6. Account linkage. Logging into any account immediately collapses anonymity across sessions. Email addresses, phone numbers, and OAuth logins are identity anchors.
7. Metadata on communications. Even with end-to-end encryption, who you talk to, when, how often, and message sizes are usually visible.
8. Hardware identifiers. MAC addresses, hardware serials exposed through the OS or browser, CPU/GPU fingerprinting via timing attacks.
9. Behavioral fingerprinting. Typing rhythm, mouse movement patterns, scroll behavior, time-of-day usage patterns. Passive and hard to defeat.
10. Third-party content. Trackers, ad networks, CDNs, embedded fonts, and analytics scripts loaded on pages you visit. A single Google Fonts call or Facebook pixel reports your presence to a third party.
11. Email. Open-tracking pixels, IP leakage in headers from naive clients, and the fact that your counterparty’s provider sees everything.
12. Mobile devices. GPS, cell-tower triangulation, accelerometer fingerprinting, app permissions, and the fact that iOS and Android are both hostile to privacy by design.
13. Payment methods. Credit cards and PayPal create a permanent financial graph of your activity. Extremely hard to break without cash or privacy-preserving crypto.
14. Physical layer. WiFi probe requests broadcast your device’s previously connected SSIDs. Bluetooth does similar things. Your router’s logs exist.

The browser and DNS together account for the majority of practical privacy loss for most people. Everything else matters, but fixing those two has the highest return. The OS sits at position four, which is why the project’s main guide is about switching it. The six layers above are how you systematically address surfaces 1 through 14: foundation (OS at position 4 plus what the OS enables you to defend at positions 1, 2, 5, 10), confidentiality (data at rest, complements 1 and 11), hardening (closes attack surfaces the OS itself opens), input vetting (catches malicious content arriving via surfaces 10 and 11 before it hits the workstation), detection (catches when defenses fail), and compartmentalization (limits damage when one layer falls).

Sizing your effort

The cheapest security is the exposure you never create. You can’t lose a password you never set, leak an account you never opened, or have metadata correlated that you never emitted. Before any tool: shrink your footprint. Fewer accounts, fewer identifiers tied to each other, less posted, less linked, less said. That shrinks every surface on the list above at once and costs nothing but discipline. Do it first, and keep doing it; no tool below recovers privacy you gave away by oversharing.

Then size how much of the rest is worth doing, and in what order. The standard framework is four questions, asked in order.

What am I protecting? Be specific. “My data” is too vague. Concrete answers: my financial records and tax history; my private communications with my source; my cryptocurrency wallet’s seed phrase; my notes on a sensitive medical condition; my draft of the article that will be published next month; the photos on my phone that no one else should see; the credentials that let me deploy code to production. List them. The list will reveal that you have several different protect-targets and they probably need different layers.

Who is plausibly trying to get it? Be honest. For most people: opportunistic malware authors; ad networks and data brokers building behavioral profiles; thieves who would resell a stolen laptop; a vengeful former associate; a corporate employer monitoring its devices; an ISP selling browsing data. For some, additionally: investigators on behalf of a hostile state, a corporate adversary in active litigation, organized criminals targeting cryptocurrency holders, intelligence services targeting journalists’ sources. Knowing who is actually after you doesn’t cap how far you climb (climb as far as you can afford) but it tells you which surfaces to close first.

How likely are they to succeed, and against what? A vague threat model produces vague defenses. If your threat is “opportunistic malware,” the question is whether a browser exploit gets your home directory or just your browser sandbox; AppArmor and hardening fix that. If your threat is “stolen laptop,” the question is whether the disk is encrypted; LUKS fixes that. If your threat is “ISP-level surveillance,” the question is whether encrypted DNS and HTTPS-everywhere cover your traffic; you set those up once. If your threat is “I might be specifically targeted by a state-level actor with physical access to my hardware,” the question is whether Heads-coreboot detects the tampering; that’s a months-of-work answer with hardware purchases.

What happens if they succeed? The effort a defense is worth scales with the cost of failure. If the stakes are “I lose some embarrassing photos,” LUKS plus a reasonable browser is enough. If the stakes are “a source is identified and arrested,” Qubes-Whonix on Heads coreboot starts to look reasonable. If the stakes are “I lose my cryptocurrency cold-storage seed phrase,” the answer involves air-gapped machines and physical paper backups in safe deposit boxes.

Answer all four for each thing you’re protecting. The protect-targets usually vary, your photos need baseline protection; your source’s identity needs the anonymity branch; your cryptocurrency seed needs cold storage, so the answer is to use different layers for different things rather than apply one setting to everything. A common mistake is fixating on the protect-target that feels most dramatic (“I have a journalist friend, I should run Qubes”) while leaving the mundane surfaces that actually get most people (the unencrypted disk, the reused password, the un-vetted PDF) wide open. Close the boring surfaces first.

One more thing, because it cuts against the instinct to remove every annoyance: friction in your setup is often doing protective work you can’t see. The LUKS passphrase you type at every boot, the AppArmor profile that breaks one workflow per quarter, the hardware-token tap for sudo, the USBGuard block when a colleague hands you a stick, every one of those is a problem you could “solve,” and every such solution removes the friction that was the protection. Some problems are immune systems. Be deliberate about which ones you remove.

The ascent: do the baseline, then climb as far as you can afford

The old way to read this was “pick the user you are.” The better way: everyone does the baseline, then climbs. The goal is to be as safe as you can afford in time and money, not as safe as some adversary forces you to be. Aim high. The only real ceiling is the one in the guardrail at the end of this section: don’t build higher than you’ll keep running.

The baseline, everyone, regardless of who’s after you

Low-friction once it’s set, and it helps against every threat from opportunistic malware to a stolen laptop. Build it in this order; each step assumes the last is done.

1. Replace the OS. If you’re still on Windows or macOS, this is the first piece of security work and nothing else here matters until it’s done. Devuan (this project’s default) or Mint; the picker is in os.md. The migration closes the entire vendor-surveillance surface in one move.
2. Full-disk encryption. Read choosing-encryption-tools.md for the tool overview, then run devuan-luks2-install.sh on a fresh install. FDE can’t be retrofitted painlessly, so do it at install time. Long passphrase, written down and stored separately, plus a LUKS header backup on a USB stick kept in a different room, plus off-machine encrypted backups via Borg. Without all three you don’t have working encryption at rest; you have a brick-when-the-disk-dies, a panic-when-you-forget, or a one-fire-from-total-loss setup.
3. Harden the running system. devuan-secure-workstation.md, the install plus runtime-hardening sections, done in order. Skip kernel lockdown unless you understand the trade-offs (it can break DKMS modules and hibernation). Most of this is apt install plus a few config files; hours, not days.
4. Vet documents before opening them. Install doc-malware-scan.sh (choosing-document-scanning-tools.md); it auto-installs the stack on first run. Scan every PDF, EPUB, or Office document from an un-vetted source, then open it in firejail --net=none even after a clean scan. Document-borne malware is the most common compromise path for a normal user, and this returns more protection per minute than almost anything else on this list.
5. Monitor integrity. apt install aide debsums auditd rkhunter, capture a clean baseline with aideinit, then cron daily AIDE and weekly debsums/rkhunter (choosing-hids-tools.md). The hard part isn’t setup, it’s reading the output, decide how you’ll actually see the reports before you turn monitoring on.

Plus the cross-cutting items below, set up once: a hardware token for SSH and 2FA, encrypted DNS (covered in the hardening step), and a privacy-respecting browser (LibreWolf as default, Tor Browser for sensitive sessions). For most people the baseline is the realistic stopping point, and it is already far harder to attack than what almost anyone runs. A motivated weekend-and-a-half gets you here.

Going further, two branches, take either or both as you can afford

These are different axes, not later rungs of one ladder; you might want one and not the other.

Anonymity: for activity that shouldn’t trace back to you. Sessions that need to stay uncorrelated with your daily identity: a journalist’s source contact, a whistleblower’s disclosure prep, research into hostile groups, coordination before a public action. Tools: Tor Browser for browsing, a Tails USB for amnesic sessions (the live-OS scenario in os.md), Whonix for whole-system Tor, a separately-encrypted USB for files that travel with these sessions. The discipline matters more than the tools: never log into a personal account from the anonymous context, never reuse a username across the two, time-separate the patterns. The full landscape of transports is in choosing-networking-tools.md; messaging that fits this branch is in choosing-communication-tools.md.

Isolation: for workloads or identities that can’t safely share a machine. When one app being exploited is genuinely dangerous because of what the next app over holds: a developer who also handles sensitive customer data, a cryptocurrency operator, a lawyer with privileged files and a personal life on the same hardware. Qubes OS is the maximalist answer (every app in its own VM; the hardware floor is real, 16GB RAM, IOMMU-capable CPU, supported laptop; the compartmentalized-daily-driver scenario in os.md). Short of Qubes, AppArmor gives per-program confinement on hardened Devuan, and privacy-setup.md covers identity separation across users and VMs. For credentials specifically, keep the machine that writes separate from the machine that holds push/sign keys: Qubes’ split-GPG and split-SSH, or the bundle-queue pattern for the git-push case.

Mobile runs parallel to all of this and can be climbed any time after step 1. choosing-phone.md (Pixel plus GrapheneOS) and grapheneos-install.md. At higher postures it stops being optional: a hardened workstation paired with a stock phone that betrays you is a half-measure.

The maximal posture, for active targeting

Both branches combined, on hardware you’ve taken control of. This is the stack for someone whose threat model genuinely includes nation-state-class adversaries: journalists where journalism is prosecuted, activists under state-aligned surveillance, people whose community is in an aggressive state’s crosshairs. The shape: Qubes-Whonix on Heads-coreboot hardware (the hardware purchase is itself part of the security work, supply-chain interception is in the threat model; choosing-hardware.md); LUKS with /boot on a USB you carry and Argon2id key derivation; full hardening of every qube template with kernel lockdown and USBGuard enforced; DNS routed over Tor via the Whonix gateway; documents quarantined through a no-network analysis qube before they touch anything; integrity monitoring with off-host log delivery; multiple persona qubes; an air-gapped offline qube for signing; a hardware token for every credential; GrapheneOS on the phone; and operational discipline that treats every network interaction as logged. Setting it up properly is weeks; maintaining it is ongoing. At this level the documents in this project are the entry point, not the destination, read PrivacyGuides.org, the EFF Surveillance Self-Defense guide, Tails’ operational-security docs, and the Whonix forum.

The guardrail

Climb only as high as you will actually keep running. Security you abandon is worse than security you never built: it costs you the setup effort and then pays you back in false confidence, the Qubes install you stopped maintaining, the AIDE reports nobody reads, feel like protection while protecting nothing. “Affordable” includes attention, not just money and a setup weekend. So aim high, but pick the highest posture you’ll sustain and run that, not the one that looks most impressive in a threat-model diagram.

Choosing the right operating system: in 2026

A guide for people who want to stop letting Microsoft and Apple own their computer. Written for beginners. Updated as things change.

TL;DR

If you’ve never used Linux and you’re coming from Windows, install Debian stable with the Cinnamon or XFCE desktop. Modern Debian installs about as easily as anything else, because since Debian 12 it ships the non-free firmware that used to make a first install painful[^debian-firmware], and it sits on the trunk rather than a corporate derivative, so it will still be the right call in five years. If you want the same thing without systemd, install Devuan instead. Linux Mint is still the most Windows-like landing if visual familiarity is the one thing you care about, but it now carries real caveats (see its entry), and this guide no longer steers first-timers to it by default.

Debian or Devuan is a destination, not a waystation; for most people it is where you can stop. If after six months you want to go further, the directions branch. For a leaner no-systemd system, Void. For bleeding edge, Arch. For total control and understanding, Gentoo. For privacy as a first principle, Qubes OS with Whonix templates, or Tails on a USB stick for sessions where nothing should persist.

Do not install Ubuntu in 2026. Its minimum RAM requirement (6GB) is now higher than Windows 11’s (4GB), its CPU requirement (2GHz dual-core) is double Windows 11’s (1GHz dual-core)[^ubuntu-req], Snap keeps silently replacing apt packages, and a local-root privilege escalation in the default Snap stack (CVE-2026-3888) was disclosed in March 2026[^cve-3888]. Ubuntu is the distro Linux was supposed to be an alternative to.

The rest of this doc shows the work. If you don’t know what a “distro” is, start at Terms you’ll need.

Why your OS matters: the biggest attack surface you have

Most people think about privacy the wrong way around. They worry about which browser they use, which VPN they pay for, whether to turn off cookies. These matter, but they’re all downstream of a larger question: what operating system is quietly watching you the whole time?

Your OS is the software that runs everything else. It sees every keystroke before your browser does. It sees every file before your password manager encrypts one. It sees every network request before your VPN touches it. If your OS is hostile, nothing you do on top of it is private. This is not theoretical: Windows sends telemetry to Microsoft by default, macOS sends analytics to Apple by default, and both platforms have quietly expanded what “telemetry” means over the last decade without really asking. Windows 11’s Recall feature takes periodic screenshots of your screen and stores them. Microsoft account login is now required for setup. Apple scans files on your own machine against a hash list and decides whether to let you open them (“notarization”). You paid for the hardware; they took a seat at the table anyway.

Worse, data brokers have quietly industrialized what used to be the domain of intelligence services. Companies like Palantir and its peers don’t need to hack you: they buy you. Location history from weather apps, purchase history from loyalty programs, browsing patterns from the ad SDKs embedded in every free app, social graphs from anywhere you’ve logged in with Google or Facebook, call and text metadata from telco partnerships, all of it is aggregated, packaged, and resold into detailed profiles that intelligence contractors, law enforcement, and private clients purchase legally. Epstein’s little black book was not a fluke of one man’s networking. It was a demonstration of what happens when powerful people collect leverage over other powerful people, and in 2026 that same function has been industrialized and automated at scale, except now it’s your data too, not just the elite’s, and the buyers are not always people you would trust with it. Running Linux does not sever every pipe (your ISP still sees your traffic, your phone still pings towers) but it removes one of the largest and most leaky attack surfaces: the OS itself. Windows telemetry and macOS analytics are direct feeds into ecosystems where Microsoft and Apple are themselves data businesses with partner relationships you’ve never consented to in any meaningful sense. Linux running a hardened configuration with a decent firewall, combined with a trustworthy DNS resolver, means you’ve closed the hole that most people don’t even know is open. It won’t make you invisible, but it stops you from being low-hanging fruit, and in a world where data is used to build leverage, being harder to profile than the next person is not paranoia, it’s basic hygiene.

In rough priority order, the largest attack surfaces for a normal person’s online privacy are: the browser (fingerprinting, JavaScript, cookies, localStorage); DNS (every domain you resolve is visible to someone upstream unless you use encrypted DNS); your ISP (contractually knows who you are); the OS itself (telemetry, update servers, captive portal checks, NTP pings); your IP address; account linkage (logging in anywhere collapses anonymity across sessions); communication metadata (who-when-how-often is visible even on encrypted chat); hardware identifiers (MAC addresses, CPU serials); behavioral fingerprinting (typing rhythm, mouse patterns); third-party content (ad trackers, analytics, embedded fonts and pixels); email (open-tracking pixels, IP leakage); mobile devices (GPS, cell towers, app permissions, iOS and Android are both hostile by design); payment methods (cards and PayPal are a permanent financial graph); and the physical layer (WiFi probe requests, Bluetooth beacons).

The browser and DNS together account for most of a normal person’s practical privacy loss, which is why most advice starts there. So why does this project start with the OS instead of the browser? Because the browser runs on top of the OS, and a hostile OS sees everything the browser does before any browser setting takes effect. Windows 11’s Recall feature periodically screenshots your screen; an OS can log your keystrokes before the browser ever receives them. Hardening the browser on Windows or macOS is building on sand, because the substrate underneath can quietly undo it. You cannot fix the OS while you are still on Windows or macOS, because the whole system is designed around a relationship with the vendor. So you switch the OS first, which is the one move that closes the entire vendor category and gives the browser solid ground to stand on, and then you harden the browser early, as the very next thing you do. Starting with the OS is not a claim that the OS matters more than the browser; it is that the browser fix only holds once the OS underneath it is trustworthy.

A note to privacy experts and freedom fighters

The Iron Law of Oligarchy and the entryism behind it are laid out in why-secure-your-system.md; this section is its distro-watching corollary. People who know how an operating system works have a specific obligation to people who don’t. If you are a maintainer, a security researcher, a long-time Linux user, or anyone else who can read a commit graph and recognize a capture pattern when you see one, you are the canary. The rest of us (people who don’t know how to build a kernel, who bought the machine to do our actual work) depend on you to tell us when a project has drifted, when a maintainer has been co-opted, when a corporate acquisition has changed what a distro is actually doing underneath. The pursuit of privacy is a never-ending chase because the institutional forces pulling the other way never rest. You are the ones in a position to see it first.

Specifically, if you see evidence that a widely-recommended beginner distribution has been quietly changed (telemetry added, corporate sponsorship introducing policy changes, governance captured, package trust compromised) post about it loudly, write it up, make it findable. Cautionary cases already exist. Ubuntu was the friendly, accessible Linux for a decade; the Snap enforcement and the 2026 system-requirements jump are the current symptoms of a long drift. Linux Mint is the recommendation most experienced users still give to Windows migrants today, and its very popularity makes it the most obvious next target for capture or compromise. The project is currently clean (independent wireshark audits in 2025 found no Mint telemetry beyond basic update-server contact[^mint-telemetry]) but there is already an unexplained Cinnamon memory leak on Mint 22.2 that was reported to the official repository on 22 December 2025, four months ago, and remains open and unassigned as of this writing (GitHub issue linuxmint/cinnamon#13298[^mint-leak]). One open bug does not prove capture, but it does prove the project’s quality-control is slipping at exactly the moment its install base is growing fastest on the back of the Windows 10 end-of-life migration. Users who upgraded to Mint 22.2 and noticed their machine getting slower over the winter were not imagining it.

If you’re someone who can tell when the pattern is getting worse, please tell us.

The practical takeaway for your own setup: treat every recommendation in this document, including the strong ones, as true right now, not forever. Check on your distro once a year. Read the project’s commit history and governance disputes, not just its marketing page. When a project starts behaving differently, leave.

Contents

• TL;DR
• Why your OS matters: the biggest attack surface you have
• A note to privacy experts and freedom fighters
• Why Windows and Mac stopped being good
• Terms you’ll need
• Picking a desktop: what Cinnamon, GNOME, XFCE, KDE actually mean
• Decision shortcut
• What not to pick in 2026
• How this doc is structured
• What counts as “widely used” in 2026
• The install-once-and-forget axis
• The systemd problem
• Community politics and the vibe test
• Debian
• Ubuntu
• Linux Mint
• MX Linux
• Fedora
• Arch Linux
• EndeavourOS, CachyOS, Manjaro (Arch derivatives)
• Omarchy
• Devuan
• Artix
• Void Linux
• Gentoo
• NixOS
• Slackware
• openSUSE Tumbleweed and Leap
• OpenMandriva
• Pop!_OS
• Zorin OS
• Bazzite and SteamOS (gaming)
• Alpine
• High-security operating systems: how to pick
• Qubes OS
• Whonix (and the VPN question)
• Tails
• Kicksecure
• Vendefoul Wolf
• Trisquel and PureOS (libre)
• Where Linux comes from: the Unix lineage
• BSDs as an adjacent path
• Further reading: how to actually learn Linux
• Changelog

Why Windows and Mac stopped being good

Windows 10 reached end-of-support on 14 October 2025[^w10-eol]. Microsoft’s answer is Windows 11, which requires TPM 2.0, Secure Boot, and a CPU from an approved list; hundreds of millions of otherwise-working machines don’t qualify. For the ones that do, the default install now includes: Recall taking periodic screenshots and storing them, a forced Microsoft account at first boot, Copilot wedged into the shell, advertisements in the Start menu and on the lock screen, ads in the file manager, Cortana, and a steady background push toward OneDrive and Microsoft 365 subscription services. Telemetry is on by default and deliberately hard to fully disable, even when you think you’ve switched everything off in Settings, you haven’t fully. Updates install when Microsoft decides, not when you do, and they have bricked machines more than once. You need a Microsoft account just to finish setup. The OS phones home constantly. You are running a surveillance platform that also plays games.

Windows defenders will say “just tweak the settings.” The fact that you have to spend hours un-defaulting your way out of corporate surveillance to use your own machine is the indictment, not the defense.

Mac is cleaner and more coherent, and Apple locks you in tighter than anyone. The hardware is proprietary. The software ecosystem is a walled garden. Repairs require Apple’s permission, financially if not literally. Gatekeeper actively blocks software Apple hasn’t blessed (notarization puts Apple in the middle of software distribution). Every few years Apple drops support for hardware that is still perfectly capable, you paid $3,000 for a machine that Apple will tell you, three to five years later, can no longer run the current OS. Applications get sandboxed more aggressively every release. The M-series chips are fast and efficient; you are also not allowed to know what is running on them at a deep level. The privacy posture Apple markets hard is real only up to the point where Apple’s own telemetry starts, and Apple scans files on your own machine against a hash list to decide whether to let you open them. You’re not buying a computer. You’re renting an expensive aesthetic.

Both companies have decided that your computer is a client for their cloud. Linux is the last mainstream way to run your own computer. It isn’t perfect, but it is yours, you choose the kernel, the init, the shell, the filesystem, the package manager, the desktop, and which parts of it phone home (in most distros, none). In 2026 you can also run Linux on the same hardware most people buy, watch Netflix in a browser, join a Teams call, edit photos, write code, and stream to a TV, all without a subscription or a vendor account. For most people, the remaining friction is smaller than the friction of putting up with Windows 11.

When keeping a Mac for one specific thing is OK

A realistic concession the doc owes you: for some specialized production work, Mac’s software stack is genuinely hard to match on Linux, and the honest posture is not “leave Mac, period” but “leave Mac for general computing, and keep a Mac as a dedicated tool if you actually need what it does.” Final Cut Pro and Logic Pro have no Linux equivalent. DaVinci Resolve runs on Linux but the free version strips out AAC and MP3 in/out, several codecs need manual workarounds, and driver integration is rougher than on Mac. Adobe refuses to ship for Linux and the open-source alternatives (GIMP, Krita, Inkscape, Kdenlive) are good but not drop-in replacements for a working Creative Cloud pipeline. Logic, Pro Tools, and most of the commercial audio-production ecosystem assumes Mac or Windows.

Bryan Lunduke, who is otherwise one of the louder voices against consumer-OS drift and runs OpenMandriva as his daily driver, still uses a Mac for video production on The Lunduke Journal. That isn’t a contradiction. It’s a working adult’s acknowledgment that the right tool for one specific professional workflow is not always the right tool for the rest of computing. A sovereignty-minded posture does not require that every machine in your house run Linux. It requires that your general-purpose computing (the machine you browse on, the machine you write on, the machine that holds your files) is not a surveillance client. A Mac Mini or MacBook kept offline-ish as a dedicated video workstation is a tool, not a surrender. The distinction that matters is whether a given machine is your general computing life or a specialized appliance.

Terms you’ll need

A handful of words get used constantly in the Linux world and almost never defined. If you’re new, this section is here so the rest of the doc reads normally.

Kernel. The single piece of software that talks directly to your hardware, CPU, RAM, disk, keyboard, screen, network. Every other program asks the kernel instead of touching hardware itself. Linux is, strictly, just a kernel. Linus Torvalds wrote it in 1991 and still runs its development. When people say “Linux,” they usually mean a kernel plus a bundle of other software built around it.

Distribution, or distro. A specific bundle: the Linux kernel plus a package manager, a set of default applications, a desktop environment, and a bunch of defaults about how things are configured. Ubuntu is a distro. Debian is a distro. Fedora is a distro. There are hundreds. They mostly differ in which bundle they ship and how it’s maintained.

Base distro, upstream, downstream. Some distros are built from scratch (Debian, Arch, Gentoo, Slackware, Void, Alpine). Others are built on top of those. Ubuntu is built on Debian. Linux Mint is built on Ubuntu. When something is fixed, it often gets fixed at the source first and flows down: Debian fixes a bug, Ubuntu pulls the fix from Debian, Mint pulls it from Ubuntu. The distro your distro is built on is called its upstream. A distro built on top of yours is downstream. This matters mostly for two things: when something breaks, the fix usually originates upstream; and when you search for help, tutorials for your upstream mostly apply to you with small adjustments.

Package manager. The tool that installs, removes, and updates software on your system. apt on Debian and Ubuntu and Mint, dnf on Fedora, pacman on Arch, zypper on openSUSE, xbps on Void, apk on Alpine. They are all solving the same problem (don’t download installers from random websites; let a vetted repository handle it) with different commands.

Init system. The first program the kernel starts after boot. It then starts everything else, networking, logging, your graphical login screen. For two decades the default was a simple script-based system called sysvinit or its descendants (OpenRC, runit). In 2010 a Red Hat engineer named Lennart Poettering started a replacement called systemd. Systemd does much more than just start processes; this is the core of the controversy. See The systemd problem below.

Desktop environment. The visible part of Linux: the taskbar, window decorations, file manager, default theme. GNOME, KDE Plasma, Cinnamon, XFCE, MATE are the main ones. A single distro usually offers several; the “flavor” or “edition” you download picks one. See Picking a desktop below for what each one actually means in practice.

Rolling release vs stable release. A rolling distro updates individual packages continuously: Arch, Void, Tumbleweed, Gentoo, Devuan’s testing branch. You get new software fast but occasionally something breaks. A stable-release distro freezes the whole package set for two or three years, only shipping security fixes: Debian stable, Ubuntu LTS, Mint, Rocky Linux. You get older software but almost no surprises.

LTS. “Long-Term Support.” A specific release is promised updates for five or ten years. Ubuntu LTS releases (every two years, supported five years without Ubuntu Pro, ten years with it) are the famous example.

systemd-free. Shorthand for distros that still use sysvinit or OpenRC or runit instead of systemd. Devuan, Artix, Void, Gentoo (by default), Alpine, Vendefoul Wolf. See The systemd problem.

Live USB. You can put Linux on a USB stick and boot your computer from it without installing anything. The distro runs from the USB, touches nothing on your hard drive. Good for trying a distro, or for privacy tools like Tails where the whole point is leaving no trace. If you can’t even make a USB stick (locked-down work laptop, no admin, no spare hardware) Fabrice Bellard’s JSLinux (bellard.org/jslinux) runs a Linux shell (Alpine, Buildroot, or Fedora RISC-V) directly in a browser tab via WebAssembly. It is a curiosity, not a distro evaluation: the hardware is fake, so it tells you nothing about whether your real Wi-Fi or trackpad will work, and the demo images are tiny. Good for thirty seconds of “so this is what a shell looks like.”

Flatpak and Snap. Two different ways to install an application without worrying about system libraries, the app ships with everything it needs. Flatpak is community-run, cross-distro, and generally fine. Snap is Canonical’s (Ubuntu’s parent company) version; its backend is proprietary and its behavior has been controversial. This is why Linux Mint strips out Snap on install.

AUR (Arch User Repository). A community-run collection of build scripts for Arch Linux that makes almost any software installable via a simple command, even if it isn’t in the official Arch repos. Roughly 100,000 packages. Only applies to Arch and its derivatives (EndeavourOS, CachyOS, Artix).

Immutable base filesystem. A newer approach, used by Fedora Silverblue, Bazzite, and a few others. The core operating-system files are marked read-only and updated as a single transaction, either the whole update succeeds and you reboot into the new system, or it fails and you stay on the old one. You install your applications in a separate layer (usually Flatpak, or a container tool like Distrobox). The practical consequence: you can’t easily apt install a system package, but you also can’t break your system by accident.

Hardening. In OS security, “hardening” means changing the defaults so that if something goes wrong, the damage is smaller. Vanilla Linux boots with a lot of features enabled that most people never use (some kernel modules, some network protocols, some permission combinations) and any of those can become an attack surface if a bug turns up. A hardened system switches those off by default and tightens the remaining ones. See the Kicksecure entry for a concrete example.

Picking a desktop: what Cinnamon, GNOME, XFCE, KDE actually mean

A desktop environment (DE) is the visible part of Linux, the taskbar, the window decorations, the file manager, the default look. The DE is separate from the distribution; most big distros offer several, and you pick the one you want at install time by choosing an “edition” or “flavor” (e.g. “Fedora Workstation” ships GNOME, “Fedora KDE Spin” ships KDE Plasma, same underlying Fedora). Here’s what the main options actually are.

Cinnamon. Looks like Windows. Taskbar at the bottom, start-menu-style application launcher in the corner, system tray, minimize/maximize/close in the top-right of each window. Moderately heavy, uses more RAM than XFCE but less than GNOME. Developed by the Linux Mint project, which is why it’s Mint’s flagship edition. Runs well on anything with 4GB of RAM and up. Pick this if you’re leaving Windows and want the transition to be invisible. First-class on Linux Mint; available as a package on Debian, Fedora, Arch, and most others.

GNOME. Looks like macOS, or like a tablet-first desktop. No taskbar by default; an “Activities” overview for switching apps; a single top bar. Heavy on RAM (roughly 1.5–2.5 GB idle). Minimalist to a fault, many options Windows/KDE users expect require installing extensions. Opinionated in ways that frustrate some users and delight others. Ships as the default on Fedora Workstation, Ubuntu, Pop!_OS, Zorin OS, and Debian’s default edition. Pick this if you liked the Mac, or you want a clean, gesture-first desktop and don’t mind trading configurability for coherence. Note: GNOME’s project leadership has been politically active in ways some readers will care about; see GNOME’s political turn and the Code of Conduct asymmetry for the record.

KDE Plasma. Maximally configurable. Every visible element (panel, menus, widgets, animations, keyboard shortcuts, notification behavior) can be moved, swapped, or reconfigured. Surprisingly lean in recent versions, Plasma 6 idle RAM is competitive with Cinnamon. Ships as the default on openSUSE, Fedora KDE Spin, KDE Neon, Kubuntu, and the KDE edition of Linux Mint. Pick this if you want control over every pixel, or if you’re the kind of person who likes tweaking the desktop to match your exact preferences.

XFCE. Minimal, lightweight, traditional. Looks approximately like a classic Windows desktop out of the box, taskbar, menu, basic panel. Uses very little RAM (roughly 400–700 MB idle) and runs well on modest hardware. Stays out of your way. Ships as the default on Xubuntu, MX Linux, and the XFCE editions of Linux Mint, Debian, and Manjaro. Pick this for a ten-year-old laptop, or for a daily driver where you want nothing fancy.

MATE. A fork of the old GNOME 2 (pre-2011 GNOME, before the GNOME 3 redesign that alienated much of the existing user base). Traditional, stable, modest RAM use. Similar role to XFCE but with a different aesthetic lineage, slightly more polished, slightly less minimal. Ships on Ubuntu MATE and the MATE edition of Linux Mint. Pick this if you specifically liked GNOME 2 or you want a traditional desktop with a bit more finish than XFCE.

Two smaller DEs worth knowing about if your hardware is genuinely old: LXQt (lightweight Qt-based desktop, even leaner than XFCE at ~300 MB idle; default on Lubuntu) and Budgie (a Solus-project desktop, modern-looking, moderate weight; ships on Ubuntu Budgie and Solus). Neither is a primary recommendation here but you may see them mentioned.

One thing worth naming about XFCE in particular: if you install the XFCE edition of Mint, Debian, Devuan, Artix, Ubuntu, and Void and line them up, they look almost the same out of the box. That’s not a coincidence and it’s not laziness. XFCE’s upstream ships a coherent set of tools (Thunar file manager, xfce4-terminal, Mousepad text editor, Ristretto image viewer, Xarchiver archive manager, xfce4-taskmanager, xfce4-screenshooter, xfce4-appfinder, xfce4-panel, xfwm4 window manager, xfdesktop background) that distros rarely deviate from. The init system underneath (systemd, OpenRC, runit, sysvinit) doesn’t change which XFCE apps are installed. Init is the plumbing; the desktop is the visible furniture. Practical consequence: if you learn XFCE on Mint you know XFCE on Devuan and Void; moving between them is a matter of learning the package manager and the init system, not a new desktop.

Practical summary: Cinnamon if you’re coming from Windows. GNOME if you’re coming from Mac or you want modern-and-minimal. KDE Plasma if you’re a tinkerer who wants control. XFCE if your hardware is modest or you’re conservative. MATE if you specifically miss GNOME 2.

Decision shortcut

If you’re coming from Windows and want it to feel similar:

1. Primary: Debian stable with the Cinnamon or XFCE desktop. Installs cleanly on modern hardware (firmware included since Debian 12), sits on the trunk rather than a corporate derivative, and is the choice you will not have to revisit. Cinnamon gives you most of Mint’s Windows-like layout on the trunk. Pick this unless raw visual familiarity is the only thing you care about.
2. Linux Mint Cinnamon. The closest cosmetic match to Windows, rock-solid feel, huge community, strips out Snap. Pick this if visual familiarity overrides everything else, and read the caveats in the Linux Mint section first (a shipped Cinnamon memory-leak regression, and capture risk that grows with its popularity).
3. Zorin OS. Windows-lookalike polish, explicit migration tooling, GNOME under the hood. Pick this if you want even more hand-holding and are fine paying for Zorin Pro for extra layouts.

If you’re coming from Mac and want the craft and cohesion:

1. Primary: Fedora Workstation with GNOME. The closest Linux comes to the Mac “everything designed together” feel. Pick this if you valued the Mac’s coherence and you’re OK with annual major-version upgrades.
2. openSUSE Tumbleweed with KDE Plasma. Pick this if you want a rolling release with serious testing and you prefer KDE’s configurability to GNOME’s opinionation.
3. Debian with GNOME. Pick this if you want Fedora’s feel without the Red Hat / IBM corporate backing.

If you want something stable, quiet, and long-lived (my pick for most people once they’re off training wheels):

1. Primary: Debian stable with XFCE. The default for people who want to install once and stop thinking. Non-corporate governance, massive repo, boring on purpose.
2. Devuan stable. Pick this if you want Debian without systemd. See The systemd problem.
3. MX Linux. Pick this if you want Debian stable with nicer defaults and a friendlier installer.

If you want no systemd on principle or practice:

1. Primary: Devuan. Debian-compatible, largest non-systemd ecosystem, supports sysvinit / OpenRC / runit / s6. Pick this if you want apt and Debian stability without the systemd dependency tree.
2. Void Linux. Independent from the ground up, runit, xbps, rolling. Pick this if you want something leaner and more modern than Devuan.
3. Artix. Arch minus systemd. Pick this if you want AUR access and rolling-release freshness with init freedom.
4. Gentoo. Pick this if you want source-based compilation and full control over every build flag.

If you want bleeding edge:

1. Primary: Arch Linux. Best wiki in all of Linux, AUR has almost everything, rolling. Pick this if you’re comfortable reading documentation and want the canonical Arch experience.
2. EndeavourOS. Pick this if you want Arch with a friendly installer and nothing else added.
3. CachyOS. Pick this if you want Arch tuned for performance on modern CPUs.
4. openSUSE Tumbleweed. Pick this if you want rolling release with real QA behind it.

If privacy is your first principle:

1. Primary: Qubes OS with Whonix templates. Virtualization-based isolation; every app in its own VM; Tor-forced qubes available. Pick this if you have the hardware (16GB+ RAM, compatible CPU) and you’re willing to learn the model. 1.1. Primary: Qubes with default Fedora and Debian templates only, then add Whonix templates once you’re comfortable. 1.2. Qubes with Whonix-Gateway and Whonix-Workstation as the first templates. Pick this if Tor isolation is the reason you’re installing Qubes.
2. Tails. Pick this if you want an amnesic live-USB that routes everything through Tor and forgets when you shut down.
3. Whonix on your existing host. Pick this if you want Tor-isolated VMs without the Qubes commitment.
4. Kicksecure. Pick this if you want Whonix’s hardening without Tor in the mandatory path.

If you want gaming to just work:

1. Primary: Bazzite. Fedora-based with an immutable base filesystem, preconfigured for Steam/Proton, desktop and handheld variants, HDR/VRR supported.
2. CachyOS. Pick this if you want bleeding-edge kernels and Arch-style maintenance.
3. Nobara. Pick this if you want a Fedora-based approach with a more traditional (mutable) filesystem.
4. OpenMandriva (as a general daily driver that also games). Pick this if you want a community-governed RPM-family distro with KDE Plasma, Proton pre-packaged in the repos, and no corporate owner. Not specialized for gaming the way 1–3 are, but Bryan Lunduke runs it as his primary machine and games on it; if that posture fits your priorities better than frame-rate optimization, this is the pick.

If you want BSD:

1. Primary: OpenBSD. Smallest audited codebase of any general-purpose OS, security-correct by default. Pick this if minimalism-as-security is the goal.
2. FreeBSD. Pick this if you want a production-serious Unix for servers or a NAS.
3. GhostBSD. FreeBSD’s base with a MATE desktop, graphical installer, and ZFS-by-default. Pick this if you want FreeBSD for desktop use without the bring-your-own-DE assembly FreeBSD vanilla expects.
4. NetBSD. Pick this if you have unusual hardware or want extreme portability as a principle.

My pick for most readers leaving Windows: Debian stable with the Cinnamon or XFCE desktop on the primary machine, Tails on a USB for anything sensitive. Mint is the fallback only if Windows visual familiarity is the one thing you cannot compromise on. My pick if this doc had to compress into one sentence: Void Linux, independent, lean, no systemd, not owned by a corporation, built by people who ship software instead of running a political project.

What not to pick in 2026

1. Ubuntu. The reason most of this doc exists. Ubuntu 26.04 LTS (released 23 April 2026, same day as this doc) raised the minimum RAM requirement to 6GB, up from 4GB; Windows 11 still lists 4GB minimum. Ubuntu now requires 50% more RAM and double the CPU speed of the proprietary OS it was supposed to be a lightweight alternative to[^ubuntu-req]. Snap installs silently replace apt install commands; the Snap backend is proprietary; and in March 2026 Qualys disclosed a local-root privilege escalation (CVE-2026-3888, CVSS 7.8) in the interaction between Snap and systemd-tmpfiles that affects default Ubuntu Desktop 24.04 and later installations[^cve-3888]. An “AI bubble” has driven RAM prices sharply higher at exactly the moment Canonical decided to require more of it. The 2026 RAM bump affects every Ubuntu user at a moment when the project’s answer to “why” is essentially “because modern GNOME needs it.” This is not the Ubuntu that existed in 2015. Do not install new machines on it. Mint and Debian can both do everything Ubuntu does for you.
2. Manjaro. Arch-derived but ships its own delayed repo that’s regularly the cause of breakage, and the team has shipped expired SSL certificates on their own website more than once. If you want Arch, use Arch or EndeavourOS. Manjaro is strictly worse.
3. elementary OS. Beautiful but the project has been slow since the 2022 co-founder feud. Fine if you’re already on it; not a fresh recommendation.
4. Deepin. Chinese-state-adjacent, ships its own desktop with closed components, repeated privacy concerns around telemetry. Avoid.
5. CentOS Stream as a server base. Red Hat turned CentOS into upstream-of-RHEL rather than downstream. Rocky Linux and AlmaLinux are the real CentOS successors; Debian is the no-corporate-middleman alternative.
6. Pop!_OS while waiting for COSMIC. COSMIC is still alpha/beta as of early 2026. If you’re starting fresh, start on Fedora or Mint and switch later if COSMIC ships well.
7. Anything marketed as an “AI Linux” or “crypto-native OS.” These are usually re-themed Ubuntu with a credential-stealing angle.

How this doc is structured

This is a living reference, not a ranking. The Linux landscape has a stable core (Debian, Fedora, Arch, Slackware, Gentoo) and a churning frontier (immutable distros, Nix-style systems, gaming-focused Fedora remixes). The advice here separates the two so the stable parts age slowly and the volatile parts can be updated without rewriting the doc. Each per-distro entry covers: what it is, who owns it, who it’s for, what it’s bad at, current trajectory, and roughly where it sits on the community politics axis.

What counts as “widely used” in 2026

Desktop Linux crossed roughly 4.7% global share in 2025 and is tracking toward 5-6% by end of 2026, driven substantially by Windows 10’s October 2025 end-of-support[^linux-share]. The United States crossed 5% in June 2025. India sits at roughly 16% as of mid-2024, the highest major-economy share. Steam’s Hardware Survey has Linux at around 2.3-3% through 2025, with SteamOS (Arch-based) the single largest Linux slice.

Distribution-level numbers are fuzzier because most tracking is pageviews or downloads, but roughly: Ubuntu remains the largest desktop Linux by install count, though its enthusiast mindshare has cratered since the Snap controversy. Linux Mint is consistently in DistroWatch’s top few and dominates the Windows-refugee segment. Fedora has grown as developers leave Ubuntu. Debian’s raw footprint is enormous in hidden form (it is the base of Ubuntu, Mint, Kali, dozens more) but its direct desktop count is smaller. Arch has 17–25% mindshare among technical users depending on how you count derivatives. Gentoo, Void, NixOS, and openSUSE Tumbleweed are each smaller but steady niches with strong retention.

A few specific data points worth knowing (treat all of these as directional, they come from different measurement methods with different biases):

• Stack Overflow’s 2024 Developer Survey (65,000+ respondents) has Ubuntu at roughly 27.7% of developer personal use and 27.7% professional use, with Debian around 9.8% personal and 9.1% professional. Other Linux distributions (Arch, Fedora, NixOS, Pop!_OS, etc.) together account for 17.6% personal / 16.7% professional.
• DistroWatch’s page-view tracker (which measures community interest, not actual installs) regularly has Linux Mint in the lead at roughly 2,400 daily hits, followed by MX Linux (~2,280), EndeavourOS (~1,640), and Manjaro (~1,400+). Note this under-represents enterprise distributions; RHEL, used by a majority of Fortune 500 companies, ranks in the 50s by page views.
• Enlyft’s corporate-adoption data puts Ubuntu at roughly 29% of Linux distribution market share across their tracked deployments.
• Canonical’s revenue grew from $175M in 2021 to $251M in 2023, a 43% increase, largely Ubuntu Pro and cloud licensing rather than desktop.
• Red Hat generates roughly $1.87 billion in quarterly revenue after the IBM acquisition, with about 67% share of the paid enterprise server market.
• Arch and its derivatives (EndeavourOS, CachyOS, Manjaro, Garuda, SteamOS) together capture roughly 47% of DistroWatch poll respondents when counted as a family. Among Linux kernel developers specifically, Fedora is the most common distribution at roughly 45%, followed by Arch at roughly 30%.

“Widely used” in this doc = anything on Debian / Ubuntu / Mint / Fedora / Arch / Manjaro-EndeavourOS / openSUSE / Gentoo / Void / NixOS / Devuan / Artix / Pop!_OS / Zorin / Bazzite / Alpine / Qubes / Tails / Whonix, plus the BSDs and a few niche principled entries.

One note on specialized distributions: Kali Linux, Parrot OS, BlackArch, REMnux, and similar distros exist but are not in this guide. They are penetration-testing or security-research tools, live-USB or VM images built to ship with hundreds of offensive-security utilities and with their hardening and defaults tuned for that work, not for daily use. If someone recommends Kali as your first Linux to switch to, that recommendation is wrong; use Mint or Debian. If you later become a security professional and need those tools, you already know where to find them.

The install-once-and-forget axis

One axis most distro comparisons miss: how much attention a system needs from you per month once it’s set up. This is orthogonal to stability and to package freshness.

Low-attention: Debian stable, Linux Mint, AlmaLinux, Rocky, Devuan, openSUSE Leap, FreeBSD. You install them and they update cleanly for years; the only reboots are kernel security updates. Debian stable is the canonical example, a Debian 12 install from 2023 will run through 2028 without breaking user-space.

Medium-attention: Fedora Workstation, openSUSE Tumbleweed, EndeavourOS, Pop!_OS, NixOS. You’ll touch them weekly or monthly. Fedora major upgrades every six months, Tumbleweed’s rolling updates, NixOS channel bumps. Nothing breaks often, but you’re in the loop.

High-attention: Arch, Manjaro, Void, Gentoo, Artix, Kali. The social contract is “read the news before you update.” Gentoo adds the compile-time axis. These reward attention and punish neglect; if you disappear for six months and run an update, you will spend an afternoon unwedging things.

The axis matters because the right answer depends on the machine. Daily-driver laptop for work → low-attention. Tinkering machine or homelab → medium or high. Don’t put a rolling release on your grandmother’s computer.

The systemd problem

For the first two decades of Linux, the program that started everything else at boot was a small thing called sysvinit: it read shell scripts, it launched daemons, it got out of the way. In 2010, a Red Hat engineer named Lennart Poettering released a replacement called systemd. Technically systemd starts faster and manages service dependencies better than sysvinit did. Culturally, systemd has become one of the most divisive projects in open-source history, for reasons that are now much clearer than they were in 2010.

The first objection was philosophical. Unix has a founding principle: each program should do one thing and do it well, and programs should be composed together with simple interfaces. Systemd does not follow this. It started as an init system and has grown to absorb logging (journald), network management (networkd), device management (udevd), user management (homed), DNS resolution (resolved), time synchronization (timesyncd), container management, and now user database schemas. If a system component touches systemd, it tends to become hard to replace, because systemd provides and consumes its own interfaces. This is the opposite of what Unix was.

The second objection is governance. Red Hat (now IBM) employs many of systemd’s core maintainers, and systemd became the default init on nearly every major Linux distribution (Debian, Ubuntu, Fedora, openSUSE, Arch) by a mix of technical argument and ecosystem momentum, over the objections of significant portions of those projects’ communities. Debian had a public, bitter vote about adopting systemd in 2014; the people who lost that vote left to create Devuan. The grievance wasn’t only technical. Systemd was one of the first major examples of a corporate-employed maintainer successfully imposing a new piece of critical infrastructure on the broader Linux ecosystem over community objections. Once in, it was hard to remove.

The third objection is recent and concrete. In March 2026, systemd merged pull request #40954, adding a birthDate field to its user-record schema[^systemd-pr]. The stated reason was compliance with new age-verification laws in California (AB-1043), Colorado (SB26-051), and Brazil (Lei 15.211/2025). The PR was submitted by a first-time contributor and received 37 negative reactions to 1 positive before a Microsoft employee merged it. When the community filed a revert PR the next day (#41179), Lennart Poettering closed and locked it without merging, saying the field was optional and enforced no policy[^systemd-revert]. The community’s objection was not the technical merit of an optional JSON field; it was the governance pattern. A critical Linux infrastructure component had its user-identity schema changed to accommodate state age-verification legislation, the change was approved by a corporate-employed maintainer against overwhelming community opposition, and dissent was closed and locked by the project’s founder. This is exactly what the Iron Law of Oligarchy looks like in software.

The corporate affiliations that made this possible are worth naming directly, because they’re all public record. Kinvolk GmbH, the German systemd-adjacent company, was acquired by Microsoft in April 2021. Lennart Poettering moved from Red Hat to Microsoft around July 2022. Christian Brauner, another core systemd maintainer, moved from Canonical to Microsoft around the same period. Amutable GmbH was incorporated in August 2025 by former Kinvolk / Microsoft personnel and announced publicly in January 2026, seven months after incorporation and roughly two months before the birthDate merge. The Sovereign Tech Agency (a German-government program that funds open-source infrastructure) has invested approximately EUR 855,000 into systemd across two funding rounds; the contractor-of-record is not publicly disclosed on the agency’s funding page, which is a matter of the agency’s choice rather than a conspiracy, but it does mean the money trail can’t be independently traced. All of the incorporation filings are retrievable from the German Handelsregister. None of this proves capture on its own. All of it is consistent with the pattern the Iron Law describes: systemd’s commercial gravity now sits with a Microsoft-adjacent cluster of personnel, and one of Linux’s most critical pieces of infrastructure has a corporate center that is no longer Red Hat.

The fourth objection came two days later. On 17 March 2026, Qualys disclosed CVE-2026-3888, a local privilege escalation in Ubuntu Desktop 24.04 and later[^cve-3888]. The bug arises from the interaction between Canonical’s snap-confine and systemd-tmpfiles, neither broken alone, but the combination lets an unprivileged local user gain root by manipulating the timing of systemd’s scheduled temp-file cleanup. CVSS 7.8, high severity. The exploit window is a 10-to-30-day delay, which is not safety; it’s a timer. This is the kind of failure mode that the “do one thing well” principle exists to prevent: when systemd is responsible for cleaning up temp files, and also indirectly setting up snap sandboxes, and also managing service schedules, the interactions between these responsibilities become the attack surface.

What happened after the investigation was published is the part that’s hardest to dismiss as coincidence. TBOTE (the project that compiled the corporate-affiliations record above) documented coordinated automated reconnaissance against its own site in the days after publication: roughly 1,285 requests from 70 IPs traced to Meta infrastructure, 1,659 requests from 18 IPs traced to Microsoft’s OpenAI crawlers, over 5,500 requests from 1,100+ IPs in a 72-hour window traced to Google Cloud scanner clusters, and probing by Palo Alto Networks’ Cortex Xpanse, an enterprise attack-surface-management product whose licensing runs into six figures per year. None of this is dispositive on its own; corporate infrastructure scrapes a lot of small websites, and a single project’s logs are the project’s own characterization rather than independently verified evidence. But the pattern of who specifically showed up to map a site that named them, with tools that cost real money, is the kind of empirical correlate the Iron Law predicts. Read TBOTE’s logs directly at tboteproject.com if you want to form your own view.

The practical takeaway: systemd is not going away, and on most distros you will use it. But the case for choosing a systemd-free distro (Devuan, Artix, Void, Gentoo with OpenRC, Alpine, Vendefoul Wolf) got materially stronger in March 2026. If you were on the fence, the birthDate merge and CVE-2026-3888 landing in the same week are good reasons to step off.

Community politics and the vibe test

The distros themselves don’t have politics. The communities, founders, and corporate backers do, and that shapes what features get prioritized, what governance disputes erupt, and how a project behaves under pressure. Roughly sorted right to left:

Right-coded: Gentoo, Void, Devuan, Artix, Alpine, Slackware, Vendefoul Wolf, OpenMandriva, Omarchy, GhostBSD, OpenBSD. These communities cluster around sovereignty, minimalism, anti-dependency posture, skepticism of corporate and political gravity. Gentoo attracts the purists; Void and Devuan are explicitly reactive against systemd (Red Hat’s piece of OS centralization); Alpine’s ethos is small-and-audited; Vendefoul Wolf ships with explicit anti-AI, anti-telemetry, anti-Wayland defaults. OpenMandriva is a French 1901 non-profit that has stayed structurally outside corporate orbit and is the most visible non-Mageia continuation of the Mandrake lineage; Omarchy is DHH’s opinionated Arch + Hyprland distribution and sits here by the cultural alignment of its maintainer. GhostBSD recently replaced X.Org with XLibre (the fork that split off from X.Org over governance) and is quietly one of the clearest “engineering over ideology” moves any desktop OS made in 2026. OpenBSD sits here by ethos more than politics, Theo de Raadt’s project is as far from Silicon Valley progressivism as any mainstream OS gets. These communities will leave you alone if you don’t bring politics into their issue trackers.

Center: Debian, Linux Mint, MX Linux, Arch Linux, EndeavourOS, openSUSE, Slackware-adjacent projects, FreeBSD. Technocratic-neutral. Debian has a formal democratic constitution and has resisted capture by any single direction; its one major recent political fight was the systemd vote in 2014. Mint is pragmatic and anti-Canonical-bloat. Arch ships software and documents it. openSUSE and FreeBSD have corporate backers but the communities are engineering-first.

Center-left: Ubuntu / Canonical, Pop!_OS / System76, Manjaro, Zorin. Silicon-Valley-adjacent defaults. Canonical is a British-headquartered corporate entity with standard corporate apparatus; the Snap controversy is a corporate enclosure move more than a political one, but the overall posture is progressive-corporate. System76 is a Denver hardware company whose politics track standard US tech-left. Zorin is a commercial shop with mainstream progressive branding.

Left-coded: Fedora / Red Hat / IBM, GNOME, NixOS, Tails, Trisquel, PureOS, Qubes, Whonix. Fedora is directly a Red Hat property, and Red Hat under IBM runs one of the most visible corporate DEI programs in open source. GNOME has been involved in some of the loudest code-of-conduct fights in desktop Linux over the last decade. NixOS had a major governance crisis in 2024 around military-adjacent contracts (Anduril) and CoC enforcement; the fallout produced the Lix fork and reshaped the NixOS Foundation. Tails is explicitly aligned with activist-left communities, their own documentation links to a tutorial called “Tails for Anarchists.” Trisquel and PureOS live inside the FSF’s political orbit. Qubes and Whonix are academic-privacy-left in community composition though generally lower-drama in operation.

If you want software whose community will leave you alone: Void, Devuan, Gentoo, Alpine, Debian, OpenBSD. If you want corporate-grade polish and don’t mind the cultural package: Fedora, Ubuntu, Pop!_OS. Everyone else is mostly just shipping an OS.

How the camps see each other

A lighter companion to the previous section. If you want to understand the culture of each camp, listening to what they say about each other is clarifying.

What Arch users say about Debian: “Your packages are ancient. You’re running software from two years ago and calling it stable.” And about the bureaucracy, it takes a formal governance process for a package to enter Debian proper, which Arch users see as sclerotic.

What Fedora users say about Debian: “At least we’re somewhat current. Debian stable is a museum.” And on release cadence: “‘Whenever it’s ready’ means never.” Plus a sharper point about technical leadership: Fedora ships new desktop infrastructure first (PipeWire, Wayland, Btrfs by default) while Debian inherits it years later.

What Debian users say back: with complete calm, “My server has been running for four years without a reinstall. Can you say the same?” They point out that Debian is the mother of a hundred distros, that Ubuntu exists because of Debian, that apt is the most copied package-management model in existence. They also note that Arch users spend more time fixing their system than using it, and Fedora users are essentially unpaid Red Hat beta testers. The Debian user’s energy: “We were here before you, we’ll be here after you, and we don’t care what you think.”

What Arch users say to Devuan users: respectful, “Good call ditching systemd, but why run a Debian fork when you could run something that doesn’t hold your hand at all? Come to Artix.” Weird mutual appreciation between the two contrarian camps.

What Fedora users say to Devuan users: dismissive, “Why are you clinging to sysvinit like it’s 2004? Systemd exists for a reason, just learn it.” Fedora is Red Hat’s proving ground, and systemd is a Red Hat project, so Fedora users tend to be true believers.

What Gentoo users say to Arch users: condescending, “You use Arch because you couldn’t setup Gentoo.” Gentoo’s identity is built around source compilation and USE-flag-level control; from that vantage, Arch’s binary-package convenience reads as a shortcut. The meme below is the canonical version of this dynamic.

None of this is meant to decide anything for you. It’s a sanity check: whatever distro you pick, here is approximately what the other camps will think about your choice.

GNOME’s political turn and the Code of Conduct asymmetry

GNOME has the highest “loudest CoC fights per capita” rate in desktop Linux, and the pattern that’s emerged across 2024–2026 is specific enough to name. Prominent GNOME maintainers, using either the project’s own blog network at blogs.gnome.org or personal accounts that publicly identify them as GNOME contributors, have repeatedly characterized people and companies they politically disagree with as “Nazis,” “fascists,” or “racists.” The GNOME Code of Conduct[^gnome-coc] explicitly forbids personal attacks, demeaning language, and “any other conduct which could reasonably be considered inappropriate in a professional setting.” No publicly disclosed CoC enforcement action has followed any of the incidents below.

The documented record:

• April 2025: Tobias Bernard, a GNOME design contributor, published “The Elephant in the Room” on blogs.gnome.org over the suspension of board member Sonny Piers. The post repeatedly closes with the line “Fuck Nazis, GNOME is Antifa.” A subsequent post from Allan Day (also blogs.gnome.org, also defending the GNOME Foundation’s handling of the case) treats this framing as unremarkable rather than flagging it.[^gnome-bernard][^gnome-day]
• July 2025: Jordan Petridis (GNOME release team member, runtime maintainer, GNOME Nightly maintainer, Newcomers experience lead, Podcasts maintainer per his own GNOME wiki page[^petridis-roles]) and Jeremy Bicha (Canonical engineer, Ubuntu/Debian GNOME packager) modify the XLibre wiki page to label the Xorg fork a “Nazi project” / “Nazi bar.” XLibre is the engineering fork of Xorg that GhostBSD shipped as default in its 26.1 release, a fork led by a developer they politically disagree with.[^xlibre-deface]
• October 2025: Framework Computer announces sponsorship of the Hyprland project and continues promoting Omarchy (DHH’s Arch + Hyprland distribution). The GNOME OS Team responds by stating that Framework “supports Fascist and Racist s***heads” and that GNOME “does not feel comfortable in further collaboration.” The GNOME Foundation reportedly opens lines to other projects considering disengaging from Framework over the same issue.[^gnome-framework]
• November 2025: Petridis publishes “DHH and Omarchy: Midlife crisis” on blogs.gnome.org, framing DHH’s politics as the “alt-right pipeline” and characterizing the Framework partnership as alignment with that pipeline.[^petridis-omarchy]
• April 2026: After Framework ships another pre-release Panther Lake laptop to DHH, Petridis posts on Mastodon: “Framework listens to their audience so much that they send another pre-release laptop to DHH. Enjoy the new Nazibook 13 pro.”[^petridis-nazibook]

Two patterns are worth pulling out. First, the Bernard and Petridis posts are hosted on blogs.gnome.org itself (the project’s own infrastructure) not on personal sites the project could disclaim. Allowing them there is an editorial choice. Second, GNOME’s CoC was the same instrument used to remove Sonny Piers from the project in 2024 over a private dispute the Foundation has refused to publicly detail. The CoC moves quickly when a board member becomes inconvenient and not at all when prominent maintainers publicly call paying customers and partner companies Nazis. That asymmetry is the actual signal.

This matters for choosing a desktop because GNOME ships as the default on Fedora Workstation, Ubuntu, Pop!_OS, Zorin OS, and Debian. If you pick GNOME you are picking, by gravitational rather than direct payment, the project that produces this output. KDE Plasma, XFCE, Cinnamon, MATE, and LXQt have not generated comparable patterns. If GNOME’s politics are not something you want to subsidize culturally, the desktop choice is where you can opt out without giving up the distro you otherwise want.

The XLibre / ArchWiki case (April 2026): a contrast

A useful contrast case landed in April 2026. On 15 April, ArchWiki administrator Alad deleted the XLibre project page; the next day, XLibre published a reflection post on X framing the action as part of an “abuse of CoCs as Codes of Censorship,” disclaiming xlibre.net as outdated, identifying the GitHub README as the project’s authoritative source, and announcing plans to “become louder” on CoC abuse, build a central place for third-party XLibre Arch packages, and possibly “liberate some information.”[^xlibre-arch-deletion][^xlibre-reflection] The framing wants to slot the deletion into the same bucket as the GNOME pattern above. It doesn’t fit, and the reasons it doesn’t fit are worth naming.

Arch’s stated basis was the Arch CoC’s “Respect” clause, which prohibits “maligning other FOSS projects or distributions, or any other operating systems and their users.”[^arch-coc] The cited content is real and still live: xlibre.net’s About page calls Xorg contributors “toxic elements” and “moles from BigTech,” and closes with the slogan “Together we’ll make X great again!” XLibre’s response argues that xlibre.net is outdated and that the GitHub README (from which the “moles” passage was removed in commit 4839966 on 25 July 2025) is the only authoritative source. The site is the project’s own domain, runs the project’s own copy, and is what a reader Googling “xlibre” lands on. Disclaiming a domain you control while leaving its content in place does not move the rhetoric to a different actor.

The other half of the post’s framing is that the Arch CoC is being “applied outside of Arch Linux.” Arch removing content from Arch’s own wiki is the CoC being applied inside Arch, hosting decisions on community-run infrastructure are exactly where CoCs operate. The page was not removed from XLibre’s GitHub. The xlibre-server package was not removed from the AUR. Arch chose not to host a documentation page about a project whose public-facing site contains the kind of language Arch’s rule was written to exclude.

This matters for the capture-risk frame because it cuts the other way from the GNOME case. The GNOME pattern is asymmetric application: the same CoC that removed Sonny Piers in 2024 has not moved on prominent maintainers calling paying customers and partner companies Nazis on the project’s own blog network. The Arch case is symmetric application: a stated rule against maligning other FOSS projects, applied to a project that publicly maligns another FOSS project on its own homepage. You can dislike either CoC regime on principle, but treating the two cases as instances of the same problem flattens a distinction the underlying record actually makes. XLibre is also separately a serious engineering project, the GhostBSD entry below shows an engineering-led BSD adopting it as default specifically over governance turbulence at X.Org, and that judgment stands on technical grounds independent of how the project’s public-facing rhetoric has been received elsewhere.

How different commentators sort distros

Different people filter distros through different lenses. Naming the lenses explicitly helps, because a recommendation makes more sense when you know what it’s optimizing for.

Capture-risk lens (this doc’s primary frame). Who owns this project’s governance, and what could change it? Corporate distros score worse because they can be acquired, monetized, or rebuilt around revenue (Red Hat → IBM, elementary OS’s funding struggles, Canonical’s Snap enclosure). Foundations and democratic projects score better (Debian’s constitution, OpenMandriva’s French 1901 non-profit, the BSDs’ permissively-licensed codebases maintained by independent foundations). The systemd governance story sits at the center of this lens because it shows the Iron Law at work inside infrastructure most users never see.

Project-culture lens (Bryan Lunduke and adjacent voices). Does project leadership publicly stake the project on identity-politics or DEI commitments that could override technical merit in hiring, moderation, or roadmap decisions? Lunduke’s August 2025 “non-woke software list” names OpenMandriva (his personal daily driver), GhostBSD, Omarchy, and Devuan as his picks on this axis, and he is critical of projects where he reads the leadership’s signaling the other way (Mozilla, NixOS after the 2024 governance fight, the recent Debian DPL election). Overlap with the capture-risk lens is real (both flag Mozilla, both flag corporate-IBM Red Hat) but the lenses diverge. Debian scores well on capture-risk (democratic, dispersed, cannot be bought) and ambiguous-to-critical on the project-culture lens depending on how recent governance fights read to you. Fedora scores badly on both.

Technical-pragmatism lens (Greg Kroah-Hartman, Linus Torvalds, most working kernel developers). Which distro lets me get work done with the least friction? Torvalds uses Fedora because it stays current and out of his way. Kroah-Hartman uses Arch for the same reason plus the wiki. This lens is indifferent to governance and culture; it only cares whether the distro ships what the user needs and stays out of the way.

Libre/freedom lens (FSF, Trisquel, PureOS, RMS). Does the distro ship only free software, and does it refuse non-free firmware even when that means hardware doesn’t work? A small but principled axis. Most readers will find it too strict for daily use; a few will find it the only serious axis.

These lenses are not a hierarchy. They’re filters. A reader whose primary concern is cultural drift in a project will weight Lunduke’s lens heavily; a reader whose primary concern is state or corporate capture will weight the capture-risk lens heavily; a reader whose job is kernel work will weight the pragmatism lens heavily. The distros that score well on more than one lens (Debian, Devuan, OpenBSD, GhostBSD, OpenMandriva) are the ones that tend to show up on multiple people’s lists for different reasons.

Sovereignty-minded figures and their distro choices

Who uses what, when the person has the technical literacy to choose deliberately. Treat specific-person endorsements as dated quickly and re-verify before leaning on any of them.

• Linus Torvalds: Fedora. His stated reasoning is roughly “I want a distribution to be easy to install, so that I can just get on with my life, which is mostly kernel.” He moved off openSUSE to Fedora in 2020 and has criticized Debian as “too technical.” Pragmatic-lens, indifferent to governance.
• Greg Kroah-Hartman: Arch Linux since 2019, including for his team’s cloud instances. His reasoning: “Their idea of a constantly rolling, forward-moving system is the way to go. It’s neutral, it’s community-based, it has everything I need.” He calls the Arch Wiki “amazing.” Keeps test environments on Fedora, Debian, and Gentoo for kernel QA.
• Lennart Poettering: spent fourteen years at Red Hat, joined Microsoft in 2022, now reportedly also involved in Amutable GmbH. His distro use has followed his employer.
• Miguel de Icaza: GNOME co-founder, switched to macOS in 2013 and has been a vocal critic of Linux desktop fragmentation ever since.
• Bryan Lunduke: OpenMandriva as daily driver (since late 2024 / early 2025). Uses a Mac for video production on The Lunduke Journal. Has vouched for GhostBSD, Omarchy, and Devuan as the alternatives on his project-culture axis.
• David Heinemeier Hansson (DHH): Omarchy, which he created and ships publicly via the Basecamp GitHub org. Arch + Hyprland + full-disk encryption, keyboard-driven, developer-focused.
• Luke Dashjr: Gentoo and custom builds, prioritizing maximum control for Bitcoin-related work.
• Adam Back: air-gapped systems, typically Debian-based or custom builds.

Among Linux kernel developers as a group, informal reporting puts Fedora at roughly 45%, Arch at roughly 30%, Ubuntu at roughly 15%, openSUSE at roughly 7%, and Gentoo at roughly 3%. Fedora dominates because Torvalds uses it and because Red Hat employs many kernel maintainers. Among serious Bitcoin operators running cold storage or signing hardware, informal community surveys suggest roughly Debian 35% / Arch 25% / Ubuntu 20% / specialized (Qubes, Tails) 15% / Mint 5%, treat those numbers as indicative, not measured.

A mental shorthand before the per-distro entries

If you want a light mnemonic to keep the families straight before we go distro by distro: the kernel is the bean, and every distro is a different way of preparing it. Debian is the dark roast everyone else blends from. Red Hat and Fedora are the corporate espresso. Arch is the pour-over you make yourself and then tell people about. Gentoo is roasting your own beans. Slackware is your grandfather’s percolator. Void is the small-farm import the serious coffee person prefers. Ubuntu and Mint are the friendly café with consistent service, the one most people start at. Windows is instant coffee.

The analogy breaks down where all analogies break down, but it’s enough to hold the shape of the landscape in your head before we go through it one entry at a time.

Debian

The universal operating system. If you don’t know what to pick long-term, this is usually the answer.

Short version for newcomers: Debian is the slow, stable, conservative grandparent of most Linux distros you’ve heard of. Ubuntu, Mint, Kali, MX, Raspberry Pi OS, and dozens of others are built on it. When you learn Debian, you understand the foundation every other apt-based distro is papering over.

Built on: its own packaging ecosystem (dpkg, apt), developed since 1993. Governed by a formal constitution and an elected project leader, not owned by any company.

License: FOSS, with non-free firmware for WiFi and GPU support available in a separate opt-in repository.

Good for: servers, long-lived desktops, anything you install once and ignore. The stable release cycle is roughly every two years with about five years of support (three years from the project plus two from LTS). The package repository is vast (~60,000 source packages) and quality is consistently high. A few things worth naming specifically that tend to get lost when people compare distros:

• apt and dpkg are mature and battle-tested. The packaging system has been refined for nearly thirty years. Dependency resolution works. Package metadata is reliable. Edge cases that bite users on newer or less-standardized package managers rarely bite you here.
• No corporate agenda can be bolted on. Red Hat / Fedora answer to IBM. Ubuntu answers to Canonical. openSUSE answers to SUSE. Debian is governed by its developers under the Debian Social Contract and the Debian Free Software Guidelines. No company can decide to change the license terms, sunset your version early for revenue reasons, or push a product you didn’t ask for. Debian cannot be acquired.
• Security response is serious. The Debian Security Team issues advisories and patches promptly. The frozen nature of stable dramatically shrinks the attack surface compared to rolling distros where new code constantly enters the system.
• Debian runs everywhere. The same distro runs on x86, ARM, RISC-V, MIPS, POWER, and more architectures than any other general-purpose Linux distribution. If you ever move to embedded systems, servers, or exotic hardware, your knowledge transfers directly.

Bad at: shipping recent desktop software. Debian stable is conservative by design, your GNOME or KDE will be 6–18 months behind upstream. Backports help; Flatpak fills the rest. For cutting-edge desktop software, run Debian testing/sid or use Flatpak aggressively. This is a deliberate trade, and for most people the stability is worth more than the freshness.

Trajectory: stable and essential. Debian 13 “Trixie” released August 2025, current stable through roughly 2028. Not going anywhere.

Use if: you want the universal default, don’t want to be on a corporate Linux, and value longevity over freshness.

Ubuntu

Canonical’s commercial Debian derivative. Historically the default beginner distro, now a cautionary tale. See What not to pick in 2026.

Short version for newcomers: Ubuntu is Debian repackaged by a company called Canonical, with some of their own additions on top. For a long time it was the friendliest way into Linux. Over the last few years it has gotten heavier, more corporate, and less aligned with what most users actually want, to the point that Linux Mint (which is built on Ubuntu) strips out Ubuntu’s additions on install.

Built on: Debian, plus Canonical’s own layers (Snap, Livepatch, Ubuntu Pro).

License: FOSS, including non-free firmware by default for broad hardware support. Some Canonical tooling is source-available but governed by contributor agreements that assign rights to Canonical.

Good for: matching production cloud environments, running on hyperscaler images, following most third-party Linux tutorials without translation. LTS releases (every two years) are supported ten years with Ubuntu Pro (free for personal use).

Bad at: being Debian. Snap is Canonical’s proprietary-backend package system that silently replaces apt install targets for certain popular packages (apt install chromium has installed a Snap for years). Snap performance is noticeably worse than native or Flatpak for many apps. Telemetry is on by default. GNOME is modified in ways that fight upstream. And in 2026, Ubuntu Desktop requires 6GB RAM and a 2GHz dual-core CPU, more than Windows 11 asks for[^ubuntu-req]. As a directional comparison on overall weight, Bryan Lunduke noted in April 2026 that the Ubuntu 24.04 LTS desktop ISO is roughly 6.3 GB while the Windows 11 ISO is roughly 5.8 GB: the free “lightweight alternative” now ships a larger installer than the proprietary OS it was pitched against. ISO size is not runtime weight, but the trend line on both axes (ISO and RAM) points the same direction. The March 2026 CVE-2026-3888 disclosure, which allows local privilege escalation via the Snap-systemd-tmpfiles interaction in default installs, is the most visible recent symptom of that architecture[^cve-3888].

Trajectory: still the largest deployed desktop Linux by install count but visibly losing enthusiast mindshare to Mint, Fedora, and Arch.

Use if: you need Ubuntu specifically (a work requirement, a vendor only ships .deb packages for Ubuntu) and can tolerate the direction. Otherwise use Debian for desktop or server.

Linux Mint

The most Windows-like Linux landing, and for years the reflexive recommendation for non-technical friends leaving Windows. Cinnamon desktop by default, with XFCE and MATE editions. Based on Ubuntu LTS but with Snap completely removed and replaced with Flatpak. This guide no longer makes it the default first install (Debian gets that; see the TL;DR and Decision shortcut), and the caveats below are why.

Short version for newcomers: Mint is what happens when a small team takes Ubuntu, strips out the parts of Ubuntu they don’t like, and ships a friendly, Windows-looking desktop on top. For someone leaving Windows today, Mint is the easiest landing. You’ll be productive in an hour.

Built on: Ubuntu LTS, with an insurance-policy branch (LMDE, Linux Mint Debian Edition) that builds directly on Debian instead of Ubuntu.

License: FOSS, including non-free firmware by default for broad hardware support.

Good for: people migrating from Windows. Cinnamon is the closest Linux comes to a traditional Windows layout. The update manager is unintimidating, the community forum is friendly, and the project’s identity is partly built on saying no to Canonical’s Snap push.

Bad at: being cutting edge. You’re on an Ubuntu LTS base, so packages are conservative. Not ideal if you need the latest developer toolchains or very recent hardware support.

Flags (read this if you install Mint): An independent Wireshark-based analysis in February 2025 found no OS-level telemetry from Mint, no analytics, no Canonical callbacks, no hidden logs beyond basic update-server contact[^mint-telemetry]. That’s the good news. The less-good news: on 22 December 2025, a bug report filed on the official linuxmint/cinnamon GitHub repository (issue #13298) documents the Cinnamon process on Mint 22.2 leaking memory from a clean 3GB post-boot footprint up to 70% of 32GB RAM over about four hours of normal use, forcing the user to run cinnamon --replace twice a day as a workaround[^mint-leak]. As of this writing (four months later) the issue is still open, still unassigned, and labeled only as a bug. Multiple other users have reported similar symptoms across forum threads during the same window. If you installed Mint 22.2 during the Windows 10 migration wave and felt your machine getting slower over the winter, you were probably not imagining it. Either a cross-machine memory leak exists in shipped Cinnamon and the Mint team has not addressed it in four months, or something else in the 22.2 release is regressing on real workloads. Either possibility matters. The structural point, separate from this specific bug, is that Mint has become the most-recommended Windows-escape distro on the internet, which makes it the most obvious capture target for anyone who wants to bend a large downstream user base. Mint is currently clean, but a shipped memory-leak regression left unaddressed for months is a quality-control signal, not just one bug, and a project’s quality-control slipping while its install base balloons is exactly when to be cautious rather than to make it the default. That is why this guide now points first-time migrants to Debian and keeps Mint as the choice for readers who weight Windows visual familiarity above everything else. Watch the project the way you should watch anything that gets that popular.

Trajectory: strong growth post-Windows-10-EOL. The project has publicly said an LMDE-only future is possible if Canonical ever poisons Ubuntu beyond the point where Mint can work around it.

Use if: Windows visual familiarity is your single overriding priority and you want the fastest cosmetic match with no homework. Otherwise start on Debian, the default this guide now recommends for migrants. If you do start on Mint, plan to move to Debian or Devuan in 6–12 months as you get comfortable.

MX Linux

Debian-based, developed jointly by the MX Linux team and the antiX community since 2014. Consistently top-three on Distrowatch’s most-visited-distros listing for the past five years. Ships sysvinit by default with optional systemd available at boot, meaning the same distro can run either init system depending on what you pick at the boot menu. That property alone makes MX interesting on a sovereignty axis: a Debian-based distro that does not force the systemd choice on you.

Built on: Debian Stable base plus MX-specific repositories that add updated packages where the Debian Stable version is too old to be practical (Firefox, kernel, some user-facing apps). The default desktop is XFCE; KDE Plasma and Fluxbox editions also exist.

License: FOSS, same disposition as Debian Stable (which it inherits from). Non-free firmware is enabled by default for hardware support.

Good for: a working desktop that’s faster than Mint on older hardware, more current than pure Debian Stable, and gives you the sysvinit option without leaving the Debian ecosystem. The MX-tools suite (a collection of graphical utilities for system administration, backup, package management, kernel selection, snapshot creation) is genuinely useful and one of the project’s distinctive features.

Bad at: nothing in particular for general desktop use. The MX-specific repositories are an additional trust surface compared to pure Debian; not a deal-breaker, but worth knowing.

Trajectory: stable, growing slowly. The combination of Debian-base plus sysvinit-by-default plus active maintenance is rare enough to keep the project alive without needing to chase trends.

Use if: you want a Debian-based desktop that doesn’t force systemd, you want better-than-Stable package freshness without going to Testing, or you want MX-tools’ graphical-administration suite. Acts as a less ideologically-loaded entry into the no-systemd world than Devuan or Artix.

Fedora

Red Hat’s community-facing distribution, upstream of RHEL. Cutting-edge but carefully tested. GNOME by default; KDE Plasma, XFCE, and others are first-class spins.

Short version for newcomers: Fedora is the Linux that developers and Mac refugees tend to end up on. It gets new features first (Wayland, Pipewire, systemd-homed, btrfs) and is more polished out of the box than most alternatives. It is also directly owned by Red Hat, which is owned by IBM, which you may or may not care about.

Built on: its own RPM-based packaging, dnf.

License: FOSS, with historically aggressive patent-avoidance. Fedora won’t ship MP3 support or certain codecs until their patents expire; RPM Fusion is the third-party repo for the things Fedora won’t ship. Non-free firmware for hardware support is available but not enabled by default.

Good for: developers, GNOME fans, anyone who wants recent packages with real QA. Fedora 43 released late 2025; Fedora 44 expected April–May 2026. Fedora Atomic variants (Silverblue, Kinoite, and the Bazzite-adjacent stack) are the interesting frontier, these use the immutable-base-filesystem model described in the glossary.

Bad at: long-term stability. Every six months you do a major-version upgrade; each release is supported about 13 months. Not for set-and-forget machines. And the upstream parent is IBM, which comes with the CentOS Stream controversy, RHEL source-access restrictions, and the general direction Red Hat has taken post-acquisition.

Trajectory: rising sharply as developers leave Ubuntu.

Use if: you want a Mac-like coherent desktop with current software and you’re willing to upgrade annually.

Arch Linux

The hacker’s distribution. Minimal by default, rolling release (meaning you get new versions of each package as soon as upstream releases them, with no fixed release cycle), you build the system up from a working shell. The Arch Wiki is the best piece of Linux documentation in existence and is used as a reference by users of every other distro.

Short version for newcomers: Not for newcomers. Come back after six months on Mint or Debian. Then the Wiki will reward you.

Built on: pacman (its package manager), plus the AUR, a community-run repository of build scripts that covers roughly 100,000 packages. The AUR is one of the main reasons people use Arch, almost any piece of software you can think of is one command away.

License: FOSS. Non-free firmware is available via the linux-firmware package, which most users install.

Good for: people who want to understand their system. You choose every component, init, bootloader, desktop, everything. Rolling release puts you within days of upstream for kernel, Mesa, and desktop versions. Greg Kroah-Hartman (the stable kernel maintainer) switched his team to Arch for the rolling model and the wiki.

Bad at: being forgiving. You read the news before updating or you occasionally break things. Not a good fit on a machine you rely on but don’t want to maintain.

Trajectory: stable and central. The Arch family (Arch + EndeavourOS + CachyOS + Manjaro + Garuda + SteamOS) is probably the second-largest Linux family after Debian/Ubuntu.

The “customize Arch once and use it forever” argument. A strategy experienced users sometimes propose: install Arch, customize it exactly how you want, write a script that rebuilds that setup from scratch, and then run it on all your machines and just maintain it with minor tweaks. It’s a legitimate strategy and plenty of people do it. The honest catch is that your rebuild script can’t freeze upstream. Rolling release means you’re implicitly trusting that every update to every package in the chain plays nicely together, forever. Sometimes it doesn’t, a mesa update breaks Wayland, a kernel update and a GPU driver update land a week apart and conflict, systemd changes something subtle. You can pin packages or run a partial local mirror to truly lock things, at which point you’re doing more work than Debian stable would have required. Plenty of Arch users run this pattern successfully; just know that the “set it up once and forget it” promise is only honored if breakage is rare enough in your specific setup to ignore, which is a bet you can’t make in advance.

Use if: you want bleeding edge and you’re OK reading the Wiki regularly.

EndeavourOS, CachyOS, Manjaro (Arch derivatives)

Three different takes on “Arch, but with an installer.”

EndeavourOS: the faithful one. Arch plus a graphical installer, a few sane defaults, and a friendly forum. Uses Arch repositories directly (so you get Arch packages at the same time Arch users do) and full AUR access. Once installed it behaves like Arch.

CachyOS: Arch tuned for performance. Custom kernel with scheduler tweaks, binaries compiled specifically for modern CPU instruction sets (x86-64-v3 or v4), gaming-optimized defaults. Measurably faster than stock Arch on recent hardware. Growing fast in early 2026.

Manjaro: do not use. Ships its own delayed repo (Arch packages held back ~2 weeks) that regularly causes breakage with the AUR (because the AUR expects current Arch, not two-week-old Arch), and the team has shipped expired SSL certs on their own website more than once. Strictly worse than both Arch and EndeavourOS.

Use if: EndeavourOS for vanilla Arch with install ergonomics; CachyOS for gaming or performance.

Omarchy

Arch + Hyprland, preconfigured and opinionated. DHH’s (David Heinemeier Hansson, of Rails and Basecamp) answer to “I want Arch’s power but I don’t want to spend three days on dotfiles.” Maintained under the Basecamp organization on GitHub.

Short version for newcomers: Not a newcomer distro. Omarchy is for someone who already knows what Arch is and wants DHH’s taste applied to it out of the box. If you’re coming from Windows, Mint is your answer, not this.

Built on: Arch Linux. Installs via a dedicated ISO (recommended) or a one-line script on top of a bare Arch install. Uses Hyprland (a tiling Wayland compositor), btrfs, and mandatory LUKS full-disk encryption. Ships Neovim, tmux, Alacritty, Chromium, a curated set of developer tools, and a keyboard-first workflow (no display manager, no login screen after disk-decrypt).

License: FOSS. Arch repos plus a small Omarchy repo for the opinionated pieces.

Good for: developers who want a tiling-window-manager workflow with good defaults. Terminal colors, Neovim theming, Waybar, Hyprlock, and the rest all move together when you switch theme, cohesive in a way a hand-assembled setup usually isn’t. Curated themes include Catppuccin, Gruvbox, Tokyo Night, Everforest, Flexoki Light. The Omarchy Menu (Super + Alt + Space) is the main control surface; almost everything happens via keyboard.

Bad at: being flexible about taste. You are installing DHH’s preferences, the fonts, the color palettes, the editor, the window-manager keybindings, the software choices. Everything is overridable (they’re still config files in ~/.config) but you’re starting from someone else’s aesthetic, not a blank slate. Also: no mouse-first workflow (you literally cannot do much without keyboard on first boot), no support for legacy BIOS (UEFI only), and the installer wipes the target drive, dual-boot is awkward and requires two physical disks.

Trajectory: actively developed, growing community, DHH is personally invested. Current releases in the 3.x series as of early 2026.

Use if: you’re already Linux-literate, you want a tiling-WM developer setup, and you want someone else’s good decisions as the starting point. If your reaction to “mandatory full-disk encryption, keyboard-only, Hyprland-first” is “yes, finally,” you’re the audience.

Devuan

Debian minus systemd. Same repositories, same packaging, same support cycle; just sysvinit / OpenRC / runit for init, and eudev for device management.

Short version for newcomers: Devuan is what you install after you’ve spent a year on Mint or Debian and decided systemd isn’t for you. See The systemd problem.

Built on: Debian.

License: FOSS, with the same non-free firmware repository structure as Debian.

Good for: anyone who wants Debian’s ecosystem and stability without the systemd dependency chain. If you know Debian, you know Devuan. The delta is intentional and small.

Bad at: cutting-edge packages. Tracks Debian stable. A small maintainer team means security patches occasionally lag upstream by a day or two.

Trajectory: steady. Devuan 6 “Excalibur” (tracking Debian 13 Trixie) released 2025. The project has delivered on time for every Debian stable release for a decade.

Use if: you want Debian without systemd and you want apt. Lowest-friction non-systemd path for a Debian user.

Artix

Arch minus systemd. Supports OpenRC, runit, s6, dinit. Shares Arch’s repositories and AUR through a compatibility layer.

Built on: Arch Linux.

License: FOSS, same model as Arch.

Good for: people who want Arch’s bleeding edge and the AUR without systemd.

Bad at: being as battle-tested as Arch. Fewer users means fewer people have hit and fixed the edge cases. When an AUR package assumes systemd is present, you’re on your own.

Trajectory: small but healthy.

Use if: Arch, but no systemd.

Void Linux

Independent from the ground up. Not based on anything. Its own package manager (xbps), its own init (runit), its own repos, and a choice between glibc and musl C libraries. Rolling release.

Short version for newcomers: Not for newcomers, but worth knowing about. Void is the answer to “what if we built a rolling Linux from scratch, kept it lean, and refused corporate sponsorship?” Small community, small repo, very fast.

Built on: itself.

License: FOSS, with non-free firmware available in a separate repository.

Good for: people who want a lean, fast, opinionated rolling distro with no corporate owner, no systemd, no legacy baggage. runit is among the simplest init systems in existence; xbps is noticeably faster than apt or pacman; boot times are short. musl is a real working path if you want a smaller, stricter libc (musl is an alternative to glibc, the GNU C library almost everything else uses, musl is smaller and stricter but some proprietary software expects glibc and won’t run on it).

Bad at: ecosystem size. Fewer packages than Arch or Debian; the xbps-src templates are smaller than the AUR. Small community is also its charm.

Flag worth naming: the 2018–2019 maintainer-absence scare. Void’s original lead developer, Juan RP (xtraeme), disappeared from the project for an extended period in 2018 without warning, no commits, no responses, no handover. For several months the domain registration and infrastructure hung in limbo. The rest of the contributor team eventually organized, moved to a new domain (voidlinux.org instead of the original), and distributed the governance so no single maintainer absence could cause the same problem again. The project recovered, kept shipping, and is in good shape in 2026. But the episode is the single clearest bus-factor warning in the independent-distro space, and the lesson stuck: even a well-run small project can have one person whose disappearance freezes the whole thing. Void’s governance is more distributed now, but the user base is still small, and “what happens if the maintainers lose interest” is a real question for any independent distro.

Trajectory: slow, steady, healthy. Quiet in the way a good distro should be.

Use if: you want something independent, lean, non-systemd, and opinionated. If someone asked me “what’s the distro closest to the spirit of this document,” this is it.

Gentoo

Source-based. You compile everything from Portage, with fine-grained USE flags controlling which features get compiled into each package. OpenRC by default; systemd also supported; rolling release; excellent handbook.

Short version for newcomers: Gentoo is the Linux people run when they want to understand every part of their system and don’t mind waiting for it to compile. Not a first distro, but probably the best learning distro once you’re ready.

Built on: its own build system and ports-style tree.

License: FOSS. Non-free firmware is available if enabled via build flags.

Good for: total control. A full browser compile takes hours; a kernel compile is normal. In exchange you get a system with exactly the features you enabled, compiled for your CPU. ChromeOS is Gentoo-derived.

Bad at: fast iteration. Changing a USE flag globally rebuilds everything that depends on it. World updates can run for hours.

Trajectory: stable, small, beloved. Healthy long-term niche.

Rolling done right. Gentoo’s init story is one of the cleanest in mainstream Linux. OpenRC is the default, systemd is supported but never imposed, and the choice is a profile selection at install time rather than a compatibility hack grafted on after the fact. USE=-systemd at the build level keeps systemd’s daemons from being compiled into anything that doesn’t strictly need them, so even on a hybrid system the surface area stays small. Compare to Arch, where systemd is the default and the assumption: shipping the same package set without it requires a parallel project (Artix), because Arch’s tooling and many AUR packages are written assuming systemd is present. Both are “rolling,” but the rolling-distro promise (upstream-fast updates with full control over what your system actually runs) is one Gentoo keeps and Arch only partially does. If init choice and build-level component control matter to you, Gentoo is the only mainstream rolling distro that treats them as first-class.

Use if: you want to understand your system deeply, you have time to compile, and you value control over convenience.

NixOS

Declarative system configuration. Your entire system (installed packages, services, users, network, firewall, cron jobs) lives in a single configuration.nix file. Rebuilds produce a new generation you can roll back to.

Built on: the Nix language and package store.

License: FOSS. Non-free firmware and software available via the unfree attribute flag.

Good for: reproducible system configuration, dev environments, anyone who wants infrastructure-as-code for their own machine. Nixpkgs is among the largest package repositories in existence (100,000+). Atomic rollbacks are genuinely useful.

Bad at: being learnable in a weekend. The Nix language is its own thing, documentation has been a long-standing weak spot, and the mental model is different from every other distro.

Flags: significant governance fights in 2024 around the NixOS Foundation, military contracts (Anduril specifically), and CoC enforcement; long-time maintainers left and forked as Lix. The foundation has restructured but the culture isn’t fully settled.

Trajectory: powerful, interesting, mid-reset. Evaluate the Lix fork if governance matters to you.

Use if: reproducibility is your top priority and you’re willing to learn a new language for it.

Slackware

The oldest surviving Linux distribution still actively developed. Released in July 1993 by Patrick Volkerding, who remains the sole maintainer. Did not adopt systemd, does not plan to, never will. Used by the contributor who taught Linus Torvalds about Linux. Distinct enough that almost nothing else looks like it.

Built on: its own package format (.tgz / .txz tarballs) and pkgtool for management. No automatic dependency resolution by default, you read the documentation, install what’s needed, and the system gets out of your way. Third-party tools like slackpkg and slackbuilds.org add dependency resolution if you want it.

License: FOSS. Conservative on accepting new code; the project’s stated value is stability and predictability over feature pace.

Good for: a sysvinit-style boot (actually BSD-style init scripts, predating sysvinit’s later refinements), no telemetry, no scheduled upgrades, no community drama because the project is run by one person who has zero interest in being captured by anything. Slackware 15.0 released February 2022; Slackware-current is the perpetual rolling beta where new packages land. Stable releases come “when ready,” which has historically meant 3-6 years between major versions.

Bad at: hand-holding. Slackware assumes you can read documentation and configure things yourself. New users typically struggle with the no-dependency-resolution default; tools like slackpkg reduce the friction. The wider ecosystem of “official Slackware spins” is non-existent, there’s just Slackware, full stop.

Trajectory: stable in the literal sense. The project has outlived most of its peers and shows no sign of changing course.

Use if: you want the oldest, most conservative, most stable Linux project, run by one person who doesn’t take corporate funding and doesn’t take input from culture war factions of any political direction. Niche but real. Not recommended for newcomers.

openSUSE Tumbleweed and Leap

SUSE’s community distribution in two flavors: Tumbleweed is rolling; Leap is periodic-release aligned with SUSE Linux Enterprise.

Built on: SUSE’s packaging (RPM, zypper, YaST).

License: FOSS, with non-free firmware and codecs available in opt-in repositories.

Good for: Tumbleweed is the best-QA’d rolling release available, uses openQA, an automated testing harness, so packages don’t land on users’ machines until a large matrix of installation and upgrade scenarios has passed. Leap is conservative enterprise-style. YaST is the most polished graphical system-administration tool in any major distro. KDE Plasma is first-class. Btrfs (a modern filesystem) with snapshots by default, so bad-update rollbacks are trivial.

Bad at: mindshare in the US. More popular in Europe, so English-language tutorials often default to Ubuntu or Fedora. SUSE’s corporate ownership has changed hands multiple times.

Trajectory: stable and underrated.

Use if: you want a rolling release that has actually been tested before it hits you.

OpenMandriva

The community continuation of Mandrake / Mandriva, maintained by the OpenMandriva Association, a non-profit established 12 December 2012 under French 1901 associations law. RPM-family, KDE Plasma by default, Clang-built, community-governed. Structurally unusual in that it cannot be acquired: the Association has a member assembly, an elected council, and published bylaws that include dissolution rights. Bryan Lunduke runs it as his personal daily driver and it’s what brought OpenMandriva into wider view on the project-culture axis in 2025.

Short version for newcomers: An option you probably haven’t heard of unless a specific commentator sent you. Possible but not the first Linux to try if you’re coming straight from Windows, the community is smaller, documentation is thinner, and Mint will be easier for your first six months. Worth knowing exists for when you want a non-corporate RPM alternative to Fedora.

Built on: its own RPM-based packaging, DNF package manager (same family as Fedora, openSUSE, Mageia). Built with Clang rather than GCC, the first major desktop Linux distribution to make that switch, in 2016. Ships x86_64 builds plus separate znver1-optimized builds for AMD Zen. Two release channels you must not mix: ROME (the rolling release, for individual users, current iteration published 11 December 2024) and Rock / OMLx 6.0 (the fixed point release, codename Vanadium, published 20 April 2025). Rock 6.0 ships KDE Plasma 6 with X11 or Wayland sessions and alternate LXQt, GNOME, XFCE, and COSMIC 1.0 alpha spins.

License: FOSS. Non-free firmware is available.

Good for: someone who wants an RPM-family distro without Fedora’s IBM-Red Hat parentage or openSUSE’s SUSE-corporate layer. KDE Plasma is first-class and feels closer to a Windows-like layout than GNOME. Proton and Proton Experimental are available directly from the repos, which means Steam / Windows-game compatibility is an dnf install away without adding third-party repos. Governance is structurally clean: French 1901 non-profit, published council, treasurer and bureau listed publicly, annual activity-and-accounts reports required.

Bad at: scale. DistroWatch hit-rank bounces around the 30s–40s, roughly an order of magnitude less traffic than Mint, Debian, or Fedora. Smaller forum, fewer tutorials, thinner third-party package coverage. Uses systemd as default init, if your primary objection is systemd, OpenMandriva doesn’t solve it (a runit package exists in repos but swapping init is DIY and unsupported). The ROSA-lineage part of its ancestry connects back to a Russian Mandriva fork, which is worth naming given the supply-chain frame elsewhere in this guide; the current Association is French and governed independently, but the code history is what it is. Not a gaming-specialized distro in the Bazzite/CachyOS sense, Proton in the repo is a package, not a tuned stack.

Trajectory: steady. Rock 6.0 released on schedule in April 2025. ROME updates continuously. The Association has not had a visible governance crisis. It remains small.

Use if: you want a non-corporate RPM-family distro with a Windows-like KDE desktop, you are OK being on a smaller distro with thinner documentation, and either governance posture or Lunduke’s endorsement is what brought you here.

Pop!_OS

System76’s Ubuntu-based distribution. Currently shipping with a heavily-modified GNOME; the COSMIC desktop (a Rust-based rewrite) is in alpha/beta as of early 2026.

Built on: Ubuntu LTS.

License: FOSS, including non-free firmware and NVIDIA drivers by default.

Good for: System76 hardware, NVIDIA GPU users (Pop ships NVIDIA drivers out of the box), tiling-friendly workflow via GNOME extensions.

Bad at: timing. The COSMIC transition has been slow; shipping Pop is in a holding pattern.

Trajectory: in transition. If COSMIC lands well, Pop becomes interesting again.

Use if: you bought a System76 machine, or you’re on NVIDIA and want a curated Ubuntu derivative.

Zorin OS

Commercial Windows-refugee distribution. Heavy migration tooling, Windows-like default layouts. Free core; Zorin Pro adds more layouts and preinstalled apps.

Built on: Ubuntu LTS.

License: FOSS core, including non-free firmware for broad hardware support; Zorin Pro is a paid tier with additional layouts and apps but no core lock-in.

Good for: people nervous about leaving Windows who want maximum hand-holding. Zorin 18 shipped October 14, 2025 (the same day as Windows 10 EOL) with explicit Windows-installer detection, OneDrive integration, and Windows-desktop layouts. The project reported roughly 100,000 downloads in the first two days, most from Windows-origin machines.

Bad at: being interesting. Under the hood it’s Ubuntu. The polish is genuine but you’re paying for curation rather than anything architecturally different.

Trajectory: rising on the Windows-10-EOL wave.

Use if: you need to migrate a non-technical family member and want the least-friction onboarding. Mint is the free alternative.

Bazzite and SteamOS (gaming)

Bazzite: community-maintained Fedora Atomic variant with Steam, Proton (the Valve tool that lets Windows games run on Linux), gamescope, and HDR/VRR preconfigured. Desktop and handheld variants. Uses the immutable-base-filesystem model, the core OS is read-only and updates atomically, applications live in Flatpak or Distrobox containers. Practically: you can’t easily dnf install a system package, but you also can’t break your system by accident, and rollbacks are trivial.

SteamOS: Valve’s own Arch-based distribution, officially shipping on the Steam Deck with broader desktop release expected. Bazzite is the practical choice for non-Deck hardware today.

License: FOSS, including non-free firmware, NVIDIA drivers, and codec support needed for gaming.

Good for: gaming hardware you want to think about as little as possible. Proton runs a large majority of top Steam titles as of 2026. HDR and VRR work on supported hardware.

Bad at: being your main development machine. Immutable base means traditional apt install / dnf install doesn’t apply; you work through Distrobox, Flatpak, or Toolbx.

Use if: you want a gaming PC that isn’t Windows 11.

Alpine

Minimalist, security-focused, musl-based. Famous as the default base for Docker containers (~8MB image vs 200MB+ for Ubuntu) but also usable as a server or desktop OS.

Built on: musl libc, BusyBox, OpenRC, its own package manager (apk).

License: FOSS. Proprietary firmware blobs are typically not installed by default.

Good for: containers, embedded, small self-hosted servers, security-minded users who want a small attack surface. Position-Independent Executables by default, stack-smashing protection on, minimal base install.

Bad at: desktop use by default. Because Alpine uses musl instead of glibc, some proprietary software that assumes glibc won’t work, some games with anti-cheat, some enterprise software, a few Steam titles.

Use if: you run containers or small servers and want the smallest most-audited base.

High-security operating systems: how to pick

The next several entries (Qubes, Whonix, Tails, Kicksecure, Vendefoul Wolf, Trisquel/PureOS) are operating systems built specifically to defend against serious threat models. Most people don’t need them; the entries above (Debian, Devuan, Fedora, Mint) plus the workstation hardening covered separately get you to a perfectly defensible setup for normal use. If you’re considering one of these high-security distros, you’re answering a specific question: “what’s the right system for this scenario?”

The six scenarios that actually drive these choices, and the picks that map to each:

A. One-off amnesic session for sensitive work. Booting from a USB stick, doing one specific thing (filing a report, accessing a sensitive account, communicating once), and rebooting back to your normal life with nothing persisted on disk. The system must leave no forensic trace on the host hardware. Pick: Tails. Designed for exactly this. Boots from USB, runs entirely in RAM, all network traffic forced through Tor by default, optional encrypted persistent volume only if you opt in.

B. Persistent Tor-anonymous workspace. You’re working on a project (research, journalism, contributing to a sensitive codebase) over weeks or months under a pseudonym, and every connection it makes should be over Tor with no way for a misconfigured application to leak your real IP. Pick: Whonix, ideally on Qubes. The Whonix-Gateway VM forces all traffic from the Workstation through Tor at the network layer; even a compromised application inside the Workstation cannot leak your real IP because it has no non-Tor route. Qubes-Whonix adds whole-machine compartmentalization underneath.

C. Compartmentalized daily driver. Your main computer is used for work, banking, personal stuff, untrusted browsing, code from random repos. You want an exploit in your browser not to compromise your banking session. Pick: Qubes OS. Each role runs in its own VM. The browser qube getting owned doesn’t touch the work qube. Requires real hardware (16GB+ RAM, virtualization extensions, whitelisted laptop) and a learning curve, but no other OS makes per-task isolation this thorough.

D. Hostile-network field work. You’re traveling and connecting through coffee-shop Wi-Fi, hotel networks, conference networks, foreign infrastructure that may be hostile. Pick: depends on whether you’re carrying a hardened daily-driver laptop or a fresh machine. If carrying your hardened Devuan workstation, add a Whonix-Gateway VM in front of its networking for the duration of the trip (forces Tor on top of existing hardening). If carrying a fresh laptop and want minimal forensic footprint, Tails on USB. If carrying Qubes hardware, Qubes with sys-net-vpn for the entire trip.

E. Air-gapped signing or cold storage. A machine that holds high-value cryptographic material (GPG primary key, Bitcoin wallet keys, code-signing keys) and never connects to the internet. Pick: hardened Devuan or Debian with networking physically removed (no Wi-Fi card, no Ethernet cable), or Tails for sessions you don’t want persisted, or OpenBSD when OS monoculture is itself part of the threat model. The OS choice matters less than the air-gap discipline. OpenBSD as a signing-machine choice has a specific argument behind it: it’s not derived from anything else this doc covers, its codebase is small enough that one person could in principle audit it, and its developers prioritize correctness over feature pace in a way that matches the air-gapped-signing role. Full OpenBSD coverage is in the BSDs section below.

F. Hardened daily driver that isn’t a Tor target. You want a normal daily-use desktop with strong defaults (disk encryption, hardened kernel, sensible permissions, integrity monitoring) without the cost of running everything through Tor. Pick: hardened Devuan, Kicksecure, or Vendefoul Wolf. All three reach roughly the same end state; the difference is whether you want to apply the hardening manually (Devuan + the workstation hardening guide) or inherit it from the distro (Kicksecure, which is Debian-based with hardening defaults; or Vendefoul Wolf, which is Devuan-based with similar defaults).

Layering: what stacks with what

Qubes-Whonix is the strongest desktop setup available. Whonix-Workstation templates run as qubes; Whonix-Gateway runs as a separate qube; other qubes (work, personal, banking) use the Gateway as their network proxy or have direct internet depending on role. The thing Qubes-Whonix is genuinely best at: a browser exploit in the Whonix-Workstation qube cannot leak your real IP (Whonix-Gateway blocks the route) AND cannot reach your work qube’s files (Qubes blocks the cross-qube access).

Whonix on a non-Qubes Linux host is the fallback when Qubes hardware isn’t available. KVM or VirtualBox on a hardened Devuan or Debian host. Worse than Qubes-Whonix because the host OS is now the trust boundary; a host compromise compromises the Whonix session. Better than running anonymous work directly on the host.

Tails on a personal laptop versus Tails on a borrowed machine is a meaningful distinction. The personal laptop has consistent hardware identifiers (MAC addresses, hardware serial numbers) that, while masked by Tails defaults, persist across sessions. A borrowed machine breaks that linkage; an internet café or library terminal that you don’t return to is the strongest version of “amnesic session.”

Hardened Devuan plus Whonix-Gateway is the lower-commitment version of compartmentalized + Tor-forced. Your daily driver runs on Devuan with the standard hardening; a Whonix-Gateway VM exists on the same host; specific applications (a separate Firefox profile, a specific user account, a Qubes-style “anon qube” that’s actually just a separate VM) route through the Gateway. Most work runs on the direct-internet Devuan host with the host’s hardening; the Tor-routed activity uses the specific Gateway-routed path.

Heads or coreboot underneath any of the above. The firmware layer (BIOS / UEFI) is below the OS, and every OS on this page assumes the firmware loading it is benign. If your threat model includes physical access to your hardware or supply-chain interception, replacing the proprietary firmware with Heads (an open coreboot payload that measures and verifies the boot chain) closes the layer everything else assumes. Hardware-specific; supported laptops include the ThinkPad X220/X230/T420/T430 lineage, Purism Librem, System76 with open firmware, and Dasharo-supported boards.

Hardware requirements at a glance

• Tails: any laptop with 2GB+ RAM that boots from USB. Forgiving.
• Whonix on Qubes: Qubes hardware requirements (16GB+ RAM, virtualization extensions, whitelisted laptop). Tight.
• Whonix on Linux host: any modern laptop with 8GB+ RAM and KVM/VirtualBox support. Forgiving.
• Qubes: 16GB+ RAM minimum, 32GB comfortable, Intel VT-x with VT-d or AMD equivalents, SSD, qubes-hcl whitelist match. Specific.
• Kicksecure or Vendefoul Wolf: any laptop that runs Debian or Devuan. Forgiving.
• Heads / coreboot underneath: ThinkPad X230/T430/X220/T420 (or Purism / System76 / Dasharo). Specific.

Threats these systems do not address

Worth being explicit. The high-security distros above defend against software-level threats: browser exploits, malicious applications, network surveillance, forensic recovery from disk. They do not defend against:

• Physical coercion. “Unlock this device or you’re not getting out of this room” defeats encryption. Defense is jurisdictional (be where coercion is illegal) or operational (plausibly-deniable hidden volumes, duress passwords that wipe), not technical.
• Side-channel attacks on the hardware. Power analysis, electromagnetic emanations, acoustic cryptanalysis. Defense is physical (Faraday-shielded room) or hardware-level (open silicon like Precursor), not OS-level.
• Stylometric fingerprinting. Your writing style identifies you across pseudonyms. The Whonix wiki explicitly warns about this. No OS can fix it; it’s a behavioral discipline.
• Time-correlation attacks on Tor. A global passive adversary observing both ends of the Tor network can correlate your traffic. Defense requires more than Tor alone (operations-time discipline, mixing in additional latency, sometimes running operations on a delay).
• Targeted hardware implants. A nation-state-level adversary who has interdicted your hardware shipment has access below the OS layer. Defense is supply-chain (buy openly, verify firmware on receipt, prefer hardware you can audit) and physical-security (don’t leave the laptop in hotel rooms when traveling).

If your threat model includes any of these, the OS choice is necessary but not sufficient. Behavioral discipline and physical-security practice matter more than which Linux you picked.

Other “anonymous” distros worth knowing why to skip

• Kodachi. Advertised as a “secure anti-forensic Linux,” but per a bitsex.net technical review, structurally Ubuntu with theming and shell scripts rather than a system-level hardened distro the way Tails is. The hardening claims don’t survive inspection. Skip.
• Subgraph OS. The original project went dormant; the team’s successor project, Citadel, has not produced production releases as of late 2025. Promising design (Grsecurity-kernel, OZ application sandboxing) but not currently a working option. Wait and see.
• Parrot OS, Kali Linux. Penetration-testing distros, not user-privacy distros. They’re configured to be useful tools for someone doing offensive security work, not to defend the user running them. Different problem; wrong tool.
• HardenedBSD. Real hardening work on FreeBSD (ASLR, W^X, segvguard). Worth considering if you’re already a FreeBSD user. For someone coming from a desktop-Linux background, the BSD learning curve is its own commitment; Tails/Whonix/Qubes on Linux are closer to where you already are.

Qubes OS

Security through compartmentalization. Built on the Xen hypervisor; every application runs in its own virtual machine (a “qube”). Work qube, personal qube, banking qube, untrusted-browsing qube; they share only a clipboard and controlled file transfer.

Built on: Xen hypervisor, with Fedora dom0 (the privileged admin layer) and Fedora/Debian/Whonix templates (the VMs your apps actually run in).

License: FOSS.

Good for: high-sensitivity threat models. Journalists, dissidents, security researchers, anyone whose threat model assumes one app will eventually be exploited. If your browser gets owned, it gets owned inside a disposable VM; the rest of your system is unaffected. Snowden is a public proponent.

Bad at: running on laptops. Needs real hardware, 16GB RAM minimum (32GB comfortable), a recent Intel or AMD CPU with virtualization extensions, and a whitelisted hardware list (check qubes-hcl before buying). Battery life takes a hit. GPU passthrough and suspend/resume are historically fragile.

Trajectory: stable and well-maintained. Qubes OS 4.3 released early 2026; 4.2 remains supported.

Use if: your threat model is serious, you have the hardware, and you’re willing to invest in the mental model.

Whonix (and the VPN question)

Two Debian-based virtual machines you run on top of another OS: a Whonix-Gateway that forces all traffic through the Tor network, and a Whonix-Workstation that only gets internet via the Gateway. Standard deployment is VirtualBox or KVM on a Linux / Mac / Windows host. The most secure deployment is inside Qubes, as Qubes-Whonix.

Short version for newcomers: Tor is a free anonymity network that routes your traffic through three volunteer-run servers, each of which only knows one hop, so no single party sees both who you are and what you’re doing. Whonix is a pair of VMs that forces every application on your computer to go through Tor whether the app knows about Tor or not. If an application tries to connect directly to the internet, Whonix blocks it. This is the part commercial VPNs cannot do.

Built on: Debian + Tor + a hypervisor (Xen / KVM / VirtualBox).

License: FOSS.

Good for: IP-hiding and anonymity as a persistent state. Unlike Tor Browser alone, Whonix forces every application in the Workstation through Tor, email, chat, VoIP, SSH, crypto wallets, everything. The “fail-closed” design means that if an app tries to bypass, it gets blocked rather than leaking.

Bad at: being a single-install OS. Whonix is the two VMs; it needs somewhere to run. Most users run those VMs on top of a regular host OS (Windows, Mac, or Linux via VirtualBox or KVM), which means the host OS is the weak link, if Windows is compromised, your keystrokes are exposed before Whonix sees them. Qubes-Whonix fixes this by running the Whonix templates inside Qubes, whose dom0 is hardened and not used for normal work. A dedicated bare-metal Whonix-Host OS (based on Kicksecure) is in development but the project explicitly says it is not yet ready for users. Physical isolation (running Gateway and Workstation on two separate physical machines) is supported but now rare. Beyond the deployment question: Tor is slower than direct internet, and some services (Cloudflare-protected sites, streaming services, banking) block Tor exits.

Is Whonix actually safer?

It depends on what you’re comparing to and what you’re trying to protect.

Safer than regular Linux for anonymity, yes. On regular Linux, applications connect to the internet directly, can see your real IP address, and can be tricked or exploited into leaking it. Whonix’s fail-closed design blocks any connection that doesn’t go through Tor, so even an exploited application can’t leak your IP because the operating system won’t let it.

Safer than Tor Browser alone for anonymity, yes. Tor Browser only anonymizes what happens inside the browser. Your email client, your chat app, your SSH client, your cryptocurrency wallet, and anything else that touches the network are still connecting directly. Whonix extends Tor to every app on the Workstation.

Not safer than Qubes OS for compartmentalization. Different axis. Whonix isolates you from the network. Qubes isolates apps from each other. The best answer is both: run Whonix templates inside Qubes, which is supported and well-documented.

Not immune to host compromise. Whatever is acting as the host of your Whonix VMs (your Windows / Mac / Linux desktop, or Qubes dom0) is where attackers will go first. If that layer is owned, your Whonix session is owned, because the host sees your keystrokes before Whonix does. The specific exploit class that crosses the host boundary is VM escape: a vulnerability that lets code running inside a guest VM execute on the host. KVM, Xen, and VirtualBox all have CVE history here; escapes are rare and patched quickly but exist. Qubes-Whonix on Xen with a minimal dom0 raises the bar against this class meaningfully higher than Whonix-on-KVM on a general-purpose Linux desktop, which is why Qubes-Whonix is the recommended deployment when the threat model includes active exploitation.

Not immune to Tor’s own limitations. A sufficiently powerful adversary that can see both ends of the Tor network (for example, a nation-state that monitors both your country’s internet and the destination site) can correlate traffic and reduce anonymity even through Tor. This is a real limitation and is why Tor is not a substitute for operational security discipline.

So: Whonix is the best practical tool for IP-level anonymity and app-wide Tor routing on consumer hardware. It is not magic, it does not protect against a compromised host, and it does not replace thinking about what you’re actually doing.

Does using Whonix mean I don’t need a VPN?

Yes. Whonix routes all your Workstation traffic through Tor. Tor is the anonymity layer, and it is a much stronger one than any commercial VPN: Tor is a three-hop encrypted anonymization network run by thousands of independent volunteers, whereas a VPN is a one-hop encrypted tunnel run by a single company that knows exactly who you are and could be compelled to say so. Stacking a commercial VPN on top of Tor does not compound privacy, one extra layer run by a single identifiable party does not strengthen three layers run by unrelated volunteers.

The Whonix project’s own documentation goes further and describes the VPN-plus-Tor debate as “the law of triviality / bikeshedding”, people argue about VPNs because VPNs are easy to argue about, while the real privacy issues (browser fingerprinting, traffic-analysis attacks, keystroke timing, guard-relay discovery) get less attention[^whonix-vpn]. Commercial VPNs also have their own problems: single-party trust, the possibility of logging, the Port Shadow attack disclosed in 2024 affecting shared-port VPN servers, and the fact that VPN companies are routine acquisition targets for surveillance operators.

You can add a VPN to Whonix if you have a specific reason: your ISP blocks Tor and you need to reach Tor through a VPN (user → VPN → Tor → internet), or you want Tor to reach a service that blocks Tor exits (user → Tor → VPN → internet). Both configurations are advanced, explicitly unsupported by the Whonix team, and not leak-tested the way the base setup is. Without one of those specific reasons, don’t do it.

The same logic applies to Tails. Tor is the protection; a VPN adds complexity without adding anonymity.

Short answer: use Whonix or Tails, don’t bother with a commercial VPN on top.

Trajectory: stable. Funded through 2026 by Power Up Privacy.

Use if: you want Tor-forced anonymity as a persistent workspace. Best inside Qubes; acceptable standalone.

Tails

The Amnesic Incognito Live System. A Debian-based live USB that runs entirely in RAM, routes all traffic through Tor, and forgets everything on shutdown. Optional encrypted Persistent Storage for things you want to survive reboots. Merged operations with the Tor Project in September 2024.

Short version for newcomers: You boot your computer from a USB stick, everything runs in memory, and when you shut down, nothing is left behind, not on the USB, not on the computer, not in the logs of your ISP (because Tor anonymized it). Good for one-off sensitive sessions, research on shared computers, any situation where the computer should remember nothing.

Built on: Debian + Tor + GNOME.

License: FOSS, with non-free firmware blobs included for hardware support (Tails prioritizes booting on arbitrary hardware over strict libre purity).

Good for: specific sessions where nothing should persist. Whistleblowing, research on sensitive sources, using a shared or borrowed computer, any situation that benefits from the machine forgetting. MAC address spoofing, Tor routing, amnesia by default, vetted software selection.

Bad at: being a daily driver. Runs from USB, every session starts clean unless you set up Persistent Storage, no GPU-accelerated gaming, narrower hardware support than a full Debian install.

What Tails protects against and what it doesn’t

Tails routes all traffic through Tor at the network layer, so destination services see a Tor exit IP and not yours. That is the strong property. The weaker property (worth understanding before relying on Tails for high-stakes work) is that Tails runs every application on the same kernel that has the real network interface. The kernel knows your real IP (it is bound to the physical NIC); Tor enforcement is a set of netfilter rules on the same system the apps run on. An exploit that gets code execution at sufficient privilege can read the real IP directly and exfiltrate it out-of-band, bypassing Tor entirely.

This is not hypothetical. The FBI’s Playpen operation in February 2015 used a Firefox vulnerability in the Tor Browser bundle to install what the agency called a Network Investigative Technique on visitors’ machines; the NIT read the real IP and transmitted it directly to an FBI server in Alexandria, Virginia, outside the Tor network. Roughly 137 federal prosecutions followed.[^playpen-nit] Tails users hit by the same class of exploit would have been deanonymized the same way: the architecture is not different on this axis. Whonix is structurally different here, the Workstation VM has no route to the real network and no awareness of the real IP, so a compromised application inside it cannot leak what isn’t there. See the Whonix entry above.

What Tails does to reduce the practical risk:

• AppArmor confinement on Tor Browser. A successful Firefox exploit lands inside an AppArmor-restricted process that cannot read most of the filesystem, cannot execute arbitrary binaries, and cannot do much beyond browsing. Reaching the privilege level needed to query network interfaces is a separate exploit on top.
• Unprivileged user account. The Tails user runs without sudo by default and the OS is read-only on the USB stick during a session.
• Amnesia bounds persistence. A successful compromise dies at reboot. There is no foothold for the attacker to maintain across sessions, which is the property a long-term surveillance operator usually wants.

These do not change the architectural fact that the kernel possesses your real IP. They raise the cost of an exploit that uses that fact and limit how long a successful one lasts. For most users at most threat levels, that is enough. For users whose adversary will burn a browser zero-day in a single session, it is not, and Qubes-Whonix is the answer.

Trajectory: stable, well-funded, well-audited. The Tor Project merger in 2024 strengthened both sides.

Use if: you need amnesic, Tor-forced, leave-no-trace sessions. Keep a Tails USB in a drawer even if you don’t use it often.

Kicksecure

The Whonix team’s hardened Debian. Same base as Whonix, same security work, but without Tor in the default path, so you get normal internet speeds and unblocked access to sites that refuse Tor exits.

Short version for newcomers: “Hardening” in operating-system security means changing the defaults so that if something goes wrong, the damage is smaller. Vanilla Linux boots with a lot of features enabled that most people never use (some kernel modules, some network protocols, some permission combinations) and any of those can become an attack surface if a bug is found. A hardened system switches those off by default and tightens the remaining ones. Kicksecure does this systematically on top of Debian.

Concretely, Kicksecure applies:

• AppArmor confinement, a Linux kernel feature that restricts what specific applications can do, even if they’re exploited. An AppArmored Firefox cannot read your SSH keys even if an attacker controls the browser.
• Kernel-parameter tightening via /etc/sysctl.d/ and boot parameters, disables kernel features that enable kernel-level exploits, restricts what processes can see about each other, turns off obscure network protocols most people don’t use.
• Anti-fingerprinting measures, removes or randomizes some of the data that websites and network observers use to identify your specific machine.
• Tightened filesystem permissions on sensitive files and directories.
• Reduced attack surface, fewer services running by default, fewer open network ports, smaller installed package set.
• Hardened memory allocator options that make certain classes of memory-corruption exploit harder.
• A hardened web browser (Tor Browser or a hardened Firefox variant) available by default.

Built on: Debian. Built by the same team that builds Whonix. In fact, Whonix itself is built on top of Kicksecure, the Whonix VMs inherit every Kicksecure hardening measure, then add Tor-forced networking on top. Kicksecure is what you get if you take Whonix and pull the Tor layer off.

License: FOSS, with the same non-free firmware repository structure as Debian.

Good for: someone who wants hardened Debian for general use without Tor in the default path. You get Debian’s package ecosystem, Debian’s stability, and a security posture significantly better than a default install, but you still use the regular internet at regular speed. If you decide later that you want Tor-forced anonymity, you can add the Whonix-Gateway on top; your Kicksecure install becomes the Workstation.

Bad at: discoverability. The project is the less-famous half of the Whonix team’s work and most tutorials don’t mention it.

Trajectory: stable. Tracks Debian stable. Active development through 2026.

Use if: you want a Debian desktop with security defaults turned up, and Tor-routed anonymity is not your primary goal. This is also the right choice if you’re curious about Whonix but not ready for the full Tor-forced setup, you can graduate to Whonix later without reinstalling.

Vendefoul Wolf

A small, principled, Devuan-based distribution. Ships with OpenRC instead of systemd, XLibre (a community fork of X.org) instead of Wayland, no telemetry by default, and no AI integration. Preinstalls LibreWolf browser and KeePassXC password manager. Primary development community is Spanish-speaking but the distribution is fully usable in English. Uses the Calamares installer.

Built on: Devuan 6 with a custom kernel.

License: FOSS.

Good for: users whose stated values match the project’s stated values, no systemd, no Wayland, no telemetry, no AI. Ready-to-use for common hardware (printers, Bluetooth) out of the box. Fast install via Calamares. Runs lean on modest hardware.

Bad at: ecosystem and bus factor. Small team, repositories largely depend on Devuan upstream, small community. If the lead maintainer loses interest, the project is in trouble. Documentation is Spanish-primary, English-secondary.

Trajectory: active, niche, regularly releasing ISOs through 2026.

Use if: your priorities match the project’s stated defaults and you’re OK with a small project. Otherwise, Devuan with manual customization gets you most of the way and has a larger base.

Trisquel and PureOS (libre)

Two of the Free Software Foundation’s endorsed fully-libre distributions. Zero proprietary firmware, zero proprietary drivers, zero non-free software in the default repos. Trisquel is Ubuntu-derived; PureOS is Debian-derived (shipped on Purism’s Librem hardware).

Good for: anyone who wants to use only free software. The FSF endorsement means every package has been audited for freedom; no closed-source binary blobs anywhere.

License: strictly FOSS, no proprietary firmware or drivers of any kind. Hardware compatibility will be narrow as a direct consequence.

Bad at: modern hardware. Most Wi-Fi cards, almost all GPUs, many laptops, and most printers need proprietary firmware these distros refuse to ship. You need specific hardware to have a working system.

Flags: both distros live inside the FSF’s political orbit, which has had its own governance tensions since 2019.

Trajectory: narrow-purpose, stable. Trisquel 12 released 2024; PureOS tracks Debian stable.

Use if: software freedom in the FSF-maximalist sense matters to you.

Where Linux comes from: the Unix lineage

Before the BSDs make sense on this list, you need the briefest history of where any of this came from. This is the one piece of context the rest of the doc depends on.

In 1969, a small team at Bell Labs in New Jersey (Ken Thompson, Dennis Ritchie, and others) built an operating system they called Unix. It was small, written in a language they also invented (C), and it established most of the ideas that modern operating systems still use: everything is a file, programs should do one thing well and compose together, text is the universal interface.

Unix was licensed by AT&T to universities and companies. One of those universities was Berkeley, where researchers modified Unix heavily and released their own version called BSD (Berkeley Software Distribution) starting in the late 1970s. BSD eventually replaced the original AT&T code, fought a legal battle with AT&T in the early 1990s, won, and became 386BSD, the ancestor of every BSD you’ll see today. The main surviving branches are FreeBSD, NetBSD, and OpenBSD. Apple’s macOS kernel also descends partly from BSD, which is why macOS feels Unix-like underneath the candy coating.

Linux took a different path. In 1991, a Finnish student named Linus Torvalds wrote a new Unix-like kernel from scratch, no Unix code, no BSD code, just the ideas and the interface conventions. He started because Unix was expensive and Minix (a teaching OS) was limited. His kernel is what we call Linux today, and it is a clean-room reimplementation of Unix ideas, not a descendant of Unix code.

So when you see OpenBSD or FreeBSD on this list, understand:

• They are not “a kind of Linux.” They are Linux’s cousins from the same family of ideas.
• OpenBSD and FreeBSD contain real lineage from AT&T Unix via Berkeley. Linux does not.
• From a user’s perspective they behave almost identically to Linux. Same shell, same file layout, same basic tools. Differences mostly matter to system administrators and kernel developers.
• BSDs tend to be more coherent by design, the kernel and the userland (the tools that run on top of the kernel, shell, file utilities, compiler) are developed together as one project, instead of the Linux model where the kernel and the GNU userland are separate projects stitched together.
• BSD licensing is permissive (do whatever you want with the code, including ship a proprietary product built on it). Linux licensing is copyleft (derivatives must stay open). This has real consequences: Apple’s macOS kernel is partly BSD-derived; no major commercial consumer OS is Linux-derived in the same way.

One footnote on the lineage that belongs in any honest telling of it: Minix, the teaching OS Andrew Tanenbaum wrote in the 1980s (the OS Linus Torvalds learned Unix ideas from before writing Linux) is still alive, but in a place you can’t see. Intel ships a small Minix 3 instance embedded inside every modern Intel CPU as part of the Management Engine firmware, running underneath your operating system with access to parts of the hardware you do not have. Tanenbaum himself did not know Intel had done this until it was reported publicly. If you are running a modern Intel CPU right now, a Minix instance is running on it that you cannot inspect, disable, or audit. This is not an argument for or against any particular distro; it is a reminder that the software that runs on your computer is not only the software you chose.

This matters for the decision ahead. If you want something that behaves like Unix because it literally descends from Unix, smaller and more conservative and audited more carefully than any Linux distribution, keep reading.

BSDs as an adjacent path

The BSDs share several properties that distinguish them from Linux: the kernel and base userland are developed as one coherent system, the codebases are smaller and more auditable, licensing is permissive (BSD / ISC) rather than copyleft, and governance is more centralized and engineering-led. They’re not on the “sovereignty right” end of the politics axis because they’re a different axis entirely, technical conservatism over political alignment.

OpenBSD

Theo de Raadt’s project, forked from NetBSD in 1995. Security and correctness through code auditing, minimalism, and aggressive default hardening. The smallest audited codebase of any general-purpose OS.

License: strictly BSD/ISC. OpenBSD goes further than most BSDs in refusing to ship code it can’t audit.

Good for: security-critical servers, firewalls (pf is one of the best packet filters in existence), anyone who values a system designed to be correct by construction. Default install is minimal. OpenSSH, LibreSSL, and pf are OpenBSD projects that the rest of the world uses.

Bad at: desktop polish, gaming, modern hardware (NVIDIA in particular is not supported), running recent proprietary software.

Use if: you want security through minimalism and you’re OK giving up modern desktop conveniences.

FreeBSD

The largest of the BSDs. Production-serious Unix with a vast ports tree and strong enterprise use. Netflix’s CDN runs on FreeBSD; PlayStation’s OS is FreeBSD-derived; TrueNAS (the most-deployed storage software in the world) is FreeBSD.

License: BSD.

Good for: servers (especially storage), NAS, coherent Unix desktops (see GhostBSD below for a ready-to-run desktop build). ZFS (a modern filesystem with snapshots, compression, and data-integrity checking) is first-class. Jails are lightweight containers that predate Docker by a decade.

Bad at: Linux-specific software assumptions. Gaming works via Wine + Steam + a compatibility layer but lags Linux meaningfully.

Trajectory: 15.1-RELEASE shipped 16 June 2026, the second point release of stable/15. Supported through 31 March 2027; the 15 series runs through 31 December 2029. 15.0-RELEASE reaches EOL on 30 September 2026, so a fresh install today should target 15.1.

Use if: you want a coherent Unix for a server, a NAS, or a serious desktop, and you don’t need Linux-specific software.

GhostBSD

FreeBSD made desktop-usable. Started by Eric Turgeon in 2009 to make FreeBSD installable and productive for a regular user without reading the FreeBSD Handbook first. MATE is the default desktop, with XFCE and a new Gershwin (GNUstep-based, macOS-like) community edition. The graphical installer handles partitioning and uses ZFS by default. Versioning scheme is YY.MM-R<FreeBSD base>p<patch>, for example 26.1-R15.0p2.

Short version for newcomers: Not a first-Linux replacement, because it isn’t Linux. For someone who already uses Linux comfortably and is curious what BSD feels like, GhostBSD is by far the easiest way in.

Built on: FreeBSD. Uses FreeBSD’s packaging (pkg), ports tree, and rc init (FreeBSD does not use systemd and never has).

License: BSD.

Good for: someone who wants FreeBSD’s security posture, ZFS-by-default, and jail ecosystem with a working desktop on first boot. No systemd, ever. Excellent ZFS integration. WireGuard and 802.1X enterprise WiFi in the network manager as of the 2026 release. Small project with engineering-led governance, the lead developer switched the default display server from X.Org to XLibre in April 2026 specifically because of the governance turbulence around X.Org’s reverts, which is the clearest statement of priorities a distro can make.

Bad at: gaming. Proton runs through FreeBSD’s Linuxulator compatibility layer rather than natively, with meaningfully worse coverage and performance than on any Linux distro. Steam is not officially supported on FreeBSD. Bad at: the long tail of Linux-assuming desktop software, one recent reviewer confirmed Obsidian and Joplin won’t install; Adobe, Spotify-official, and other proprietary Linux-targeted apps typically don’t exist for FreeBSD. Bad at: fractional display scaling (not supported). Bad at: Wayland (X.Org and XLibre only; Wayland is on the roadmap but not shipped). The installer only supports FreeBSD’s BIOS loader, no rEFInd, no FreeBSD boot manager option during install. In-place upgrades from 25.02 to 26.1 stay on X.Org; fresh installs boot to XLibre.

Trajectory: 26.1-R15.0p2 shipped 18 April 2026, the first GhostBSD release on FreeBSD 15.0. Default shell switched to zsh. Active small-team development, predictable release cadence aligned with FreeBSD upstream.

Use if: you are Linux-literate, curious about BSD, and want a working desktop on first boot. Not for gaming, not for first-time migrants, not for someone who needs Adobe software.

A note on the XLibre dimension: GhostBSD’s adoption of XLibre is an engineering decision and reads cleanly as one (Good for, above). XLibre as a project is messier, its public-facing rhetoric drew an ArchWiki page deletion in April 2026 that doesn’t reduce to the GNOME-side CoC asymmetry pattern. See “The XLibre / ArchWiki case (April 2026): a contrast” inside Community politics for the longer treatment.

NetBSD

The portability project. Runs on more architectures than anything else on this list, desktop PCs, ARM boards, embedded hardware, VAX minicomputers, toasters.

License: BSD.

Good for: unusual hardware, research, education, extreme portability as a design principle.

Use if: you have weird hardware or you like the idea of one OS running on everything.

Further reading: how to actually learn Linux

Once you’ve picked a distro and installed it, the natural next question is how to actually understand the system under your hands. The short answer: books build the mental model, websites help you apply it. Most people starting out go straight to websites and Stack Exchange, copying solutions without understanding them, they can operate Linux but don’t understand it. Books fix that. Websites are for applying and troubleshooting. The two serve different moments in the learning process.

If you only read one book, read this:

• The Linux Command Line, William Shotts. Free online at linuxcommand.org. The best starting point for building real competence at the shell.

From there, in a loose order:

• How Linux Works, Brian Ward. Explains what’s actually happening under the hood without requiring you to be a programmer. Good second book.
• Unix and Linux System Administration Handbook, Nemeth, Snyder, Hein, Whaley. The professional standard. Heavy. Called “the bible” in sysadmin circles.
• The Linux Programming Interface, Michael Kerrisk. For understanding how Linux works at the system-call level. Kerrisk maintains the Linux man pages, so this is authoritative.
• The Unix Programming Environment, Kernighan and Pike. Written in 1984 and still worth reading. Teaches how Unix was meant to be used and thought about.
• Advanced Programming in the Unix Environment, W. Richard Stevens. Stevens was the best technical writer in this space. Dense but authoritative.
• Linux Kernel Development, Robert Love. The accessible introduction to kernel internals, for when you want to start understanding what the kernel actually does.
• Understanding the Linux Kernel, Bovet and Cesati. Goes deeper than Love’s book.

Websites, roughly ordered by when in your journey they’re most useful:

• bellard.org/jslinux. Fabrice Bellard’s in-browser PC emulator. Boots Alpine, Buildroot, Fedora RISC-V, or Windows 2000 in a tab. Useful for playing with a real shell from a machine where you can’t install or boot anything.
• linuxjourney.com. The best answer for a true newcomer starting from zero. Not tied to any distro. Structured like a course: what is Linux, what is the kernel, how does the filesystem work, in plain language.
• The Gentoo Handbook. Even if you never install Gentoo, reading through it teaches you how a Linux system is actually assembled. Partitioning, filesystems, kernel compilation, init systems. Educational.
• The Arch Wiki. Your permanent reference. You’ll use it for the rest of your Linux life regardless of what distro you run.
• The Linux Documentation Project (tldp.org). Dated in places, but guides like the Linux System Administrator’s Guide still hold up conceptually.
• kernel.org documentation. For when you’re past intermediate and want to understand the kernel itself.
• man pages. Built into every Linux system. Underused by beginners. Learning to read man pages well is a skill in its own right.

A reasonable order: linuxjourney → Shotts → Ward → Gentoo Handbook (read, don’t necessarily install) → Nemeth → then branch into whichever direction your work takes you (sysadmin, programming, kernel). The Arch Wiki becomes a constant companion from roughly step three onward.

No amount of reading replaces breaking things and fixing them yourself. Books give you the mental model. Websites help you apply it. But the final teacher is the terminal.

Choosing Encryption Tools

A guide to encrypting your files, folders, and disks when you’ve left Windows and Mac for Linux. Written for beginners. Updated as things change.

TL;DR

If you’re still on Windows or macOS and reading this, the highest-leverage move you can make is leaving. Both platforms encrypt your disk by default and then quietly hand the keys to the vendor. BitLocker uploads your recovery key to your Microsoft account. Apple holds your iCloud encryption keys unless you’ve gone out of your way to turn on Advanced Data Protection. You are not the only one who can decrypt your data on those systems. This is upstream of every other encryption decision you’ll make.

Migrate to Linux. The companion OS guide os.md covers picking a distro. Once you’re there, encryption gets simple:

• Encrypt your laptop disk. Enable LUKS at install. Every Linux installer offers a checkbox. This solves “stolen laptop” once and forever.
• Backups. The backup tool encrypts the archive itself; choosing among Borg, Restic, and the rest is covered in choosing-backup-tools.md.
• Cloud-synced folders. Cryptomator on top of whatever sync provider you tolerate, or move to a provider that does end-to-end encryption natively (Proton Drive, Tresorit, Mega).
• Encrypting a single file to keep for yourself. GPG to your own key if you already keep one for pass and signing, since it adds no new tool and no new key to back up; age if you do not run GPG.
• Sending an encrypted file to someone. age. The recipient can install it in one minute on any platform.
• Encrypted vault on a USB stick that has to work on Linux, Mac, and Windows. VeraCrypt today; LUKSbox once it leaves pre-1.0; not BitLocker To Go (recovery keys go to Microsoft).
• GUI drag-and-drop for one file. Picocrypt.
• Plausible deniability under coercion. VeraCrypt hidden volumes, the only tool in this category, and the Windows side of it is at risk through 2026 (more below).

If you’ve never encrypted a file in your life and you don’t know where to start, do this in order: turn on LUKS the next time you reinstall, install Borg and back up your home directory once a week (the backup tools themselves are covered in choosing-backup-tools.md), and don’t touch the rest of this guide until those two are habits.

What this artifact is not: a how-to. There are no commands here. This is for picking the right tool for the right job and understanding the politics of each. The companion devuan-luks2-install.sh in the project covers the actual install procedure for users who want to bypass distro installer wrappers. For the concepts beneath these tools (what encryption, signing, hashing, keys, and the web of trust actually are), the companion gpg-concepts.md is the home; this guide assumes those and points there rather than re-explaining.

Why your encryption choices matter: the surface you’re defending

You don’t need a threat model that includes nation-states for any of this to matter. The default threat model is much more banal:

• A laptop gets stolen out of a coffee shop or airport. Without encryption, the thief mounts the drive on another machine and has every file you’ve ever opened, every browser session, every cached credential. With encryption, they have a brick.
• A used SSD gets resold. Modern flash retains data far past what rm deletes. Without encryption, the buyer recovers your tax returns. With encryption, they recover noise.
• A cloud provider is compelled to hand over data, gets breached, or has an employee curious about you. Without client-side encryption, the provider can read everything. With it, they hand over ciphertext.
• A backup drive lives in a friend’s apartment or a parent’s house for off-site storage. Without encryption, anyone in either household can read it. With encryption, the off-site arrangement actually works.

This is the stuff encryption-at-rest defends against. What it does not defend against:

• Malware on your unlocked, logged-in machine. The data is decrypted in memory while you’re using it; encryption-at-rest does nothing here.
• Keyloggers. If the password gets captured, the math is irrelevant.
• Coercion. A government with rubber hoses gets the password. (The narrow exception below: VeraCrypt hidden volumes.)
• Backups of the unencrypted versions of files. If you ever decrypted a file and then synced your home directory to OneDrive in plaintext, that’s the version that exists.
• Cold-boot attacks on a powered-on or suspended laptop. RAM holds keys for several seconds after power is cut, longer if cooled. “Locked screen” is not “encryption is engaged.” Real protection means powered off.
• Evil-maid attacks on an unattended laptop. An attacker with a few minutes of physical access to a powered-off machine can install a malicious bootloader that captures the password on next boot. Secure Boot, TPM-measured boot, and Heads are partial countermeasures.
• “Self-encrypting” SSDs. Most consumer drives marketed as SEDs were shown by Radboud University researchers in 2018 to have firmware-level flaws that make their hardware encryption effectively useless against a determined attacker. Encrypt at the OS layer regardless of what the drive claims.

Why proprietary disk encryption is hostile

Both BitLocker and FileVault encrypt your disk competently. That’s the easy part. The hostile part is what they do with the keys.

BitLocker, in its default Windows 11 configuration, uploads your recovery key to your Microsoft account during initial setup. The official Microsoft documentation calls this a feature: “If you forget your password, we can help you recover your data.” What it actually means is that your encryption key is in Microsoft’s possession, available to anyone who can compel Microsoft to hand it over (subpoena, National Security Letter, internal compromise) and exposed in any future Microsoft account breach. There is a way to encrypt without uploading the key (you have to create a local account, refuse online setup, and use manage-bde from the command line) but Microsoft makes this path increasingly difficult with each Windows update. The ordinary user enables BitLocker and a copy of their key sits on Microsoft’s servers.

Apple’s FileVault doesn’t upload the disk-encryption key by default. But your iCloud data (including iMessage backups, photos, notes, files in iCloud Drive, and Safari history) is by default encrypted with keys Apple holds. Apple can read it. Apple can hand it over. Apple has handed it over, repeatedly, to law enforcement requests. The opt-out is called Advanced Data Protection, was introduced in iOS 16.2 in late 2022, requires you to set up at least one recovery contact or recovery key, and is off by default. Most Mac users have never heard of it. Their iCloud is end-to-end encrypted in the marketing material and not end-to-end encrypted in fact.

These aren’t bugs. They’re product decisions. Microsoft and Apple sell convenience and recoverability; they price that against your sovereignty over your own data, and the price is your sovereignty. This is the same pattern as Recall (Windows 11’s screenshot-everything feature), notarization (Apple deciding what software you can run on your own machine), and account-required setup. The OS vendor takes a seat at the table you didn’t offer them.

The Linux equivalent (LUKS) does not phone home. The key derivation runs locally; the master volume key never leaves your machine; there is no recovery service. If you forget your password, the data is gone. That’s the deal. It’s a worse experience for the careless user and a categorically better one for everyone else.

This is the upstream argument for migrating in the first place: the encryption layer on Windows and Mac is not on your side. The companion OS guide covers what to migrate to. This guide covers what to do once you’re there.

A note on phones

Modern Android (since 10) and modern iOS encrypt the user data partition by default, tied to the screen lock. This is competent, hardware-backed encryption and for typical threat models you don’t need to do more. The tools below are for desktops, laptops, and external storage. If you’ve migrated your computer to Linux but your phone is still iOS or pre-GrapheneOS Android, that’s a separate conversation; see GrapheneOS, CalyxOS, and the phone section in the OS guide.

Out of scope here: password managers (KeePassXC, Bitwarden), encrypted messaging (Signal, Matrix), and email encryption layered on top of plaintext mail.

Four families of tool

Encryption-at-rest tools cluster into four families. The right tool depends almost entirely on which family fits the job.

Whole-disk / full-disk. The whole partition or disk is encrypted. You unlock it once at boot with a password; everything on it is then transparently readable until shutdown. Examples: LUKS (Linux), VeraCrypt system encryption. Use case: laptop or external drive that you want to be a brick if stolen.

Container vault. A single file (often hundreds of MB or several GB) that mounts as a virtual drive when unlocked. Inside it looks like a folder; outside it looks like one opaque blob. Examples: VeraCrypt containers, Tomb, LUKSbox. Use case: a vault on top of an otherwise-unencrypted system, or a portable encrypted volume on a USB stick.

Per-file or per-folder. Each file is encrypted individually, and the encrypted output is a file you can move around, sync, or share. Some present as a transparent virtual folder where you drop plaintext and it gets encrypted on save. Examples: age, GnuPG, Picocrypt, Cryptomator, gocryptfs. Use case: cloud-syncable encryption, sharing a single encrypted file with someone, or per-directory encryption on Linux.

Encrypted backup archive. A backup-format-first tool where encryption is built into the format. The output is a deduplicated, compressed, encrypted archive that only the backup tool understands. These are backup tools first, with encryption as a property of the format rather than the point of the tool; choosing among them (Borg, Restic, and the rest) is covered in choosing-backup-tools.md, and this guide does not re-select them. Use case: serious backup of large amounts of data to local or remote storage.

A sub-mode that crosses these categories: filesystem-native encryption. ZFS native encryption and Linux’s fscrypt operate at the filesystem layer, neither block-level (whole-disk) nor application-level (per-file). They appear once in the tools list below but conceptually they sit alongside the four families rather than inside any one of them.

The tools

For each, a short description, who maintains it, what it fits, its political and ideological lineage, and the main tradeoff. Each entry closes with a “Use if:” line.

LUKS / dm-crypt

The Linux kernel’s native full-disk encryption. dm-crypt is the kernel module; LUKS is the standard on-disk format on top of it; cryptsetup is the userspace tool. Every mainstream Linux distribution offers it as the “Encrypt my disk?” checkbox in its installer.

Maintainers: Linux kernel team. Milan Broz is the most prominent userspace maintainer.

Fits: laptop disk encryption, encrypted partitions, encrypted external drives that will only ever be used on Linux.

LUKS comes in two on-disk formats. LUKS1 is the original; LUKS2 (default on most distributions since around 2019) supports Argon2id key derivation, multiple keyslots with stronger metadata, and re-encryption of an existing volume. New installs should use LUKS2. Stay on LUKS1 only if your bootloader doesn’t support LUKS2 yet, which is increasingly rare.

A related modern variant: systemd-homed, which encrypts each user’s home directory in its own LUKS image rather than encrypting the whole disk. This gives “home only” protection (your data is encrypted; system files aren’t) and lets the encryption follow the user across machines. Politically it’s part of the systemd ecosystem and inherits everything that comes with that; see the systemd discussion in os.md.

For users who don’t want to type a password every boot, modern Linux supports TPM2-backed unlock via Clevis or systemd-cryptenroll. The TPM holds the key and releases it only if the boot chain hasn’t been tampered with. Convenient; not appropriate against an attacker who has physical access to the powered-off device.

For users who want to skip the distro installer and configure LUKS by hand, the project’s devuan-luks2-install.sh is a worked example: GPT + LUKS2 + LVM + ext4 with keyfile-in-initramfs and GRUB cryptodisk, on Devuan. Read it before you run it.

Political leaning: Linux-kernel mainstream. The boring engineering option. Ships in every distribution, gets used at industrial scale, has no political enemies to make. Trusts the Linux kernel security model implicitly.

Tradeoff: Linux-only. An encrypted LUKS partition is not natively readable from Windows or macOS.

Use if: you have a Linux laptop, period. There is essentially no reason not to enable this.

VeraCrypt

The successor to TrueCrypt, the legendary tool that mysteriously shut itself down in 2014 with the cryptic message “WARNING: Using TrueCrypt is not secure.” VeraCrypt forked from TrueCrypt 7.1a, fixed the bugs the audit had found, and continued. It does whole-disk encryption, encrypted container files, and (uniquely) hidden volumes: a volume inside a volume, where you give one password to reveal the outer (decoy) contents and a different password to reveal the inner (real) contents. The point is plausible deniability under coercion: if forced to surrender a password, you give up the decoy.

Maintainer: Mounir Idrassi at IDRIX (France). Effectively a single-person project. Audits funded by OSTIF (Open Source Technology Improvement Fund).

Fits: cross-platform encrypted containers (works on Linux, Windows, macOS); Windows full-disk encryption; the rare cases where hidden-volume plausible deniability genuinely matters.

Political leaning: post-cypherpunk, Snowden-era state-resistant. TrueCrypt was the tool of choice for journalists, dissidents, and activists; VeraCrypt inherits that lineage. Hidden volumes only make sense as a feature if you take seriously the threat of compelled disclosure by a state actor.

Current state matters. Version 1.26.27 (September 2025) added Argon2id, modernizing the password hashing. But in early 2026, Microsoft terminated Mounir Idrassi’s developer account without explanation. He has been unable to sign new Windows drivers or bootloaders since. Linux and macOS releases are unaffected; the most recent upload to SourceForge is from late April 2026. Existing Windows installs continue to work, but the certificate authority used for the VeraCrypt bootloader expires in late June 2026 and Microsoft is revoking it in July 2026, after which Secure Boot may refuse to load new VeraCrypt installations on Windows. Idrassi has called the situation a potential “death sentence” for VeraCrypt-on-Windows. This is not a cryptographic failure or a state-vs-encryption story; it’s a platform-vendor termination story, and the lesson is that even hardcore state-resistant tools depend on cooperation from the platforms they ship through. Note that this is also a strong argument for migrating off Windows, the same gatekeeping that’s killing VeraCrypt is what makes BitLocker the default.

Tradeoff: single-maintainer risk, now actively materializing on the Windows side. On Linux and macOS the tool is unaffected.

Use if: you need a cross-platform encrypted container right now, you’re on Linux or Mac, and LUKSbox isn’t mature enough yet. For Windows specifically, plan for migration off either Windows or VeraCrypt before mid-2026.

LUKSbox

A Rust-based encrypted-container tool from Sébastien Dudek at Penthertz (a French pentesting firm). Released as v0.1.0 on 7 May 2026, two days before this guide. Cross-platform: Linux and macOS via FUSE3, Windows via WinFsp. AES-256-GCM-SIV by default with Argon2id (256 MiB / 3 / 4) for password-based keyslots. FIDO2 (YubiKey, Titan, Nitrokey, Windows Hello) and TPM 2.0 keyslots for hardware-bound unlock. Hybrid post-quantum keyslots using ML-KEM-768 or ML-KEM-1024 with separate .kyber seed files. Detached header sidecar, the vault file alone is opaque random with no magic bytes, so plausible-deniability via detached-header is built in. Anchor sidecar for external rollback detection. Apache 2.0.

Maintainer: Sébastien Dudek, Penthertz.

Fits: cross-platform encrypted containers, especially when stored in untrusted cloud, on shared media, or on USB sticks that travel. Direct competitor to VeraCrypt with all of VeraCrypt’s known issues (single-maintainer-but-newer, no PQ until very recently, no hardware keys, C/C++ codebase, Microsoft-signing dependency) addressed by design choices.

Political leaning: hybrid lineage. The threat model is straight cypherpunk, the README’s “harvest now, decrypt later” framing for the post-quantum slot is canonical post-Snowden paranoia. The tooling lineage is modernist: Rust, Apache 2.0, hardware-key-first, post-quantum-first, FUZZing-and-audit-first development culture. Sits at the intersection of the two camps in a way no other tool on this list does. The fact that it ships from a French pentesting firm rather than a single maintainer or a German privacy company is worth noting, that’s a different funding base than any of the established tools.

Current state: pre-1.0. Nine internal audit rounds. 200+ tests passing, 30M+ fuzz iterations across 10 harnesses. No external audit yet (engagement scope package available on request). The on-disk format is locked; the cryptographic primitives are NIST/RFC standards built on RustCrypto.

Tradeoff: it’s brand new. The pre-1.0 status is real: for sensitive data today, VeraCrypt’s ten-year track record matters more than LUKSbox’s better design. Watch this project. If it gets an external audit and reaches 1.0 cleanly, it likely becomes the default cross-platform recommendation, especially given the VeraCrypt-on-Windows situation.

Use if: you’re paying attention to where the encrypted-container space is going. For production use today, default to VeraCrypt unless you specifically need FIDO2 or TPM keyslots and accept the pre-1.0 risk.

Cryptomator

Per-file encryption purpose-built for cloud sync. You create a “vault” (really a directory tree of individually encrypted files, with encrypted filenames) and point it at your Dropbox / Google Drive / OneDrive / iCloud folder. Cryptomator presents the vault as a virtual unencrypted drive locally; you work with it normally; the sync client uploads only the encrypted versions.

Maintainer: Skymatic GmbH (Germany). Cryptolib audits by Cure53.

Fits: storing sensitive files in commercial cloud storage without the provider being able to read them. Probably the best UX in the entire space for non-technical users.

Political leaning: Eurocentric privacy-commerce. The product assumes you’ve already adopted commercial cloud storage and want a layer of privacy on top, not that you’re going to self-host. Freemium business model, desktop free, full mobile functionality paid. Audited, professionally maintained, GDPR-compliance-shaped, sponsorship-funded. Calling this “least ideological” misses the politics: GDPR-Europe privacy-as-a-product is itself a stance, distinct from American surveillance-capitalism-with-encryption-as-an-afterthought and distinct again from cypherpunk self-hosting.

An alternative to the Cryptomator-over-Dropbox stack: pick a cloud provider that does end-to-end encryption natively. Tresorit (Swiss, proprietary, expensive), Proton Drive (Swiss, partially open-source clients, Proton ecosystem), and Mega (New Zealand, open-source clients) all encrypt client-side by design. You trade Cryptomator’s provider-independence for a more integrated experience and the politics of whichever provider you pick. For users still on iCloud or OneDrive, Cryptomator on top of those services is one of the few ways to make commercial-cloud use defensible.

Tradeoff: filenames and contents are encrypted, but file count, individual file sizes, and modification timestamps leak to the cloud provider. Mobile apps cost money. Single-company project, not a community.

Use if: you’re keeping a commercial-cloud sync provider and want them not to read your files.

age

A modern, minimalist file encryption tool, designed as an explicit reaction against GnuPG. The philosophy is in the README: “small explicit keys, no config options, UNIX-style composability.” Public keys are short strings starting with age1...; private keys are short strings starting with AGE-SECRET-KEY-1.... There are no key servers, no web of trust, no signing, no compression options, no hash algorithm choices. There is one cipher (ChaCha20-Poly1305 at the symmetric layer, X25519 at the asymmetric layer), and it works. age can also encrypt directly to an SSH public key, so anyone with an ssh-ed25519 key already has an age recipient, which makes distributing to a small team as simple as sharing the SSH keys they already publish.

Maintainers: Filippo Valsorda (independent maintainer, sponsorship-funded; previously crypto lead at Cloudflare and on the Go standard library) and Ben Cox.

Fits: encrypting a single file or directory to send to someone; encrypting CI/CD secrets (sops integrates with age); encrypting backups for archival; replacing every old “I use gpg -c for this” workflow.

Political leaning: post-PGP modernist. The age project is implicitly an indictment of GnuPG and the FSF-era cryptographic UX. Filippo’s writing makes this explicit: GPG is a “legacy” tool, age does “one thing brilliantly.” Stylistically and ideologically, age sits in the Go-ecosystem, Stripe-Cloudflare, modern-minimalist-crypto school. Funded by independent sponsorships from companies like Ava Labs, which gives it a more commercial-tech-adjacent profile than the FSF crowd would prefer. Reproducible builds attested via Sigsum, which is a real distinguisher from the older tools.

Current state: 1.3.0 (December 2025) added an X25519+ML-KEM-768 hybrid post-quantum recipient. Builds are reproducible and Sigsum-attested. A fully compatible Rust port, rage, reads and writes the same file and key format, for anyone who prefers a single static binary or the Rust toolchain.

Tradeoff: deliberately doesn’t sign. If you need authenticated provenance (“this file came from me”), you combine age with signify, minisign, or ssh-keygen -Y sign. There’s no built-in revocation story.

Use if: you want to send an encrypted file or directory to someone, or you want to encrypt one for yourself and do not already keep a GPG key. If you do keep a GPG key for pass and signing, encrypting your own at-rest files to that key consolidates onto one key you already protect; see the GnuPG entry. This is still the right default for “I want to encrypt this thing” when no GPG key is already in the picture. For the conceptual comparison of age versus GPG, what each does cryptographically and why age drops signing and the web of trust, see gpg-concepts.md.

GnuPG / GPG

The old guard. The canonical implementation of the OpenPGP standard. Encryption, signing, key management, web of trust, key servers, smartcard integration. It does everything; it does most of it badly by 2026 UX standards; and it remains essential because so many existing systems (apt signing, git commit signing, legacy email encryption, OS package distribution) depend on it.

Maintainer: Werner Koch and the GnuPG team. The funding situation has been precarious at multiple points, with bursts of corporate sponsorship after each near-collapse becomes news.

Fits: legacy interop, verifying package signatures, signing git commits, encrypting email under PGP/MIME, anything that already speaks OpenPGP. Also encrypting your own files at rest when you already keep a GPG key for pass and signing, where using that one key avoids adding a second encryption tool and a second key to back up. Rarely the right starting point for a new file-encryption workflow that has no GPG key behind it already.

Political leaning: FSF-adjacent cypherpunk old guard. GPL-licensed. Built around the web-of-trust philosophy of the 1990s, that key authenticity should be established peer-to-peer rather than via certificate authorities; gpg-concepts.md covers how the web of trust actually works and why it never scaled. The project’s institutional rhythm is famously slow, the UX is famously hostile, and the codebase is famously baroque. Modernizers (age, Sequoia-PGP) treat it as the cautionary tale.

Tradeoff: the worst available choice for someone with a free hand and no existing GPG key, since age is the cleaner file primitive; the sensible choice when you already hold a GPG key and want one key to cover signing, pass, and your own files at rest.

Use if: you need to interoperate with an existing OpenPGP system, or you already keep a GPG key for pass and signing and want to encrypt your own at-rest files to it rather than maintain a separate age identity. To send a file to someone who does not already use GPG, age.

Picocrypt

A very small, very simple, GUI-first file encryption tool. Drag a file in, set a password, get an encrypted blob out. Cross-platform, written in Go, distributed as a single binary that needs no install. Uses XChaCha20 with Argon2id key derivation. Optional Reed-Solomon error correction for long-term archival. A “paranoid pack” includes reproducible builds, source archives, and backup binaries.

Maintainer: Evan Su (single developer).

Fits: occasional encryption of one or several files via a GUI. The “I just want to put a password on this PDF and email it to my accountant” use case.

Political leaning: minimalist tooling with cypherpunk-flavored populism. The cryptographic primitives are modern (XChaCha20, Argon2id) and the interface is austere in the age tradition, but the marketing copy invokes “three-letter agencies like the NSA”, that’s a cultural tell about who the project imagines its user is. Sits between age (austere, developer-focused) and VeraCrypt (heavy, cypherpunk-traditional). The “paranoid pack” with reproducible builds is a supply-chain-trust marker shared with age and LUKSbox.

Tradeoff: single-developer project. Limited CLI; the workflow really wants the GUI. Not designed for cloud sync, each operation produces a static encrypted file.

Use if: a non-technical relative needs to put a password on a file and you want them to succeed on the first try.

Tomb

A shell script wrapping LUKS to give you “tombs”, .tomb files that act like portable encrypted folders. Created by the dyne.org foundation, the same Italian hacker collective that has been making Free Software for two decades. The whole tool is a few thousand lines of Zsh.

Maintainer: Denis “Jaromil” Roio at Dyne.org. Active since 2007; recent FIDO2 security key support added.

Fits: an encrypted vault on a Linux box, with a workflow simpler than VeraCrypt and more discoverable than raw cryptsetup. Steganographic key hiding (in JPEG images) is supported.

Political leaning: explicitly hacker-collective and autonomist-leaning. Dyne.org’s other projects (Devuan-friendly tooling, dyne:bolic, free-culture audio software) sit in the European free-software-as-political-practice tradition. Of all the tools on this list, Tomb is the most explicitly ideological: the project’s documentation and its surrounding community make no secret of where they sit politically.

Tradeoff: Linux-only. Bash- and Zsh-based, which some find fragile. CLI-first; GUI wrappers exist but vary in quality.

Use if: you want a Linux-only LUKS-backed vault with a friendlier workflow than raw cryptsetup, and you’re sympathetic to the dyne.org political register.

gocryptfs

A FUSE-based encrypted overlay filesystem. You point it at a directory of encrypted files, give it a password, and it mounts a virtual directory where the files appear in plaintext. Per-file encryption with encrypted filenames. Designed as the modern successor to EncFS, which had known cryptographic weaknesses.

Maintainer: Jakob Unterwurzacher.

Fits: encrypted home directory or per-user encrypted folders on Linux/macOS; cloud sync where you want each file individually encrypted (similar use case to Cryptomator, but as a CLI/FUSE primitive rather than a polished app).

Political leaning: engineering-pragmatist; no ideology. The “do EncFS again, but correctly” project.

Tradeoff: Linux/macOS only (FUSE-dependent). Not a sync tool itself; you bring your own.

Use if: you want Cryptomator-style per-file encryption but as a CLI primitive you can script.

ZFS native encryption

Filesystem-level encryption built into OpenZFS. Each ZFS dataset can be independently encrypted with its own key, using AES-256-GCM or AES-256-CCM. Encrypted datasets can be sent and received between machines without the destination ever seeing plaintext (zfs send -w).

Maintainer: OpenZFS project, originally Sun (now Oracle for the legacy Solaris fork; OpenZFS is the community fork that runs on Linux, FreeBSD, and macOS).

Fits: serious storage setups, TrueNAS systems, ZFS-based backup destinations, large local pools where you want encryption granular to the dataset rather than the whole disk. Particularly strong for replicating encrypted backups across machines.

Political leaning: enterprise-storage-pragmatist. Originated at Sun, now driven by a mix of corporate (iXsystems, Lawrence Livermore) and community contributors. CDDL-licensed, which causes friction with the GPL Linux kernel and is itself a small political artifact. No threat-model ideology beyond getting filesystem semantics right.

Tradeoff: requires you to be on ZFS in the first place, which is its own significant decision. CDDL/GPL incompatibility means ZFS support on Linux ships out-of-tree and depends on DKMS or precompiled modules. Not appropriate for someone who just wants to encrypt a folder.

Use if: you’re already on ZFS or building a NAS / backup target and you want filesystem-native encryption that supports zfs send -w to untrusted destinations.

rclone crypt

Rclone’s encrypted backend. Rclone is “rsync for cloud storage” and supports something like 70+ providers; the crypt backend wraps any of them in transparent client-side encryption. Filename encryption is optional.

Maintainer: Nick Craig-Wood.

Fits: encrypted sync to almost any cloud storage backend you can name. Often paired with Restic or Borg for the backup-format layer.

Political leaning: pragmatist sysadmin. Trust nothing in particular, use anything you have to.

Tradeoff: rclone itself has a learning curve. The crypt layer is straightforward once rclone is set up.

Use if: your backup or sync destination is a cloud provider rclone speaks and you want client-side encryption layered on top.

Mentions and exclusions

EncFS is deprecated due to known cryptographic weaknesses. Use gocryptfs.

CryFS and securefs are alternative FUSE-based encrypted filesystems. CryFS chunks everything into fixed-size blocks to hide file sizes; securefs is smaller and less actively maintained. Either is usable; for most people gocryptfs has more momentum.

7-Zip with AES is encryption tacked on top of an archiver and has had crypto bugs in past versions. Functional for casual use, not crypto-first design.

eCryptfs (per-directory encryption that Ubuntu used to ship for ~/.private) is essentially abandoned. fscrypt (kernel-native per-directory encryption in ext4 / f2fs / UBIFS) is the current Linux successor and is fine if you want native kernel support without FUSE. Like ZFS encryption, it’s filesystem-native rather than fitting cleanly into the four families above.

Sequoia-PGP is a Rust rewrite of OpenPGP, German-funded, the most credible modernization attempt for the GnuPG ecosystem. Worth knowing about if you’re stuck with OpenPGP for legacy reasons but want a less hostile codebase. Not a starting point for someone who has a free choice, pick age instead.

Kryptor is a single-developer age-like project with solid cryptography but a much smaller user base. Mentioned for completeness; not recommended over age.

DiskCryptor was a Windows-only TrueCrypt-era full-disk encryption tool. Abandoned; ignore older recommendations for it.

BitLocker (Windows) and FileVault (macOS) are excluded from the comparison because they’re proprietary and OS-vendor-controlled. See the “Why proprietary disk encryption is hostile” section above for what’s wrong with them. Section 6 covers what to do with existing BitLocker / FileVault volumes when migrating to Linux.

Decision matrix

Pick the row that describes your use case. The recommended tool is in column two; alternatives are in column three.

The last row is for completeness: signing and encryption are different operations. If your goal is signing, GnuPG and ssh-keygen -Y sign are the practical options; the rest of this guide is about encryption. Why the two operations differ, and what authenticity and integrity each give you, is covered in gpg-concepts.md.

Two pieces of practical advice not captured in the matrix.

First: full-disk encryption on the system you actually use is the highest-value action by a wide margin. Everything else here addresses narrower problems. A stolen unencrypted laptop is the typical real-world threat; LUKS solves it in one checkbox. If you’re still on Windows or Mac, the answer is the same as the answer everywhere else in this guide: migrate.

Second: don’t try to learn all of these. Pick one tool per job and stick with it. The most common failure mode in this space is people stacking three encryption layers, forgetting one of the passwords two years later, and losing the data permanently. Encryption is the most reliable way to lose your own data. Account for that in the choice.

Political leaning summary

Encryption-at-rest tools cluster into several ideological lineages. Knowing which one your tool belongs to predicts its threat model, its funding sustainability, and how it’s likely to behave under stress.

Linux-kernel mainstream. LUKS, fscrypt. Engineering-above-ideology, ships in every distribution, gets used at industrial scale, has no political enemies to make. The least flashy category and by far the most reliable.

Self-hosting sysadmin. rclone crypt, gocryptfs, ZFS native encryption. FOSS-pragmatist by temperament, but with a discernible preference for “your own server, your own disks” deployment patterns. Less ideological than the cypherpunks but more opinionated than pure infrastructure-as-utility. Stable funding because these tools are the backbone of competent self-hosting. The backup tools that also sit in this lineage (Borg, Restic) are covered in choosing-backup-tools.md.

Cypherpunk / post-Snowden state-resistant. VeraCrypt, GnuPG. Built around the assumption that adversaries include nation-states. Hidden volumes, web of trust, paranoid threat models, sometimes-difficult UX as a feature rather than a bug. Currently the most fragile category, the VeraCrypt-Microsoft situation in early 2026 is a state-resistant tool being shut down not by a state but by a platform vendor, and the lesson is that even hardcore cypherpunk tools depend on cooperation from the platforms they ship through.

Modern minimalist / post-PGP. age, with Sequoia-PGP as a sibling on the OpenPGP-modernization side. A reaction against GnuPG complexity by a younger generation of cryptographers. Smaller maintainer pools, smaller surface area, far better ergonomics, less institutional inertia. Funded by individual sponsorships and tech-company patronage rather than foundations. Reproducible builds and signed-binary attestation (Sigsum, transparency logs) are markers of this lineage.

Cypherpunk-flavored populism. Picocrypt. Modern primitives, austere interface, but explicitly populist threat-model framing, the marketing names the NSA. Sits between modernist tooling and the older cypherpunk culture.

Modernist-cypherpunk hybrid. LUKSbox. Cypherpunk threat model (post-quantum, hardware-key, detached header for plausible deniability) implemented with modernist tooling (Rust, Apache 2.0, fuzzing-and-audit-first development, cross-platform via FUSE3 / WinFsp). The first tool on this list to combine those two camps deliberately. Funded by a French pentesting firm rather than a maintainer-of-one or a privacy-product company.

Eurocentric privacy-commerce. Cryptomator. Built for the user who already uses Dropbox and wants the provider not to read the files. Freemium business model, professional German company, audited, GDPR-shaped. The most commercially mature project on the list and the easiest to recommend to non-technical people. The politics aren’t absent; they’re “European data-protection law as a market position,” which is its own coherent stance.

Hacker-collective / autonomist. Tomb (Dyne.org). The most explicitly political category. European free-software-as-political-practice tradition. Bash-script aesthetics, KISS philosophy, leftist hacker culture.

The takeaway: the encryption itself is not the interesting variable. AES-256, ChaCha20-Poly1305, Argon2id, these are mature primitives and any of the actively maintained tools above gets the math right. What differs is the threat model the project assumes, the funding base that sustains it, and the behaviors you’re being signed up for when you adopt their workflow. Pick on those.

Migration from Windows or Mac

The companion OS guide makes the case for leaving in detail. The encryption story is one piece of that case. Here’s how to handle the encryption layer during the migration itself.

If you’re on Windows with BitLocker. Decrypt the BitLocker drive before migration: Settings → Privacy & security → Device encryption → Off, or manage-bde -off from an admin command prompt. Wait for decryption to complete, back up the unencrypted contents to an external drive, install Linux fresh with LUKS enabled at install time, copy the data back. Don’t try to dual-boot a BitLocker-encrypted Windows partition alongside a LUKS-encrypted Linux partition with shared data, the boot complexity isn’t worth it.

If you can’t or won’t decrypt the original BitLocker drive (you’re keeping Windows as a fallback during transition, or the drive is read-only legacy), dislocker on Linux can mount BitLocker volumes read-only with the recovery key. Use it for one-time data extraction; not a long-term workflow.

For cross-platform portable drives going forward, use VeraCrypt today; LUKSbox once it reaches 1.0. Don’t use BitLocker To Go, Microsoft uploads recovery keys to your account.

For files that were in OneDrive: assume Microsoft has read them. If they’re sensitive, treat them as compromised and rotate any credentials they contained. Going forward, Cryptomator on top of OneDrive (or a switch to Proton Drive / Tresorit / self-hosted Nextcloud) closes the leak.

If you’re on macOS with FileVault. Same pattern: decrypt FileVault before migration (System Settings → Privacy & Security → FileVault → Turn Off), back up cleanly, install Linux fresh with LUKS.

apfs-fuse on Linux can read APFS volumes including FileVault-encrypted ones with the password, but it’s read-only in most configurations and not maintained by Apple. Use it for one-time data extraction.

For files that were in iCloud Drive: check whether you had Advanced Data Protection turned on. If yes, your data was end-to-end encrypted and Apple couldn’t read it, clean migration. If no (the default), Apple held the keys; assume any sensitive content was readable to Apple and treat it as compromised. Move to Proton Drive, Cryptomator over your sync provider of choice, or self-hosted storage going forward.

General advice for both directions. Decrypt and copy is almost always cleaner than trying to read foreign-OS encrypted volumes natively. Schedule the migration when you have time to sit through a full decrypt-backup-reinstall-restore cycle, ideally with two separate copies of the data on different physical drives. Encryption migration is the most common point at which people lose data permanently; budget extra paranoia for it.

Once you’re on Linux. Default starting setup: LUKS at install on the system disk, plus one secondary tool for whichever specific job is in front of you (Borg for backups, Cryptomator for cloud-synced folders, age for one-off files). Add others only when a new job genuinely requires them. The companion devuan-luks2-install.sh covers the LUKS-LVM-Devuan install procedure in detail for users who want full control over the install.

How to think about choosing

Three questions, in order:

1. What threat are you defending against? Laptop theft → whole-disk. Cloud-provider snooping → per-file or container. Compelled disclosure → hidden volumes (rare; high stakes). Long-term archive integrity → backup tool with authenticated encryption.
2. Who else needs to read the data, and on what? Just you, on Linux → most options work. You and your future Mac or your Windows colleague → cross-platform (VeraCrypt, LUKSbox once 1.0+, Cryptomator, age, Picocrypt). A specific other person → age (modern) or GnuPG (legacy).
3. How much UX friction can you absorb? Zero → LUKS (set it once at install, never see it again). Low → Cryptomator. Medium → VeraCrypt, LUKSbox, Picocrypt, Tomb. Higher but more powerful → age, Borg, Restic, rclone.

A fourth consideration that’s not really a question but matters more than any of the above: don’t lose your keys. Encryption is the most reliable way to permanently lose your own data. The typical failure looks like this: someone enables full-disk encryption, picks a strong password, doesn’t write it down (because writing down a password feels insecure), uses the laptop for two years, takes a long break, comes back to a locked screen and a blanked memory. There is no recovery from that.

Concrete countermeasures, in increasing order of paranoia:

• Write the password down on paper and put the paper somewhere a thief wouldn’t look but you would (a sealed envelope in a filing cabinet, a safe-deposit box, a trusted family member’s house). Physical paper is not a meaningful threat surface for the attacks encryption defends against, and it’s the most reliable backup medium humans have for short secrets.
• Use a password manager (KeePassXC, Bitwarden) and back up its database, but the master password itself has to live somewhere outside the password manager, so this just shifts the problem.
• For LUKS specifically, save the LUKS header to a separate medium. If the header gets corrupted on the disk, the data is unrecoverable even with the correct password; the backup header restores recoverability. The command and the full procedure live in devuan-secure-workstation.md and the devuan-luks2-install.sh script, not here.
• For backups, keep at least two physically separated copies of any encrypted archive plus the password to decrypt it. The backup is no good if the password died with the laptop.

Most people who go from “no encryption” to a single working LUKS-plus-Borg setup, with the password written down in a sealed envelope at home and the LUKS header backed up to a USB stick in a different room, get more real-world security than people who spend a year reading about hidden volumes and never finish setting anything up.

Choosing Communication Tools, v9

How to choose a messenger or email setup when the goal isn’t “what does my contact use” but “what threat model does the tool actually defend against.” Covers centrally-coordinated messengers, federated networks, decentralized cryptography, P2P and offline-capable messengers, Nostr-rooted messaging, radio-grade off-grid messaging, and (in a dedicated second half) the email landscape: encrypted-mailbox providers, privacy-respecting standard providers, self-hosted mail, encryption layers, clients, and aliasing.

This doc complements choosing-networking-tools.md (the L3/L4 networking layer) by covering the application-layer messaging and email space. The two intersect at Briar and at Reticulum’s LXMF: both projects appear in both docs but with different framings.

TL;DR

The right default messenger depends entirely on your contacts. The right default messenger for sovereignty-minded readers who can influence their contacts is Signal for mass-market reach, with SimpleX as the sovereignty-aligned upgrade for the contacts who’ll move with you.

Beyond the default:

1. Mass-market, US-jurisdiction-tolerant: Signal.
2. Mass-market, US-jurisdiction-averse: SimpleX or Threema.
3. Tor-only, metadata-resistant: Cwtch.
4. Offline mesh capability: bitchat for Bluetooth-mesh on iOS and Android (Dorsey’s sovereignty-aligned project, deployed at scale in Uganda’s 2026 election), Briar for the Bluetooth-plus-Tor case on Android, Reticulum/LXMF for the radio-grade case.
5. Self-sovereign keypair identity over Nostr with mature forward-secrecy: White Noise (MLS via the Marmot protocol).
6. Self-sovereign keypair identity over Nostr with broader client ecosystem: Damus, Amethyst, 0xchat via NIP-17.
7. Federation-preferring: Matrix via Element / Element X, or XMPP via Conversations.
8. Voice and video focus: Jami.
9. Email-bridge: DeltaChat for users who already have email and want PGP-grade encryption without learning a new app.

For email specifically (the second half of this doc), the ordering is by how much you can verify rather than how polished the product is. The open-source-and-auditable endpoint is self-hosting: Stalwart or mailcow on a clean-IP VPS is the only configuration where you control and can inspect the whole stack instead of trusting a provider’s server. Hosted providers are the convenience tier: Posteo or Mailbox.org for client-free standard IMAP/SMTP on your own domain, then Proton Mail or Tuta for encrypted-at-rest mailboxes (open clients, closed servers, single-jurisdiction court-compulsion exposure). Thunderbird or nmail for clients; SimpleLogin or addy.io for aliasing; own a custom domain early so changing providers never changes your address, which is what makes the climb toward self-hosting possible at all. Be wary of newer entrants: AtomicMail is too new and not fully open to recommend, and Skiff shows why, bought and shut down inside four years. The honest baseline, stated once and meant: email leaks metadata by design, so for genuinely sensitive communication use a messenger from the first half, not email.

Avoid: WhatsApp, Telegram (the default-unencrypted product, not the Secret Chats), iMessage cross-platform, Discord for anything you care about, Wickr (Amazon-owned, end-of-life uncertain). For email: Gmail, Outlook / Microsoft 365, Yahoo for anything sensitive.

The capture-risk frame for messengers

Three architectural questions structure the messenger landscape:

What identifies the user? Phone number (Signal, WhatsApp, Telegram), email (Wire, Threema for backup), username (Threema, Matrix, XMPP), random ID (Session, Briar, Cwtch), no identifier at all (SimpleX), or a cryptographic keypair generated locally (Nostr clients including White Noise, Reticulum).

Who routes the messages? A single company (Signal, Threema, WhatsApp), a federation of servers (Matrix, XMPP, DeltaChat over email), a peer-to-peer overlay (Briar, Cwtch, Berty, Jami), a content-addressed network of relays (Nostr, SimpleX), or volunteer-run radio infrastructure (Reticulum, Meshtastic).

What’s the company’s incentive? Subscription revenue (Threema, ProtonMail), donations and grants (Signal Foundation, Briar, Tor), VC-funded growth toward an exit (no current example in this list survives this filter), or commercial-and-consortium hybrid (SimpleX is building toward this with its 2026 Foundation/Consortium launch¹).

The combination of those three answers is the messenger’s threat model. Signal answers “phone number + Signal Foundation servers + donations”, strong cryptography, real funding model, US-jurisdiction exposure. SimpleX answers “no identifier + your-choice-of-relays + commercial + foundation”, eliminates the metadata that the others handle imperfectly. White Noise answers “Nostr keypair + your-choice-of-relays + MLS encryption + Bitcoin-community-funded”, sovereignty by construction at the cost of UX maturity. Nostr DMs (NIP-17) answer the same as White Noise but with weaker forward-secrecy and no native multi-device.

Two more lenses run through both halves of this doc. The first is that open source is a floor, not a bonus: a tool whose source you cannot read is a tool whose behavior you are taking on faith, so fully-open-source-and-auditable tools are prioritized here, partially-open tools (open client and closed server, or open stack and closed app) are named as exactly that, and closed tools are flagged. Open source is necessary but not sufficient, because for any hosted service you still cannot verify that the server runs the code it publishes, which is why the sovereign endpoint in both halves is something you run yourself rather than a provider you trust. The second lens is that longevity is itself a security property: a tool that has operated for a decade through funding scares and legal pressure has demonstrated resilience a six-month-old startup cannot, and the VC-funded-toward-an-exit pattern named above is the specific failure mode to fear. Skiff, in the Email section, is the worked example, a polished, partly-open, well-funded privacy startup acquired and shut down inside four years, stranding its users with no clean migration path. New is not disqualifying, and several of the strongest entries here are recent, but new is unproven, so unproven tools carry a maturity stamp here rather than a recommendation.

A note on MLS (Messaging Layer Security)

MLS is the IETF-standardized group messaging cryptography (RFC 9420, July 2023). It provides forward secrecy (compromise of current keys doesn’t expose past messages), post-compromise security (compromise self-heals as the ratchet advances), and end-to-end encrypted groups that scale to thousands of members without rekeying every pair separately. Apple, Google, Mozilla, Cisco, Wire, and Wickr were among the participants in standardization.

Adoption in 2026: Wire was the early production deployment; Element X is migrating Matrix from Olm/Megolm to MLS; White Noise is the first Nostr-native MLS messenger via the Marmot protocol; the IETF MIMI working group is building cross-network MLS interop on top. MLS is the trajectory most serious-use messengers are converging toward.

The MIMI (More Instant Messaging Interoperability) IETF working group is the cross-network interop layer being built on top of MLS. Four drafts are active as of early 2026: draft-ietf-mimi-arch (the architecture), draft-ietf-mimi-protocol (the MIMI-over-HTTPS-and-MLS transport spec), draft-ietf-mimi-room-policy (room policies for groups and multimedia conferences), and draft-ietf-mimi-content (the message-content format)². Authors include Richard Barnes (Cisco), Matthew Hodgson and Travis Ralston (Matrix.org Foundation), Konrad Kohbrok and Raphael Robert (Phoenix R&D), and Rohan Mahy. The Matrix-side participation is heavy; Element X’s MLS migration is the most visible implementation track. What MIMI lets you do at the spec level once it lands: a user on provider A and a user on provider B exchange E2E messages without either provider operating a bridge. The bridge becomes a protocol, not a piece of infrastructure. Trajectory for the next 18 to 24 months; not deployable yet.

Tier 1: Centrally-coordinated, mass-market

The “one company, professional ops, strong crypto” tier. The right answer when reach matters and the threat model tolerates trusting a single well-resourced operator.

Signal

US-based, Signal Foundation governance, Signal Technology Foundation legal entity. Founded by Moxie Marlinspike, current president Meredith Whittaker since 2022. End-to-end encryption via the Signal Protocol (Double Ratchet, X3DH, post-quantum extension via PQXDH since 2023), which is the cryptography most other secure messengers also use under the hood (WhatsApp, the Skype business of Microsoft, Google RCS). Open-source clients on every major platform; server source-available with some lag relative to deployment.

Phone-number identity is the recurring flag. Sign-up requires a phone number; the number is also your visible identifier to contacts by default. Username support shipped in 2024 to mask the phone number from new contacts but the number remains the account anchor. SIM-swap attacks are real against any phone-rooted identity system.

US-jurisdiction shape: Signal Foundation is a US 501(c)(3); the legal compulsion model is “the FBI asks, the Foundation provides what’s technically possible to provide,” which is by design very little (account creation date, last connection date, no message content, no contact list). The 2016 Eastern District of Virginia subpoena response is the canonical example: the Foundation provided account-creation timestamp and last-connection timestamp; that was all it had to provide.

Community-politics flag: Meredith Whittaker’s public political alignments are explicit and well-known (anti-corporate AI surveillance, AI Now Institute background); how much weight that carries varies by reader. The technical posture of Signal under her presidency has remained strong (post-quantum protocol upgrade, username feature, no telemetry-creep, no monetization-creep).

Pick Signal as the default mass-market messenger if: your contacts will install it (the largest “your contacts will install it” surface area of any privacy messenger by an order of magnitude), and you understand the phone-number flag. The cryptography is genuinely strong; the structural risks are jurisdiction and phone-rooted identity, not the crypto.

Threema

Swiss, founded 2012, paid (one-time purchase, currently around €4.99). Identity is a Threema ID, eight characters, randomly generated, no phone number, no email required at registration. Optional verification of email or phone for friend-finding only; not required.

Open-source clients since 2020; the server stack remains closed. Audited multiple times; cryptography is solid (NaCl-based, custom protocol). Used by the Swiss government for internal communications, which is a credible operational endorsement.

Capture-risk shape: single Swiss company, paid subscriber base. Swiss jurisdiction provides EU-level data protections plus a cultural reluctance to assist foreign-government subpoenas. Smaller user base than Signal by far.

Pick Threema if you want the no-phone-number property, you’re willing to pay (the payment is the user-not-product signal you actually want here), and your contacts will install it. Common in DACH-region (Germany, Austria, Switzerland) friend and family networks.

Tier 2: Federated

The “no single company can shut you off; multiple operators can interoperate” tier. The trade-off is more configuration and a thinner UX layer.

Matrix (via Element, Element X, or alternatives)

Federated network of servers (homeservers) running the Matrix protocol. Element (formerly Riot) is the reference client, made by Element Software (formerly New Vector), the same company that maintains the spec. Element X is the rewrite shipping since 2024, Rust core, native iOS and Android, designed around MLS as the encryption layer rather than the legacy Olm/Megolm pair. Matrix.org is the largest single homeserver; tens of thousands more exist across self-hosted and community deployments.

End-to-end encryption: Olm/Megolm in legacy Element; MLS-based migration underway in Element X with the goal of cross-net Mimi interop. Element X ships MLS-enabled rooms by default for new conversations on supported homeservers as of 2026. The migration story for existing rooms is incremental.

Capture-risk shape: federation gives you the right to leave any homeserver and take your identity (via cross-signing keys and exported backups). The reality is messier: if you’ve been on matrix.org and that homeserver goes down, you can rejoin rooms from a new server but the operational hassle is real. Element Software the company has had funding turbulence (2023-2024); the protocol survives the company.

Pick Matrix if: you want federation as a structural property, you have contacts already using it (Linux communities, security communities, FOSS projects), and the UX rough edges don’t deter you. Self-hosting your own homeserver (Synapse, Dendrite, or Conduit) is the sovereignty-aligned configuration.

XMPP (via Conversations or other clients)

The older federation protocol; an open IETF standard. Conversations (Android) is the reference modern client; Gajim (desktop) and Snikket (turnkey self-hosted server) round out the practical stack. OMEMO is the end-to-end encryption layer, based on the Signal Protocol.

Smaller user base than Matrix; older codebase; fewer feature creep risks. The protocol itself has been stable for two decades.

Pick XMPP if: you want a protocol that has outlasted multiple companies and several federation experiments, you’re comfortable picking your own server (or running one), and you don’t need the rich-media UX Matrix invests in. Some sovereignty-minded operators specifically prefer XMPP over Matrix on the “older, simpler, more outlasted-companies” axis.

Tier 3: Decentralized cryptography

The “no central party knows the topology, even if the relay set is fixed” tier.

SimpleX Chat

The most architecturally interesting current entry. No user identifiers of any kind, not even random ones³. Conversations are routed via temporary pairwise queue identifiers; each conversation typically uses two different relay servers chosen by the participants. Your contact list isn’t on any server because there’s no server-side notion of “you.” Tor support built-in for IP-address protection.

Cryptography is end-to-end with quantum-resistant key agreement and the Double Ratchet for ratcheting. Trail of Bits cryptographic design review in 2024⁴; reproducible builds; clients on iOS, Android, Windows, macOS, Linux (Flatpak, AppImage, .deb, console).

Current version v6.5 (April 30, 2026). The April 2026 release announced SimpleX Channels (publishing with participation privacy: channel content visible to relays but participants and authors anonymous to relays and each other) and the SimpleX Network Foundation and SimpleX Network Consortium structure, a Foundation that holds the protocol IP and a Consortium agreement between Foundation and SimpleX Chat Ltd that survives the company being sold or shut down¹. This is the governance structure the sovereignty frame asks for.

Capture-risk shape: the architecture eliminates the user-identifier capture risk by construction. The relay operators still see traffic patterns (encrypted content, but timing and volume); the Tor option closes the IP-address leak. The Foundation/Consortium structure addresses the “what if the company is bought” question explicitly.

Pick SimpleX if: you can persuade your contacts to install it (smaller than Signal but growing), you want the strongest current architectural privacy guarantees in a usable mass-market-style app, and you’re aligned with the structural choices the Foundation is making.

Session

Loki/Oxen Network-based, no phone number, no email, no central server. Forked from Signal in 2019; uses the Session Protocol (a modified Signal Protocol that drops forward-secrecy in exchange for asynchronous delivery without a central server). Routes through the Oxen service node network using Lokinet onion routing.

Australian-headquartered (Session was originally a Loki Project initiative). The Oxen Network has had token-economy turbulence over the years.

Pick Session if: you want a fully decentralized routing model, you accept the forward-secrecy trade for the offline-delivery property, and the Oxen Network’s continued operation matches your time horizon. Smaller user base than SimpleX; less active development as of 2026.

Tier 4: P2P and offline-capable

The “no internet required” tier. Either pure peer-to-peer or with internet as one transport among several.

Briar

P2P messenger built around Tor for the internet case plus Bluetooth and WiFi-direct for the offline-mesh case. Messages can also be transferred via USB stick (the “censorship-resistant” use case the project explicitly designs for).

Android-first; desktop clients (Windows, macOS, Linux) exist; Linux mobile support in beta. No accounts, no phone numbers, no email; identity is a cryptographic keypair generated on the device.

The unique property: works without any internet connectivity at all. Two Briar users in the same building can communicate over Bluetooth indefinitely. Two users in adjacent buildings can communicate over WiFi-direct. Two users in the same city with no internet can relay messages via a third user who’s seen them both.

Status flag: last stable release v1.5.9 (January 2024). Active development continues per the project’s GitLab but release velocity has slowed. The protocol and architecture remain sound; treat as production for the use case it solves but watch the project’s health.

Pick Briar if: your threat model includes “the internet might go away” (protest scenarios, infrastructure disruption, hostile-state censorship), or you specifically want offline-mesh capability in your standard messenger stack. Pair with Reticulum/LXMF (Tier 7) for the radio-grade extension of the same idea.

Cwtch

Tor-only messenger built by Open Privacy Research Society (Sarah Jamie Lewis and contributors). Every conversation runs over Tor v3 onion services; no central server, no Cwtch service to compromise. Metadata-resistant: the protocol is designed so even traffic analysis between peers can’t reveal who’s talking to whom.

Smaller user base than Briar; cleaner architecture in some respects (no Bluetooth/WiFi-direct complexity). Desktop and Android clients.

Pick Cwtch if: your threat model is “metadata is the threat” (journalists protecting sources, activists in hostile jurisdictions, anyone whose social graph is the sensitive data) and you accept that everyone in your conversation must also be on Cwtch.

Jami

P2P SIP-based messenger and voice/video conferencing tool by Savoir-faire Linux (Montréal), free software, GPL-3. The closest thing to “Signal for voice and video with no central server.” Identity is a cryptographic keypair (RingID); discovery via OpenDHT (a distributed hash table); audio and video routed peer-to-peer where possible, with optional TURN-relay fallback you can self-host.

Capture-risk shape: no central server at all. Savoir-faire Linux maintains the software and runs default DHT bootstrap nodes, but the company can be removed from the loop with self-hosted bootstrap. Available on Linux, Windows, macOS, Android, iOS, including Devuan native packages.

Pick Jami if: voice and video are first-class needs (not bolted onto a chat app), and you want serverless P2P architecture. Particularly useful for self-hosting in family or small-organization contexts where the user count is low enough that DHT discovery latency isn’t an issue.

Berty

P2P over Bluetooth and Tor, similar conceptual space to Briar but with iOS support (which Briar lacks). Identity is a cryptographic keypair. Smaller project; less mature than Briar.

Pick Berty over Briar if: you need iOS support. Otherwise Briar is the more established choice in this space.

Keet

Peer-to-peer messenger built by Holepunch and backed by Tether and Bitfinex, the companies behind the USDT stablecoin and the Bitfinex exchange. Built on the Pear Runtime and the Hypercore protocol stack: identity is a cryptographic keypair, peers locate each other through a distributed hash table, and there are no servers in the middle at all. End-to-end encrypted text, voice, and video, plus unlimited-size file transfer (files move device to device with no server to cap them), with integrated Bitcoin Lightning and USDT payments. Launched in 2022, downloaded millions of times, with a download surge through early 2026⁵.

Open-source status, stated precisely because it is the flag for this audience: the foundation is open source (the Pear Runtime, Hypercore, and Hyperswarm are published under permissive licenses and are reusable by any developer for any P2P app), but the Keet client application itself is not open source. NixOS packages it as unfree, and a 2022 promise to open-source the app has gone substantially unfulfilled as of 2026. So Keet is an open-source P2P stack wrapped in a closed-source client, which is weaker than Signal (open clients, source-available server) and weaker than the fully-open P2P messengers in this tier: Briar, Cwtch, and Jami are open source end to end.

Capture-risk shape: zero servers by construction, so no operator to compel; the residual concerns are the closed client (you cannot audit what the app does with your keys) and, as with any DHT-based system, that the network maps your public key to your IP address to route traffic, so your IP is visible to the peers you connect with unless you add a network-layer cover such as a VPN or Tor. Funding flag: Tether and Bitfinex backing is what frees Holepunch from chasing subscription or ad revenue, and is also a commercial crypto-conglomerate dependency a sovereignty-minded reader should weigh on its own terms.

Pick Keet if: you want serverless P2P text with high-quality voice and video, no account and no phone number, you are aligned with the Bitcoin and stablecoin ecosystem it is built around, and the closed client is a trade you will accept for the architecture, the unlimited file transfer, and the payments. If end-to-end open source is the priority, Jami covers the same serverless-P2P-with-voice-and-video ground fully open.

bitchat

Decentralized messaging over Bluetooth Low Energy mesh, with internet-connected geohash channels via Nostr as an optional second layer. End-to-end encryption via the Noise Protocol Framework (XX pattern), no servers, no accounts, no phone numbers, no email. Built by Jack Dorsey (Twitter co-founder, Block CEO, and one of the most visible Bitcoin advocates in the technology industry) under his “and Other Stuff” open-source development collective, with significant community contribution since⁶. Repositories: github.com/permissionlesstech/bitchat (iOS, Swift, Unlicense / public domain) and github.com/permissionlesstech/bitchat-android (Android, Kotlin, GPL-3). iOS v1.5.x and Android v1.7.x current as of early 2026.

Architecture. Each device acts as both client and server in a BLE mesh; messages hop up to seven times to reach recipients outside direct range (~30m per hop). End-to-end encryption uses Noise XX, which provides mutual authentication and forward secrecy. Identity is a Noise static Curve25519 keypair plus an Ed25519 signing keypair, generated on first launch and stored in the device keychain. A user’s verifiable fingerprint is the SHA-256 hash of their Noise static public key, readable aloud or scanned via QR for out-of-band verification. Messages live only in device memory by default and self-delete unless explicitly saved. Channel-based group chats (IRC-style /join, /msg, /who) with optional password protection; store-and-forward to offline peers; emergency wipe via triple-tap.

Hybrid Bluetooth-plus-Nostr design. Geohash channels (added in 2025-2026 development) use an internet connection to bridge Bluetooth-mesh-local conversations with peers in the same geographic area who aren’t in BLE range. This puts bitchat at the boundary between Tier 4 (offline mesh) and Tier 5 (Nostr-rooted): primarily a Bluetooth-mesh messenger, with Nostr as the internet-connected complementary layer when one is available. The Tor transport for Nostr is shipped in the Android client via the self-compiled arti library (Rust Tor implementation), reducing APK size meaningfully versus earlier embedded Tor builds. Background persistence on Android lets a device serve as a mesh relay even when the user isn’t actively in the app.

Adoption signal worth naming. Ahead of Uganda’s January 2026 general election, opposition leader Bobi Wine urged citizens to use bitchat to bypass anticipated internet shutdowns. Ugandan search interest for “bitchat” surged in the run-up to the election per Google Trends. The use case bitchat was architected for (protest communication when telecom infrastructure is throttled or shut down) is the use case it’s actually being deployed for at scale in 2026. TestFlight beta hit its 10,000-slot cap within hours of Dorsey’s July 2025 announcement; iOS App Store reach (25.1k GitHub stars on the iOS repo as of early 2026) gives bitchat a distribution surface that Briar’s Android-first reach can’t match. This is the rare case where a sovereignty-aligned messenger reached the mainstream-app-store audience without compromising the architecture.

Sovereignty frame. bitchat occupies the same architectural slot in messaging that nostr-vpn (choosing-networking-tools.md) occupies in networking: a Bitcoin-aligned developer applies the no-trusted-third-parties pattern to infrastructure that used to require centralized servers, ships open source, and lets the architecture speak. Dorsey’s funding via “and Other Stuff” rather than via VC or corporate roadmap puts the project structurally in the same class as Knots and as Malmi’s work. The two flags worth naming are the iOS-side closed-app-store distribution (same flag every iOS app carries; the source is open and Android builds from source are fully supported) and Block’s commercial position (Bitcoin financial-services company; Dorsey’s personal funding is what backs bitchat, not Block’s corporate strategy).

Capture-risk shape: zero by construction (no servers, no operator). Residual concerns are the iOS distribution channel and your trust in the binary build chain; both are addressable by building from source on Android.

Pick bitchat for: dense-Bluetooth scenarios where physical proximity is the use case (concerts, protests, dense urban areas, conference floors); jurisdictions where internet shutdowns are a real threat model (Uganda 2026 is the documented example; similar cases will recur); the Apple-ecosystem case where Briar’s Android-first reach doesn’t help; and any sovereignty-aligned stack where mainstream-app-store distribution matters for contact onboarding. Pair bitchat (Bluetooth mesh) with Briar (Bluetooth + WiFi-direct + Tor) and Reticulum + LXMF (LoRa mesh) for layered offline-capable messaging across physical-layer options.

Tier 5: Nostr-rooted

The newest application-layer messaging stack. Your identity is a secp256k1 keypair you generated; messages are signed and (for DMs) encrypted, then published to one or more Nostr relays that you choose. No accounts on any server; relays can be operated by anyone; you can run your own.

Two architectural sub-tiers inside Nostr-rooted messaging: the protocol-level DM specification (NIP-17, used by most current clients) and the MLS-based approach via the Marmot protocol (used by White Noise). The two have meaningfully different threat-model properties; pick the right one for your use case.

White Noise

The sovereignty-frontier Nostr messenger as of 2026⁷. Built by Erskin Gardner (erskingardner) and Max Hillebrand (founder of Sound Money Solutions); aligned with the Bitcoin community. Built on the Marmot protocol⁸, which sits on top of three primitives: Nostr (identity and signaling), Blossom (Nostr file hosting standard for media), and MLS (RFC 9420 Messaging Layer Security for the cryptography).

What MLS gets you that NIP-17 doesn’t:

• Forward secrecy. Compromise of current keys doesn’t expose past messages; the ratchet is per-message, not per-conversation-key.
• Post-compromise security. The ratchet self-heals after compromise; future messages become inaccessible to an attacker who got the current key but doesn’t keep getting them.
• Native multi-device. MLS treats each device as a leaf node in the group; you can join the same conversation from phone, laptop, and tablet without retransmitting messages to each pair-key separately. Each device holds its own keys; nothing about the multi-device support requires uploading a master key to any server.
• Scaled group chat. MLS was designed for groups of thousands without rekeying every pair separately on each membership change. Native group support that NIP-17 lacks.

Sender-recipient unlinkability over relays: Marmot wraps the MLS messages in Nostr events in a way that hides the actual participants from relay operators, similar to NIP-17’s gift-wrapping but with MLS as the cryptographic floor instead of NIP-44.

Status: alpha. Mobile app via Apple TestFlight (Android paths under iteration); desktop builds available. Open-source alpha; the team explicitly requests audits and community feedback. Production-ready nowhere near; architecturally the most advanced Nostr messenger by some distance.

Pick White Noise if: you want forward secrecy and multi-device on Nostr-rooted identity, you can persuade your contacts to install an alpha-grade messenger, and you’re aligned with the Bitcoin community’s sovereignty frame. The architecture is what to learn from; deployment for daily use comes when the alpha stabilizes.

Nostr DMs (NIP-17) and the client landscape

The protocol-level direct-message specification has gone through two NIPs (Nostr Implementation Possibilities):

• NIP-04: the original DM spec. Encrypted but leaked metadata (sender and recipient pubkeys visible to relays, plus message timestamps and event IDs). Deprecated.
• NIP-17: gift-wrapped DMs⁹. The current spec. Three layers: a sealed event (NIP-59) wraps the actual message (NIP-44 encrypted) inside an unrelated-looking gift-wrap event with random keys. Relays see only the gift-wrap; sender and recipient identities and the timestamp are hidden from relay operators.

Most modern Nostr clients support NIP-17. The trade-offs versus Signal-style messengers (and versus White Noise specifically): no forward secrecy (NIP-44 uses long-term keys, not ratcheting) and no native multi-device key sync. Multi-device requires either sharing your nsec (the secret half of your Nostr keypair) across devices, or using remote-signing protocols like NIP-46 / NIP-49¹⁰ to keep the key in one place.

Client landscape:

• Damus (iOS, macOS): the most-polished iOS Nostr client. Built by William Casarin.
• Amethyst (Android): the most-featureful Android Nostr client. Built by Vitor Pamplona.
• Iris (web, Android): built by Martti Malmi (the same nostr-vpn author from choosing-networking-tools.md). Web-first; reasonable mobile experience.
• 0xchat (iOS, Android, desktop): privacy-focused Nostr client, NIP-17 DMs as default, recently added group-chat support.
• Primal (iOS, Android, web): UX-polished cross-platform client; integrated Lightning wallet for zaps; sometimes feels closer to a Twitter-replacement than a messenger but the DM support is real.

Status

Ethereum-rooted decentralized messenger, peer of the Nostr-rooted projects in spirit but built on the Waku protocol (libp2p-based) and the Logos network rather than Nostr relays. Identity is a public-key derived address; no email or phone; integrated cryptocurrency wallet.

Status has been in development for years; the project’s positioning has shifted over time. Production-grade in 2026 but smaller user base than Nostr clients. Token-economic structure exists which is a flag for some sovereignty-minded operators (who prefer the Nostr-rooted projects’ relay-as-a-service model over token-incentivized infrastructure).

Pick Status if: you’re already Ethereum-aligned and prefer that protocol stack over Nostr; otherwise White Noise or NIP-17 clients are the more active sovereignty-frontier messengers.

Capture-risk shape (Nostr-rooted, all variants)

Zero by construction if the implementation is correct and you choose relays you trust. Your keypair is yours; relays you can swap out; messages are encrypted such that relays can’t read them. The remaining risk is the client itself (closed-source clients on closed-source app stores remain a closed-source-client-on-a-closed-source-app-store risk, the same as for any messenger).

What’s mature: protocol-level NIP-17 support across many clients; identity portability across clients via your single nsec; relay portability by changing your relay list. What’s young: White Noise as the production-grade MLS upgrade path; Marmot protocol standardization; cross-client UX for forward-secrecy expectations.

Adjacent frontier: Pubky (Synonym)

Not a messenger today, and not Nostr-rooted. This note is parked at the end of Tier 5 as a sovereignty-frontier watch entry beside the Nostr-rooted stack, because Pubky’s stated comparison target is Nostr and a reader weighing Nostr will meet Pubky’s claims.

Pubky is Synonym’s open protocol for key-based identity and public data publishing.¹¹ It replaces accounts with an Ed25519 keypair: your public key (the project calls it your pubky) doubles as a sovereign domain name.¹² PKARR (Public Key Addressable Resource Records) publishes small signed DNS-style records under that key to BitTorrent’s Mainline DHT, a network of over ten million nodes, and PKDNS resolves those records like a DNS server.¹² The record points at a homeserver: a conventional web server that stores your data per public key, grants apps write access only to the paths you approve, and serves everything under /pub/ publicly by default.¹¹ Moving providers means republishing one PKARR record; the project calls this credible exit, and it is the architectural answer to the discovery contrast below.

The user-facing pieces: Pubky Ring (iOS and Android key manager, MIT, v1.15 June 2026) holds the keypair and authorizes apps; pubky.app is the beta reference social app; Pubky Explorer browses what any key has published.¹² Signup to the flagship homeserver is gated by invite code, SMS verification, or a Bitcoin Lightning payment as anti-spam measures.¹¹ The SMS route links a phone number to the keypair, so anyone running an anonymous identity should take the invite or Lightning path, never SMS.

The Nostr contrast, which is the reason this note lives here. Nostr replicates your events across whichever relays you choose; portability is your nsec plus a relay list, and discovery means querying relays and hoping the right ones hold your data. Pubky keeps your data in one authoritative home and publishes a signed pointer to it on the DHT, so discovery is deterministic but availability hangs on a single homeserver until you migrate. Synonym CEO John Carvalho positions Pubky as “a strict upgrade” to Nostr on exactly this discovery axis; treat that as the vendor’s framing.¹³ What Nostr has that Pubky does not: a far larger client and relay ecosystem, NIP-17 encrypted DMs in daily use, and Lightning zaps. What Pubky has that Nostr does not: deterministic data location, and a DNS-replacement layer (PKDNS) that is useful beyond social.

Funding and capture-risk shape: Synonym Software was founded by Tether in November 2021, with Tether CTO Paolo Ardoino as Synonym’s CTO at launch, the same single-sponsor concentration the Keet entry above and the Holepunch/Pear note in choosing-networking-tools.md carry; Pubky also succeeded Synonym’s earlier Slashtags project, which ran on the same Hypercore stack Keet runs on.¹⁴ On the other side of the ledger everything is MIT-licensed open source with a Rust core, and homeservers are self-hostable, including a community Umbrel package.¹²

Status: beta and public-data-only. There is no end-to-end encryption yet; the project’s own FAQ states Pubky is currently optimized for public data, with private and encrypted features planned under Pubky Noise, which exists as an early Rust repository.¹¹ So today Pubky is an identity and public-publishing layer, not a messenger, and nothing sensitive belongs on a homeserver unless you encrypt it yourself first. The sovereignty-complete configuration is a self-hosted homeserver; using the flagship homeserver is trusting one operator, softened by the migration path. Revisit when Pubky Noise ships usable end-to-end encryption and signup ungates; until then this is a thing to try with throwaway data and to watch.

Tier 6: Email-as-transport

The “use the protocol every internet user already has” tier.

DeltaChat

A messenger UX over the email protocol. Sends and receives messages via standard IMAP/SMTP against any email provider. End-to-end encryption via Autocrypt (an OpenPGP profile designed for transparent in-band key exchange) by default with other DeltaChat users; falls back to ordinary unencrypted email with non-DeltaChat contacts.

The advantage: you can talk to anyone with an email address: they see an email; you see a chat. The disadvantage: the email infrastructure leaks the same metadata it always leaks (your provider sees who you talk to, when).

Pick DeltaChat if: you already have email, you want PGP-grade encryption without learning a separate app, and you have contacts who’ll install it. Particularly useful in mixed-tech-comfort family networks where “I’ll send you an email” works and “install this messenger” doesn’t. For the email infrastructure DeltaChat rides on (which provider, self-hosting, encryption layers) see the Email section below.

Tier 7: Radio-grade and off-grid

When the internet is unavailable and Bluetooth-range isn’t enough. Network-layer treatment of these tools lives in choosing-networking-tools.md; this section covers the messaging layer on top.

LXMF on Reticulum

Reticulum’s messaging format. Reticulum is the network stack (covered in the other doc); LXMF is the application-level protocol that runs on top of it for store-and-forward messaging. Pair with Nomadnet (terminal-based) or Sideband (mobile, desktop) for the user-facing app.

Threat model: a fully sovereign messenger that works over LoRa radios, WiFi mesh, serial links, or Tor, any transport Reticulum supports. Cryptographic identity, end-to-end encryption, sender identity hidden from intermediate relays.

Pick LXMF if: you’ve decided to operate a Reticulum-capable mesh (RNode hardware, LoRa radios) and you want a messenger that runs natively on it. Also pick this for “messaging the bunker scenario” use cases where IP infrastructure can’t be assumed.

Meshtastic messaging

Meshtastic’s built-in text messaging, over LoRa. Channel-based (groups of devices share a channel key). Floods messages across the mesh; works at small community-mesh scale (hiking groups, neighborhood resilience networks).

Pick Meshtastic messaging if: you’ve deployed Meshtastic hardware for the community-resilience reasons (cheaper than RNode, simpler protocol) and want messaging as one of the use cases. Not metadata-resistant in the way Reticulum is; not E2E-encrypted between specific pairs (channel-key based).

What to avoid

WhatsApp. Meta-owned, phone-number-rooted, metadata extensively logged and shared with Meta’s ad targeting graph. Cryptography (Signal Protocol) is fine; everything around the cryptography is the problem. The 2021 privacy-policy update explicitly authorized broader data sharing with Meta. Backups in iCloud/Google Drive are not E2E-encrypted by default.

Telegram for anything sensitive. Default chats are not end-to-end encrypted; only “Secret Chats” are, and they’re not the default mode. Group chats are never end-to-end encrypted regardless of mode. Pavel Durov’s 2024 arrest in France and Telegram’s subsequent policy adjustments on cooperation with law enforcement closed a chapter where Telegram could be framed as a privacy tool; it never was, and the marketing has caught up.

iMessage for cross-platform. Apple-only end-to-end encryption. When messaging to an Android user it falls back to SMS or RCS (RCS over Google’s network is also E2E now but the iMessage/RCS bridge had a turbulent rollout); the green-bubble cross-platform message often is not E2E. iCloud message backups are E2E only if Advanced Data Protection is explicitly enabled.

Discord for anything you care about. Not encrypted at rest from Discord’s perspective. Discord employees can read your messages. The terms of service authorize this.

Wickr. Acquired by Amazon in 2021; AWS Wickr is now the only continuing product; the consumer Wickr Me product was end-of-lifed. Don’t start new use on it.

Snapchat, Instagram DMs, X DMs (formerly Twitter). Not end-to-end encrypted by default; metadata extensively logged; the platforms’ business models depend on the data. X has experimented with E2E DMs in limited capacity but the rollout has been incomplete.

Where to start

Common scenarios.

“I want a default messenger to use with my phone contacts.” Signal. The crypto is strong, the user base is large, and persuading one contact at a time to install it is a tractable project. Accept the phone-number flag or use the username feature for new contacts.

“I want a sovereignty-aligned default with one or two contacts I can move with me.” SimpleX. Architecturally the strongest current mass-market-style messenger. The contacts who move with you will be the ones who care about the same things.

“I want a messenger that works when the internet is down.” bitchat for the Bluetooth-mesh case (iOS plus Android, App Store reach, deployed in Uganda 2026); Briar for the Bluetooth-plus-Tor-plus-WiFi-direct case (Android-first; slower release velocity); Reticulum with LXMF for the LoRa/radio case; Meshtastic for the community-LoRa case.

“I’m a journalist protecting sources.” Signal as the contact-acceptance default; SecureDrop for the actual source-onboarding flow; Cwtch for sources who are themselves metadata-sensitive. Read EFF’s SSD on this scenario specifically.

“I already use Nostr; I want my DMs to live there too, with forward secrecy and multi-device.” White Noise via the Marmot protocol. Accept that it’s alpha-grade; contribute back when you find rough edges.

“I already use Nostr and I’m fine with NIP-17 DMs for now.” Damus (iOS), Amethyst (Android), 0xchat (cross-platform), or Iris (web). Accept the no-forward-secrecy and no-native-multi-device trade-offs; plan to migrate to White Noise once it matures.

“I want federation as a structural property.” Matrix via Element X (the MLS-future-proofed client) or XMPP via Conversations. Self-host the server for the sovereignty-aligned configuration. Pair with a homeserver you operate (Synapse for Matrix, Snikket for XMPP).

“I want voice and video without a central server.” Jami. Self-hostable bootstrap nodes; no signup; works across desktop and mobile.

“I want PGP-grade encryption but my contacts will only use email.” DeltaChat. Both sides install it; both sides keep using their existing email; the encryption is automatic.

“I want every property at once.” You’re picking a Pareto frontier. The sovereignty-aligned stacked stack: White Noise as primary (when stable), SimpleX as the SMS/Signal-replacement secondary, Briar as offline backup, LXMF on Reticulum as the off-grid layer. None of these is Signal in terms of contact reach; the trade-off is real.

Maturity table

Email

Everything above is messaging. This second half covers email, which is a different animal and deserves its own frame.

Why email is structurally weaker than the messengers above

Email is a 1970s federated protocol. The store-and-forward design means the metadata (who you email, who emails you, timestamps, and the subject line in transit) leaks by construction. SMTP between servers is opportunistically encrypted (TLS if both ends support it, plaintext fallback otherwise), and the message sits decrypted on the receiving server unless you’ve layered encryption on top. End-to-end encryption (PGP) is bolted on, not native, and even when used it protects the body, not the metadata.

The realistic goal for email is therefore narrower than for the messengers: provider trust plus encryption-at-rest plus optional PGP for body content. You do not get the metadata elimination that SimpleX or the gift-wrapped Nostr DMs achieve. The honest baseline, stated once and meant: for genuinely sensitive communication, use a messenger from the first half of this doc, not email. Email is for the half of your life that has to interoperate with everyone else’s email, which is most of it, which is why this section exists.

The capture-risk frame for email asks three questions, same shape as the messenger frame: who can read the body (the provider, unless E2E), who can see the metadata (always the provider and the network, PGP doesn’t fix this), and what’s the provider’s incentive (subscription, donation, or advertising-against-your-content).

Encrypted-mailbox providers

These encrypt your mailbox at rest with a key derived from your password, so the provider cannot read stored mail. Two catches apply to all of them. First, the encryption is end-to-end only between users of the same provider; mail to an outside address either goes out plaintext (subject to normal SMTP TLS) or uses a password-protected-link workaround. Second, and more important for the open-source lens this doc applies, their clients are open source but their servers are not, and even where source is published you cannot verify that the server you are talking to actually runs it, so an encrypted-mailbox provider is a trust-the-operator model no matter how good the client. That trust is bounded by a single company in a single jurisdiction, and both leading providers have a documented case of being legally compelled to act against a user, below.

Proton Mail. Swiss, operated by Proton AG under the non-profit Proton Foundation, in service since 2014. Zero-knowledge encryption at rest, open-source clients across platforms (the server is not open source), post-quantum cryptography rolling out, the most polished UX in the category, and a broader stack (Mail, VPN, Calendar, Drive, Pass) if you want one vendor for several things. The Bridge app provides IMAP/SMTP to desktop clients such as Thunderbird and nmail by running a local decryption proxy.

How Proton has misbehaved, on the record. In 2021 Proton received a legally binding order from the Swiss Federal Department of Justice, originating with French police and routed through Europol, and was compelled to log and hand over the IP address and device type of an account used by the Youth for Climate collective in Paris, which led to an arrest¹⁵. Message contents were never exposed because the zero-access encryption held, but the metadata was, and the episode mattered because Proton’s homepage had until then boasted that it kept no IP logs and put your privacy first; Proton quietly deleted that boast and reworded its privacy policy to state that a user under Swiss criminal investigation can be compelled to have their IP logged. The structural lesson is the one that recurs in this section: zero-access encryption protects stored content, but a single-jurisdiction provider can be forced to log IP and metadata on a targeted account going forward, and marketing that implied otherwise was overstatement corrected only after it was caught.

On the controlled-opposition charge. Parts of the privacy and Bitcoin community characterize Proton as controlled opposition, a service that exists to gather privacy-seekers into one auditable place, pointing to the IP-logging episode, Proton’s 2022 World Economic Forum Technology Pioneer designation, and CEO Andy Yen’s December 2024 public praise for a Trump antitrust nominee¹⁵. The primary-source record supports the narrower claims, namely Swiss court-compulsion exposure, a marketing-versus-reality reversal, and establishment-adjacent signaling a sovereignty-minded reader may legitimately weigh, but it does not establish the strong claim of deliberate intelligence control: Proton is governed by a non-profit foundation, is headquartered outside the Five and Fourteen Eyes alliances, and the Yen episode was a narrow antitrust comment the company walked back as not reflecting an official position. Treat the strong-form label as an unproven inference, and treat the narrower facts as the actual basis for deciding how much to trust a Swiss single-company mailbox.

Pick Proton if you want the most-polished encrypted mailbox, you accept Swiss-single-company trust and the court-compulsion exposure above, and an open client is enough verification for you in the absence of an open server.

Tuta. German, operated by Tutao GmbH since 2011, renamed from Tutanota in November 2023¹⁶. Encrypts more than the others, subject lines included, using its own TutaCrypt protocol with post-quantum key exchange rather than PGP, and its clients are open source. The hard trade: no PGP interoperability and no IMAP/SMTP at all, so you are locked to Tuta’s own clients (web, desktop, mobile) with no third-party-client escape hatch, at the cheapest paid tier in the category.

How Tuta has misbehaved, on the record, with the important caveat that Tuta fought it. In 2020 a regional court in Cologne ordered Tutanota to build a function to monitor a single account used in a blackmail case, and in 2021 Germany’s Federal Court of Justice upheld a version of this, requiring three months of monitoring across the implicated accounts¹⁷. The monitoring applies only to mail the account receives after the order and delivers only unencrypted messages, since Tuta cannot decrypt content already stored at rest; Tuta argued all the way to the Federal Court that it is not a telecommunications service, and lost. Germany is a Fourteen-Eyes jurisdiction under strong GDPR-plus protections. The lesson mirrors Proton’s: at-rest encryption protects stored mail, but a single-jurisdiction provider can be compelled to monitor future incoming mail on a targeted account, and resisting the order in court delays it rather than defeating it.

Pick Tuta if subject-line encryption, an open client, and price matter more than PGP interop and client freedom, and you accept German-single-company trust and the court-compulsion exposure above.

The structural catch worth repeating: encrypted-mailbox providers solve at-rest confidentiality against the provider, not metadata exposure and not in-transit confidentiality to non-users. They are a real improvement over Gmail; they are not a messenger.

Privacy-respecting standard providers

These run ordinary IMAP/SMTP (so any client works) and compete on policy and jurisdiction rather than on cryptographic-at-rest gimmicks. You trust the provider not to read your mail rather than making it cryptographically impossible. The trade is interoperability and client freedom in exchange for giving up the at-rest guarantee Proton and Tuta offer.

Posteo (German, in service since 2009, around one euro per month, runs on green energy, strips IP metadata from outgoing mail, anonymous signup and payment, no custom domains). Mailbox.org (German, since 2014, supports custom domains, integrated office suite, PGP via the webmail with server-side key handling as an option). Disroot (Dutch, a donation-funded collective running on free software since 2015, part of a broader libre-services suite, activist-aligned). Mailfence (Belgian, run by the long-established ContactOffice group, built-in PGP keystore, calendar and documents, custom domains). These run proprietary server stacks, Disroot excepted (its stack is free software), and in every case you are trusting the operator rather than verifying the server, the same bound as the encrypted-mailbox providers but without the at-rest guarantee.

Pick one of these if you want client freedom (use Thunderbird, nmail, neomutt, whatever), custom-domain support, or a specific privacy-respecting jurisdiction, and you’re comfortable with provider-trust rather than zero-knowledge as the model. Posteo and Mailbox.org are the two most-recommended in privacy circles; Disroot if the activist-collective alignment fits.

Newer and unproven, and why longevity matters

This doc treats how long a service has operated as a security signal, so newer entrants get a maturity stamp rather than a recommendation, and recently-shut-down services get named as the reason why.

AtomicMail. A new encrypted-mailbox service, EU-based (an Estonian company with servers in Germany, GDPR-compliant), whose mobile client shipped in 2025¹⁸. It offers zero-access encryption, anonymous signup with no phone number, and seed-phrase account recovery. Two things keep it out of the recommended set on this doc’s own criteria. First, it is not fully open source: its code is not fully published, which by the open-source-is-a-floor lens means its behavior is taken on faith, and it carries a higher trust requirement than even Proton or Tuta. Second, it uses its own proprietary Atomic Encryption (built on AES-256 and ECIES) rather than OpenPGP, which reproduces Tuta’s client lock-in without Tuta’s decade of operation, and there is no published independent security audit. The encryption claims may well be sound; the point is that with a closed, unaudited, very young service you cannot check, and the whole reason to leave Gmail is to stop taking such things on faith. Revisit AtomicMail if it opens its source and publishes an independent audit; until then it is one to watch, not one to trust with anything that matters.

Skiff, the cautionary tale. Skiff was a polished end-to-end-encrypted mail, calendar, and document suite, founded in 2020, partially open source, and well funded, having raised 14.2 million dollars from investors including Sequoia¹⁹. In February 2024 it was acquired by Notion, and by August 2024 the entire product was shut down, with mail forwarding limping on only until February 2025, leaving users to migrate everything out under a deadline. The lesson is the one this doc builds the longevity lens around: a slick, partly-open, VC-funded privacy startup is the profile most likely to be acquired and killed, because the funding model points at an exit rather than at operating the same service for twenty years. Weight a provider’s years in operation and its funding model accordingly, and prefer the boring decade-old options for anything you cannot afford to migrate on someone else’s timeline.

Self-hosted mail

Maximum sovereignty, highest operational cost in this entire project series. Running your own mail means you control everything and trust no provider; it also means you own the single hardest self-hosting problem there is.

Stalwart. The sovereignty-frontier pick²⁰. Rust, single binary, AGPL-3, native JMAP (RFC 8621, the modern API that supersedes IMAP) plus IMAP4rev2, POP3, SMTP, CalDAV, CardDAV, WebDAV, the whole stack in one process where the traditional approach chains Postfix, Dovecot, and Rspamd. Built-in DKIM/SPF/DMARC/ARC, statistical spam filter, web admin, automatic TLS via ACME, encrypted-at-rest mailboxes, OpenID Connect for SSO. Two independent security audits by Radically Open Security. Runs comfortably on a 512MB-1GB VPS. The modern choice; smaller community than mailcow but the trajectory is clear.

mailcow. The battle-tested pick. Docker-compose stack (Postfix, Dovecot, Rspamd, SOGo webmail, admin UI), mature, well-documented, large community, the safe default if you want a full email server replacing Google Workspace. Heavier than Stalwart (2GB+ RAM). Maddy (Go, single binary, simpler and smaller scope than either; good for a personal domain). Mailu (Docker, Postfix/Dovecot-based, simpler than mailcow).

The hard truth, named prominently because every “self-host your mail” pitch glosses it: deliverability is the problem, not the software. To not land in Gmail’s and Outlook’s spam folders you need four correct DNS records (MX, SPF, DKIM, DMARC), a matching PTR reverse-DNS record, and (critically) a sending IP with clean reputation that isn’t on any blocklist. Residential ISP connections fail this by default: port 25 is commonly blocked outbound, and residential IP ranges are blocklisted as a class. So self-hosted mail belongs on a VPS with a clean IP and unblocked port 25, not on a home connection, unless you front outbound mail through a relay.

StartOS, and why home-hosted mail is the hard case. StartOS (Start9, formerly EmbassyOS; renamed; 0.4.0 unveiled March 2026; MIT-licensed, Rust backend; serves a graphical interface as a private website; runs services over Tor v3 with clearnet hosting also supported)²¹ is the leading plug-and-play sovereign-computing OS: it lets non-sysadmins discover, install, configure, back up, and monitor self-hosted services on a home appliance from a web UI. For Nextcloud, a Bitcoin full node, Vaultwarden, a Lightning node, media servers, and most of the self-hosting catalog, it is an excellent answer and squarely in this project’s sovereignty frame. Start9 sells preflashed Server One hardware or you can DIY-install on your own box.

Mail is the exception, and it’s worth being precise about why. StartOS does not currently ship a turnkey mail server in its marketplace (its FAQ confirms a service has to meet packaging requirements and mail isn’t a first-class offering today). More fundamentally, a StartOS box lives on your home connection, exactly the residential-IP, port-25-blocked, blocklisted-by-class situation that makes home-hosted outbound mail land in spam. So StartOS is a great home for self-hosted services generally and a legitimate host for a relay-fronted mail setup (where a clean-IP VPS or a transactional relay handles outbound delivery and your StartOS box holds the mailboxes), but it is not a one-click fix for the thing that actually makes self-hosted mail hard. If your goal is sovereign mail specifically, a Stalwart instance on a clean-IP VPS is the more honest path than mail on a home appliance. StartOS’s fuller treatment as a sovereign-computing platform belongs in os.md; here it’s named for the self-hosted-mail-deployment question only.

The ladder to self-hosted mail

Sovereign mail is a climb, not a switch, and the single move that makes the climb possible is owning a custom domain from the start, because then every rung is a provider swap rather than a change of address that breaks all your accounts and contacts. The rungs, in increasing order of control and operational cost:

Rung 0, harden a hosted mailbox. Use Proton or Tuta for at-rest encryption, add per-service aliases (below), and add PGP for bodies where a correspondent supports it. You own nothing yet, but you have shrunk what the provider and a passive network can see.

Rung 1, standard provider on your own domain. Move to Posteo or Mailbox.org on a custom domain you control. You still trust the operator, but the address is now yours, so the rest of the ladder no longer costs you your identity.

Rung 2, provider holds the mailboxes, you hold the domain and the keys. Keep a hosted mailbox but treat it as replaceable infrastructure: your domain, your PGP keys, your aliases, and a local archive in Thunderbird or nmail. At this point you can change providers in an afternoon.

Rung 3, self-host on a clean-IP VPS. Run Stalwart (or mailcow) on a VPS with a clean IP and unblocked port 25. You now control and can audit the whole stack, and the only thing you are renting is the IP reputation, which is the hard part of email. This is the honest sovereign-mail endpoint for most people.

Rung 4, self-host at home, relay-fronted. Hold the mailboxes on your own hardware or a StartOS box at home, and front outbound delivery through a clean-IP relay so your residential, port-25-blocked, blocklisted-by-class connection does not sink your mail into spam folders. Maximum control, maximum operational burden, worth it only if you will keep it running.

The deliverability problem from the self-hosted section sets where the climb gets steep: rungs 0 through 2 inherit someone else’s clean IP, rung 3 rents a clean one, and rung 4 is hard precisely because a home connection cannot supply one. Climb only as high as you will maintain.

Encryption layers on top of any email

Independent of provider, you can add body encryption.

PGP / OpenPGP. Thunderbird has OpenPGP built in (no Enigmail plugin needed since Thunderbird 78). This is the standard way to encrypt email bodies end-to-end across any provider, provided your correspondent also uses PGP. The honest limitations: key management is a real burden, there’s no forward secrecy (compromise of your long-term key exposes all past mail encrypted to it), and the metadata still leaks. These limitations are precisely why the messenger tiers in the first half of this doc exist. For the mechanics of PGP keys, hardware-token storage, and the web-of-trust, see gpg-concepts.md rather than re-deriving it here.

age-encrypted attachments. For sending a sensitive file over any email, encrypting it with age (a modern, simple file-encryption tool, see choosing-encryption-tools.md) and attaching the ciphertext sidesteps the PGP-email complexity entirely: the email is just a transport for an independently-encrypted blob. Lower-ceremony than PGP-email for the specific case of “send this one file securely.”

Autocrypt. The in-band key-exchange profile that DeltaChat (Tier 6 above) uses to make PGP transparent. Some other clients support it. It trades a little security (keys exchanged opportunistically in headers) for the usability that hand-managed PGP lacks.

Email clients

Per the apt-primary, build-from-source-only-when-needed approach: most of these are in Debian/Devuan repositories.

Thunderbird / Betterbird. The default graphical client, cross-platform, OpenPGP built in, works with any IMAP/SMTP provider and with Proton via the Bridge. Betterbird is a Thunderbird fork with extra features and fixes, for users who want them. The right default for most people.

Terminal clients. neomutt (the actively-maintained mutt fork, maximally configurable, the long-standing power-user choice). aerc (modern, async, Git-friendly, good for plaintext and patch workflows). nmail (d99kris; C++/ncurses; IMAP+SMTP; local cache in optionally-AES256-encrypted SQLite; address book auto-generated from messages; Alpine/Pine-style UI; setup wizards for Gmail/iCloud/Outlook; compose in $EDITOR, view in $PAGER; MIT; v5.11.4)²². meli (Rust terminal client, newer). nmail’s distinguishing traits for this audience: the encrypted local cache and the familiar Pine-style interface; note it’s deliberately not designed to interoperate with other clients’ Maildir, so treat it as a self-contained client rather than one tool in a multi-client setup.

Mobile. Thunderbird for Android (formerly K-9 Mail, which Thunderbird adopted and rebranded) and FairEmail (privacy-focused, granular controls) on Android; on iOS the choices are more constrained, with Proton’s and Tuta’s own apps the sovereignty-aligned options for those providers.

Aliasing and forwarding

A unique email alias per service means a breach or data-sale at one service exposes only that alias, and you can sever any sender by killing one alias without changing your real address. Sovereignty-aligned because it decouples your identity from any single provider and from the senders.

SimpleLogin (open-source, now Proton-owned, self-hostable). addy.io (formerly AnonAddy; open-source, self-hostable, generous free tier). Firefox Relay (Mozilla, simplest, fewest features). Pick SimpleLogin or addy.io for the self-hostable open-source options; both can run on your own domain so the aliasing itself doesn’t introduce a new trusted third party.

What to avoid for email

Gmail, Outlook / Microsoft 365, Yahoo for anything sensitive. Content is scanned, the business model monetizes what it learns about you, and the mailbox is readable by the provider and anyone who compels the provider. Fine for low-stakes mail and mailing-list signups; not for anything you’d mind being read. The “free that isn’t a free tier of a paid product” frame from choosing-networking-tools.md applies identically: if the email is free and the company is an advertising company, you are the product.

Email maturity table

1. SimpleX Chat blog, SimpleX Channels, SimpleX Network Consortium and Community Crowdfunding, to Preserve Freedom of Speech, 30 April 2026: https://simplex.chat/blog/20260430-simplex-channels-v6-5-consortium-crowdfunding-freedom-of-speech.html. “No single company should control protocols and network that people depend on to speak freely. … we’re launching SimpleX Network Consortium within a few months, the agreement between the new SimpleX Network Foundation and SimpleX Chat company that will govern protocols and licensing, perpetual, irrevocable, surviving if any party is sold or shut down.” ↩ ↩2

2. IETF More Instant Messaging Interoperability (MIMI) Working Group: https://datatracker.ietf.org/wg/mimi/about/. Active drafts as of early 2026: An Architecture for More Instant Messaging Interoperability (draft-ietf-mimi-arch, R. Barnes, Cisco): https://datatracker.ietf.org/doc/draft-ietf-mimi-arch/; More Instant Messaging Interoperability (MIMI) using HTTPS and MLS (draft-ietf-mimi-protocol, R. Barnes / M. Hodgson / K. Kohbrok / R. Mahy / T. Ralston / R. Robert): https://datatracker.ietf.org/doc/draft-ietf-mimi-protocol/; Room Policy for the MIMI Protocol (draft-ietf-mimi-room-policy, R. Mahy): https://datatracker.ietf.org/doc/draft-ietf-mimi-room-policy/; MIMI message content (draft-ietf-mimi-content, R. Mahy): https://datatracker.ietf.org/doc/draft-ietf-mimi-content/. ↩

3. SimpleX Chat, project home page: https://simplex.chat/. “The first messenger without user IDs. Other apps have user IDs: Signal, Matrix, Session, Briar, Jami, Cwtch, etc. SimpleX does not, not even random numbers. … To deliver messages, instead of user IDs used by all other platforms, SimpleX uses temporary anonymous pairwise identifiers of message queues, separate for each of your connections, there are no long term identifiers.” ↩

4. SimpleX Chat blog, SimpleX network: cryptographic design review by Trail of Bits, v6.1 released with better calls and user experience, 14 October 2024: https://simplex.chat/blog/20241014-simplex-network-v6-1-security-review-better-calls-user-experience.html. The Trail of Bits review followed an earlier 2022 audit: https://simplex.chat/blog/20221108-simplex-chat-v4.2-security-audit-new-website.html. ↩

5. Keet by Holepunch, project site: https://keet.io/. Backed by Tether and Bitfinex; built on the Pear Runtime and the Hypercore protocol (Holepunch GitHub organization: https://github.com/holepunchto); serverless peer-to-peer over a distributed hash table, end-to-end encrypted text, voice, and video with Bitcoin Lightning and USDT payments; launched 2022. The Pear Runtime, Hypercore, and Hyperswarm are open source under permissive licenses; the Keet client application itself is not open source, packaged by NixOS as unfree (https://github.com/NixOS/nixpkgs/issues/208506), and a 2022 intention to open-source the app remains substantially unfulfilled as of 2026. ↩

6. bitchat, GitHub organization: https://github.com/permissionlesstech. iOS repository: https://github.com/permissionlesstech/bitchat. Android repository: https://github.com/permissionlesstech/bitchat-android. Technical whitepaper: https://github.com/permissionlesstech/bitchat/blob/main/WHITEPAPER.md. Jack Dorsey’s launch announcement on X, 6 July 2025; Engadget coverage of the iOS App Store release, 28 July 2025: https://tech.yahoo.com/apps/articles/jack-dorseys-bluetooth-messaging-app-185000506.html. “and Other Stuff” is Dorsey’s open-source development collective backing the project. Uganda 2026 election adoption documented in 2026 coverage of opposition leader Bobi Wine urging citizens to use bitchat ahead of anticipated internet shutdowns; Google Trends documented the corresponding Uganda search-interest surge. Wikipedia summary entry: https://en.wikipedia.org/wiki/BitChat. ↩

7. White Noise, project home page: https://www.whitenoise.chat/. GitHub repository: https://github.com/parres-hq/whitenoise and Rust core at https://github.com/marmot-protocol/whitenoise-rs. Architectural summary from the project FAQ as cited by gigazine.net coverage, 14 July 2025: https://gigazine.net/gsc_news/en/20250714-whitenoise-chat-secure-messaging-nostr/. Atlas21 founder interview with Max Hillebrand, 21 July 2025: https://atlas21.com/white-noise-is-born-the-private-messaging-app-based-on-nostr/. Bitcoin Magazine launch coverage: https://bitcoinmagazine.com/news/white-noise-anonymous-nostr-dms-and-encrypted-group-chat. Apple TestFlight beta: https://testflight.apple.com/join/c6Z7PpxC. ↩

8. Marmot Protocol, organization on GitHub: https://github.com/marmot-protocol. Combines Nostr (identity, signaling), Blossom (file hosting on Nostr), and MLS (RFC 9420 Messaging Layer Security) into a single protocol stack for sovereign messaging. White Noise is the reference application. ↩

9. Nostr Improvement Possibilities, NIP-17: Private Direct Messages: https://github.com/nostr-protocol/nips/blob/master/17.md. Specifies gift-wrapped DMs using NIP-44 encryption (versioned ChaCha20-Poly1305) inside NIP-59 sealed events, with a third gift-wrap event using random keys to hide sender and recipient identities from relays. ↩

10. Nostr Improvement Possibilities, NIP-49: Private Key Encryption: https://github.com/nostr-protocol/nips/blob/master/49.md. Specifies a ncryptsec format for password-encrypted Nostr private keys, the Nostr analog of an encrypted SSH or GPG private key. ↩

11. Pubky Core documentation: https://pubky.org/. Getting-started flow documenting Pubky Ring install, the invite/SMS/Lightning signup gates, path-scoped app authorization, and /pub/ being public by default: https://pubky.org/getting-started/. FAQ stating the MIT license, that Pubky is currently optimized for public data with private and encrypted features planned via Pubky Noise, and that Slashtags was the predecessor Synonym project on Hypercore: https://pubky.org/faq/. Retrieved June 2026. ↩ ↩2 ↩3 ↩4

12. Pubky GitHub organization: https://github.com/pubky. pubky-core (MIT, Rust, “an open protocol for per-public-key backends for censorship resistant web applications”), pkarr (MIT, Rust; README states Ed25519 keys and publication to the BitTorrent peer-to-peer network of over ten million nodes), pkdns (MIT, Rust, a DNS server resolving pkarr self-sovereign domains), pubky-ring (MIT, TypeScript/React Native; v1.15 released 2 June 2026; site https://pubkyring.app/), pubky-noise (Rust, early), and a community Umbrel App Store repository hosting the Pubky Homeserver. All actively pushed as of June 2026. ↩ ↩2 ↩3 ↩4

13. Bitfinex blog, “What is Pubky?”, interview with Synonym CEO John Carvalho, August 2025: https://blog.bitfinex.com/education/what-is-pubky/. Carvalho calls Pubky “a strict upgrade” to Nostr and argues that Nostr lacks a discovery method once a hosting server censors a user, whereas Pubky resolves the current data location via PKDNS. ↩

14. Cointelegraph, “Tether launches Synonym to boost Bitcoin adoption through Lightning Network,” 16 November 2021: https://cointelegraph.com/news/tether-launches-synonym-to-boost-bitcoin-adoption-through-lightning-network, reporting Synonym Software as a company founded by Tether Holdings. Launch coverage recording Tether and Bitfinex CTO Paolo Ardoino as Synonym’s CTO at launch: Bitcoin Takeover, “John Carvalho Presents Synonym,” 16 November 2021: https://bitcoin-takeover.com/john-carvalho-presents-synonym-the-hyperbitcoinization-company/. Tether’s own newsroom pairing the companies in the Pear Credit announcement, 28 October 2022: https://tether.io/news/tether-holepunch-and-synonym-launch-pear-credit-a-p2p-credit-system/. The synonym.to footer carries a Tether endorsement mark; current legal entity Synonym Software, S.A. DE C.V. Retrieved June 2026. ↩

15. Proton Mail’s 2021 logging episode and 2024 political controversy. TechCrunch, “ProtonMail logged IP address of French activist after order by Swiss authorities,” 6 September 2021: https://techcrunch.com/2021/09/06/protonmail-logged-ip-address-of-french-activist-after-order-by-swiss-authorities/. Proton stated it received a legally binding order from the Swiss Federal Department of Justice, originating with French police via Europol, concerning an account used by the Youth for Climate collective in Paris; message contents were not accessed. Threatpost, “ProtonMail Forced to Log IP Address of French Activist,” 7 September 2021, documents the removal of the “we don’t log your IP” claim from the homepage and the reworded privacy policy: https://threatpost.com/protonmail-log-ip-address-french-activist/169242/. Andy Yen’s December 2024 praise of the Gail Slater antitrust nomination and Proton’s subsequent neutrality statement: The Intercept, “Proton Mail Says It’s ‘Politically Neutral’ While Praising Republican Party,” 28 January 2025: https://theintercept.com/2025/01/28/proton-mail-andy-yen-trump-republicans/. Proton’s non-profit governance and non-Five/Fourteen-Eyes jurisdiction are the structural facts that weigh against the strong-form controlled-opposition reading. ↩ ↩2

16. Tuta (formerly Tutanota), project site: https://tuta.com/. Rebranded from Tutanota to Tuta on 7 November 2023 (former tutanota.com redirects to tuta.com). German company Tutao GmbH, established 2011, over 10 million users as of June 2023. TutaCrypt post-quantum encryption protocol; encrypts subject lines, bodies, and attachments at rest; no PGP interoperability and no IMAP/SMTP (clients only). Wikipedia entry: https://en.wikipedia.org/wiki/Tuta_(email). ↩

17. Tuta (Tutanota) court-ordered monitoring. TechCrunch, “German secure email provider Tutanota forced to monitor an account, after regional court ruling,” 8 December 2020: https://techcrunch.com/2020/12/08/german-secure-email-provider-tutanota-forced-to-monitor-an-account-after-regional-court-ruling/. The Cologne regional court ordered a monitoring function for a single account in an extortion case, applying only to mail received after the order and delivering only unencrypted messages, since at-rest content cannot be decrypted. CyberScoop, “Court rules encrypted email provider Tutanota must monitor messages,” May 2021, reports the Federal Court of Justice (BGH) requiring three months of monitoring across the implicated accounts after Tutanota argued it is not a telecommunications service: https://cyberscoop.com/court-rules-encrypted-email-tutanota-monitor-messages/. ↩

18. Atomic Mail, project site: https://atomicmail.io/. Estonian company (Tallinn) with servers in Germany, GDPR-compliant; zero-access architecture using a proprietary “Atomic Encryption” built on AES-256 and ECIES rather than OpenPGP; anonymous signup with seed-phrase recovery. Mobile web client launched 25 March 2025 per the company announcement carried by Barchart: https://www.barchart.com/story/news/31558429/atomic-mail-releases-mobile-version-for-private-email. Independent assessment that its code is not fully open and that it carries a higher trust requirement than Tuta or Proton: the GitHub-hosted email-provider comparison at https://opensourcereviews.github.io/email/index.html. ↩

19. Skiff (email service). Founded 2020 by Andrew Milich and Jason Ginsberg; raised 14.2 million dollars from investors including Sequoia; acquired by Notion in February 2024; all services shut down 9 August 2024 with mail forwarding continuing only until 9 February 2025; partially open source. TechCrunch, “Notion acquires privacy-focused productivity platform Skiff,” 9 February 2024: https://techcrunch.com/2024/02/09/notion-acquires-privacy-focused-productivity-platform-skiff/. Wikipedia entry, “Skiff (email service)”: https://en.wikipedia.org/wiki/Skiff_(email_service). ↩

20. Stalwart Mail Server, project site: https://stalw.art/. GitHub: https://github.com/stalwartlabs/stalwart. All-in-one mail and collaboration server in Rust (SMTP, IMAP4rev2, POP3, JMAP, CalDAV, CardDAV, WebDAV), AGPL-3, by Stalwart Labs Ltd. Built-in DKIM/SPF/DMARC/ARC, statistical spam filter, ACME TLS, encrypted-at-rest mailboxes, OIDC SSO. Two independent security audits by Radically Open Security. NLnet/NGI0 funding history: https://nlnet.nl/project/Stalwart/. ↩

21. StartOS by Start9 Labs (formerly EmbassyOS), project site: https://start9.com/. GitHub: https://github.com/Start9Labs/start-os. MIT-licensed Linux distribution for self-hosting; graphical interface served as a private website; services historically run over Tor v3 with clearnet hosting supported (Tor requirement dropped following OS v0.4.0). 0.4.0 unveiled by Start9 CEO Matt Hill on 27 March 2026. Marketplace of self-hosted services: https://marketplace.start9.com/. Per the Start9 services FAQ, not every self-hosted service is packaged and mail is not a first-class turnkey offering. Founded 2020 in Denver, Colorado. ↩

22. nmail by Kristofer Berggren (d99kris), GitHub: https://github.com/d99kris/nmail. Terminal-based email client for Linux and macOS, C++/ncurses, Alpine/Pine-style UI. IMAP and SMTP; local cache in optionally-AES256-encrypted SQLite; auto-generated address book; setup wizards for Gmail, iCloud, and Outlook/Hotmail; compose in $EDITOR, view in $PAGER. MIT License; v5.11.4 (March 2025). Per its documentation, deliberately not designed to interoperate with other clients’ Maildir. ↩

Choosing Networking Tools

How to choose a VPN, mesh, or overlay network in a way that doesn’t reintroduce the central trusted third party you were trying to escape. Covers commercial VPNs, self-hosted point-to-point, mesh with central coordinator, coordinator-less mesh, Nostr-rooted mesh, anonymity overlays, censorship-evasion proxies, and off-grid radio mesh.

This doc is the dedicated landscape for the cross-cutting concern that security-overview.md opens with its “Tor versus VPN versus self-hosted mesh” section. That section frames the choice at a high level; this doc fills in the rest of the space.

TL;DR

Most readers don’t need a VPN at all. HTTPS-everywhere plus encrypted DNS (set up in devuan-secure-workstation.md) plus a hardened browser closes most of what commercial VPN marketing claims to solve. The legitimate use cases:

1. Mesh between your own devices over hostile networks: self-hosted WireGuard or Headscale; Tailscale if convenience outweighs the corporate-coordinator capture risk.
2. Anonymity: Tor (or I2P for app-aware use). Commercial VPN is structurally weaker than Tor and not a substitute. Against a global passive adversary (nation-state-scale traffic observation), NYM’s mixnet defends where Tor’s onion routing doesn’t.
3. Bypassing geographic blocks or hostile-ISP restrictions: Mullvad or IVPN; pay in cash or Bitcoin.
4. Active censorship (deep packet inspection, Tor blocking): Tor with pluggable transports and bridges, or censorship-evasion proxies (Shadowsocks, v2ray, Hysteria, Outline).
5. Off-grid communication when the internet is unavailable or untrusted: Reticulum over LoRa, Meshtastic, or Briar’s Bluetooth mesh.

Sovereignty-frontier projects to track but not yet deploy as a daily driver: nostr-vpn¹ plus its underlying FIPS² mesh protocol. Both alpha-grade in 2026 but architecturally aligned with the Bitcoin / Nostr lens of no-trusted-third-parties.

What this doc is not: a deep-dive into any specific tool’s setup. Each entry names the tool’s positioning and the capture-risk shape; setup procedures live elsewhere.

The capture-risk frame for networking

Every networking tool answers two architectural questions: what identifies a node, and who coordinates which nodes can talk to each other. The combination of those answers is the tool’s threat model.

Commercial VPNs answer the first with a company-issued account and the second with a company-operated coordinator. The company holds both your identity and your traffic. The company can be subpoenaed, hacked, acquired, or compelled to log. This is the architecture most “VPN for privacy” marketing sold to people who wanted stronger.

Self-hosted point-to-point answers both with cryptographic keys you generated and infrastructure you operate. No company holds anything. The cost is setup and ongoing maintenance.

Mesh with central coordinator (Tailscale, NetBird, ZeroTier in their default modes) answers the first with your own cryptographic keys but the second with a company-operated coordinator. The coordinator sees who talks to whom and when (metadata, not content) and can be subpoenaed for that metadata. Headscale and self-hosted NetBird close this by letting you operate the coordinator yourself.

Coordinator-less mesh (Yggdrasil, cjdns, Nebula after PKI bootstrap, Innernet after admin signature) answers both with cryptographic identities and a routing protocol that converges without any central party. Hardest to misconfigure into a capture-risk; also hardest to set up.

Nostr-rooted mesh (nostr-vpn on top of FIPS) answers the first with a Nostr keypair (the same secp256k1 identity used for Nostr signing and for messaging via the Nostr-rooted apps covered in choosing-communication-tools.md) and the second with FIPS’s self-organizing mesh routing, with peer discovery and NAT traversal happening over public Nostr relays via gift-wrapped messages. Sovereignty-aligned by construction.

Anonymity overlays (Tor, I2P, Lokinet) answer the first with rotating ephemeral identities and the second with volunteer-run relay networks where no single party knows both who you are and what you’re doing. Mixnets (NYM) go further on the second question, adding cover traffic and timing-mixing so that even a global observer watching the whole network can’t correlate flows: the defense onion routing doesn’t provide.

Censorship-evasion proxies (Shadowsocks, v2ray, Trojan, Hysteria, Outline) answer neither question: they’re not privacy tools; they’re “make encrypted traffic look like benign TLS so the DPI box doesn’t drop it” tools. They live alongside the other tiers and stack on top.

Tier 1: Commercial VPNs

The “trust one company instead of your ISP” tier. Tolerable for narrow use cases (bypassing geographic blocks, hostile-ISP restrictions, traveler-laptop on a hotel network); not a privacy tool.

Mullvad

Swedish, founded 2009. The most-recommended commercial VPN in privacy circles because of three properties uncommon among peers: account identifiers are 16-digit account numbers with no associated email or phone, payment in cash by mail is supported, and the no-logs claim has held under independent audit and under a 2023 Swedish police raid that retrieved no user data³. Mullvad Browser is the Tor-collaboration project covered in security-overview.md.

What you trade: still a single trusted company. The Swedish jurisdiction provides EU data-protection benefits but does not eliminate the structural capture risk any commercial VPN carries.

Pick this if you specifically need a commercial VPN, understand the trust-shift not trust-elimination property, and you’re paying in cash or Bitcoin.

IVPN

Gibraltar-based, similar privacy posture to Mullvad: account numbers not emails, accepts Bitcoin and Monero, regular independent security audits, source-available clients. Smaller server footprint than Mullvad. Pick this as a redundancy second to Mullvad or where Mullvad’s server locations don’t suit.

Proton VPN

Swiss, owned by Proton AG (the Proton Mail company). The most-mainstream privacy-marketed commercial VPN. Free tier exists; the free tier’s economics depend on paid subscribers, so the free tier is real but limited. Proton AG has a public foundation structure (Proton Foundation, Swiss) governing the parent company.

Flag worth naming: Proton AG drew sustained criticism in 2024 from segments of the privacy community over CEO Andy Yen’s public political alignments. This is a community-politics flag, not a technical one, and how much it matters depends on your frame.

What to avoid

Free commercial VPNs that aren’t a free tier of a paid product (NordVPN, ExpressVPN, free apps in mobile stores). The business model that pays for the infrastructure is either logging-and-selling or malware. The “you are the product if it’s free” frame applies here particularly cleanly.

Tier 2: Self-hosted point-to-point

The “your own infrastructure, no company in the middle” tier. The right answer for accessing your home network from your laptop, or for connecting two specific machines you own.

WireGuard

The protocol. Designed by Jason Donenfeld, in-kernel on Linux since 5.6, in-tree on FreeBSD since 13.0, and the foundation that everything in Tier 3 and most of Tier 4 builds on. Configuration is a handful of public/private key pairs and a peer list. Static peer configuration: no coordinator needed if both endpoints have routable addresses.

Userspace implementations exist for cases where the kernel module isn’t an option: wireguard-go (Go, the reference userspace implementation), boringtun (Rust, originally Cloudflare’s implementation, used in early nostr-vpn releases before the FIPS data-plane migration). On Linux with a recent kernel, the in-kernel module is the right choice; the userspace implementations matter for iOS sandbox limitations, certain embedded environments, and as building blocks for higher-level tools.

What this gets you: an encrypted tunnel between two endpoints with formally-verified cryptography, near-line-rate throughput, and no third party. What it doesn’t give you: NAT traversal when both endpoints are behind NAT, or convenient onboarding of more than a handful of peers. Those are the problems Tier 3 solves.

Pick this for two specific machines you control with at least one having a routable address (your home server plus your laptop; two VPSes in different regions; a colo box plus your office). Configuration via wg-quick and /etc/wireguard/*.conf; no daemon to run beyond what the kernel provides.

OpenVPN

The pre-WireGuard incumbent. Still widely deployed; still works. Slower than WireGuard, larger attack surface (TLS-based, lots of options), older crypto stack. Reasons to still use it: a corporate VPN concentrator you don’t control still speaks OpenVPN, or you need TCP-mode tunneling for hostile-network reasons (some restrictive networks block UDP). Otherwise use WireGuard.

SSH tunnels and sshuttle

The poor-person’s VPN. ssh -D opens a SOCKS proxy through any SSH server you can reach; sshuttle wraps it so all traffic to a configured subnet transparently routes via SSH. Not a real VPN (no kernel-level integration, no UDP support), but the right answer when all you need is to bounce traffic through a remote host you already have access to. Zero infrastructure beyond an existing SSH server.

Tier 3: Mesh with central coordinator

The “convenience of a managed service, with the privacy properties of WireGuard between peers” tier. Each peer holds its own keys and talks WireGuard (or similar) directly to other peers; a central coordinator handles peer discovery, key distribution, and access policy. The coordinator does not see content but does see metadata (who talked to whom, when, from what IP).

Tailscale

The category-defining product. WireGuard data plane, proprietary closed-source coordinator. Founded 2019 (David Crawshaw, Avery Pennarun, David Carney), Toronto-based, raised $275M+ across four rounds; Series C of $160M in April 2025⁴. Acquired Border0 in March 2026. 318 employees as of April 2026⁵.

Authentication requires a third-party identity provider (Google, Microsoft, GitHub, Apple, Okta) which means signing into your mesh requires signing into a corporate account.

Capture-risk shape: late-stage VC-funded enterprise networking company. The mandatory third-party identity provider authentication is the single largest sovereignty problem. The product is excellent technically; the structural position is the opposite of sovereignty-aligned. Pivoting toward AI-agent governance and enterprise data-residency as of 2026⁶, which is the direction late-stage networking startups go when looking for exits.

Pick this only if convenience genuinely outweighs the capture-risk, you’re using it for mesh between machines you control rather than for any privacy property, and you understand you’re authenticating through Google or Microsoft.

Headscale

Open-source reimplementation of Tailscale’s control server, started by Juan Font⁷. Talks the same protocol the official Tailscale clients use, so you keep the Tailscale client UX while running the coordinator yourself. Active development; v0.26.x current as of early 2026. Pairs well with Headplane (open-source web UI) since Headscale itself ships only the daemon.

This is the sovereignty-aligned migration path off Tailscale that keeps the operational experience intact. Pick this if you already use Tailscale clients across devices and want to move the coordinator under your control without retraining users.

What you trade: you run the coordinator (a small Go binary plus Postgres or SQLite); you maintain compatibility with Tailscale client releases. Tailscale Funnel, Tailscale Serve, and some MDM features are Tailscale-proprietary and don’t work with Headscale.

NetBird

Fully open-source mesh VPN (Apache 2.0), Berlin-based, founded 2021. Both client and management server are open source. SSO via standard identity providers (Keycloak, Authentik, Okta, Auth0); first-class DNS management; access policies and groups built in⁸. Available as a cloud-hosted service from NetBird the company, or fully self-hostable in production.

Capture-risk shape: same as Tailscale architecturally if you use their hosted service, but with the escape hatch that self-hosting is fully supported and well-documented. Smaller company, earlier-stage, less VC pressure than Tailscale.

Pick this if you want a Tailscale-equivalent built from the ground up as open source, with a self-hosted-or-cloud choice you can flip later. Currently the best one-stop answer in Tier 3 for users who don’t already have Tailscale clients deployed.

ZeroTier

Open-source clients (BSL 1.1 license), commercial coordinator service operated by ZeroTier Inc. (Irvine, California). Self-hostable controller exists (my-zerotier-controller) but is less polished than the hosted service. Older than Tailscale by several years; uses its own protocol rather than WireGuard.

Capture-risk shape: closer to Tailscale than to NetBird because the self-hosting story is workable but not the primary path. The BSL license on clients is a yellow flag (BSL is source-available, not OSI-approved free software).

Pick this only if you specifically need ZeroTier’s older Layer-2 emulation (it can bridge Ethernet broadcast domains in a way WireGuard-based mesh can’t) for a use case like running legacy multicast-dependent protocols across sites.

Others worth knowing

The Tier 3 class has more entrants than the four above; these are the architectural classes worth naming, and the rest mostly recombine the same trade-offs. Briefly, for completeness: Firezone (WireGuard-based, open-source, self-hostable, policy-driven access), Netmaker (WireGuard mesh with a self-hostable control plane, kernel-WireGuard performance focus), OpenZiti (zero-trust overlay with application-embedded networking rather than host-level tunnels), Defguard (open-source WireGuard with enterprise SSO and MFA), and Twingate (closed-source zero-trust access, commercial, the most Tailscale-like in positioning). These are commercial-backed startups for the most part; expect some to be acquired or to pivot. Evaluate any of them against the same two questions the capture-risk frame poses: who holds the identity, and who operates the coordinator. If the answer to the second is “a company, with no self-hosting path,” it’s a Tier 1 risk shape wearing Tier 3 clothes.

Tier 4: Coordinator-less mesh

The “after bootstrap, no central party exists” tier. The setup step is heavier (you generate certificates or signed configurations once); the running mesh has no coordinator that could be compelled to produce logs.

Nebula

Open-source mesh built at Slack and open-sourced in 2019; the original authors (Nate Brown and Ryan Huber) left Slack in 2020 to found Defined Networking, which now maintains Nebula and sells a managed Nebula-based product. The open-source project itself remains independent.

PKI-based: you stand up a certificate authority once, sign peer certificates with it, and after that peers authenticate to each other using the CA’s signature. No coordinator runs continuously; “lighthouses” exist to help peers find each other across NAT but hold no authority, any peer can be a lighthouse. UDP-based, kernel-bypass userspace networking via tun/tap.

Slack-to-Salesforce note: Slack was acquired by Salesforce in 2021; Nebula itself predates that acquisition and the maintainers spun out a year before it closed. The Nebula codebase has no Salesforce dependency.

Pick this if you’re operating a fleet (dozens to thousands of machines) and want PKI-style host-to-host authentication without an always-on coordinator. Heavier ops burden than Tailscale-style products; pays off at scale.

Innernet

Rust-based mesh from tonarino, MIT licensed. CIDR-based organization rather than flat-list-of-peers: a network is a CIDR, sub-CIDRs are sub-networks, hosts get IPs within their CIDR. Admin-signed peer invitations bootstrap new nodes; after bootstrap there’s no coordinator. Quieter project than the others in this tier; not enterprise-targeted.

Pick this if the CIDR-based mental model fits your use case (it does for some homelab configurations) and you want a smaller, more focused codebase than Nebula.

Yggdrasil

End-to-end encrypted IPv6 overlay network. Tree-based routing with public-key-derived addresses (your IPv6 address is a hash of your Yggdrasil public key). Can operate as a public global network (the default, anyone running Yggdrasil can reach anyone else), as a private mesh (filter peer connections to specific keys), or layered (private mesh peered with the public network).

Mature: development since 2017, current 0.5.x series, in Debian and Devuan repositories, kernel module supported via the yggdrasil-go userspace daemon.

Pick this for: a private IPv6 overlay between machines without setting up your own CA or coordinator (you exchange public keys with peers and that’s it); a fallback identity-routed network during real-internet disruptions; an experimental sovereignty overlay that interoperates with a public mesh.

cjdns

The Hyperboria project’s network protocol. End-to-end encrypted IPv6, source-routed (each packet carries its path), keypair-derived addresses similar to Yggdrasil. Older than Yggdrasil (2011 onward); smaller deployment today but the design influenced Yggdrasil heavily.

Pick this if you specifically want to connect to Hyperboria or you have a use case where source-routing is the right primitive. For most readers, Yggdrasil is the more practical answer in the same shape.

Tier 5: Nostr-rooted mesh (sovereignty frontier)

The newest entry in the landscape. Two projects, two authors; this section names the distinction because secondary commentary tends to conflate them.

FIPS (Free Internetworking Peering System)

The underlying mesh networking protocol, built by jcorgan (jmcorgan on GitHub)². Rust, MIT licensed. Uses Nostr secp256k1 keypairs as node identities (your npub is your network address; node_addr is a SHA-256 hash for routing). Designed to operate over any transport, UDP overlay on the existing internet today, with Ethernet, Bluetooth, Tor, and serial transports as design targets. Tor transport support landed in v0.2.0 (March 2026)⁹.

Architecture in one sentence: a self-organizing encrypted mesh where nodes establish peer connections, authenticate each other via Nostr keys, and route traffic for each other without any central authority or global topology knowledge. End-to-end encrypted between any two nodes regardless of hop count, with re-encryption at every hop.

Status: v0.1.0 alpha as of self-description, v0.2.0 current. The author’s framing is direct: “if it breaks, you get to keep both pieces.” Passed simulation testing and small-scale deployments; not production-stable.

Note on the acronym: clashes with the well-known U.S. cryptographic-standard FIPS, which has drawn public criticism¹⁰. Worth knowing if you reference the project in your own writing.

nostr-vpn

Tailscale-style mesh VPN application built by Martti Malmi (mmalmi)¹, the Bitcoin developer who worked alongside Satoshi in 2009-2011 and received the first Bitcoin transaction. Uses FIPS as its data plane per the current README. Earlier releases (v0.2.x in March 2026) used WireGuard tunnels via boringtun with Nostr relays for signaling; the architecture migrated to FIPS-backed during the version-4 series, with v4.x current as of late May 2026 and rapid iteration ongoing (the project has been shipping multiple releases per week through the v4 series, so any specific version number in this doc should be assumed stale by the time you read it; check the GitHub releases page).

Authorship-split: Malmi built the user-facing nostr-vpn application. FIPS (which the secondary commentary tends to credit to “Bitcoin developer Malmi”) was built by jcorgan, who is a separate person. Both are sovereignty-aligned developers; only one of them is the Satoshi-era figure.

What the May 2026 v4 release line added (Malmi’s announcement, May 19): native multiplatform desktop apps (macOS, Linux, Windows), Android app, Nostr-based multihop routing for when NAT holepunching fails, improved network management UX, FIPS as the unified data plane. Shipped 11 releases in seven days from v0.2.2 through v0.2.13 in March; the v4 series followed in April-May with ongoing rapid iteration. Production-grade nowhere near.

What’s worth picking up today: read the README, run it between two machines you control, learn the model, contribute back. What’s not worth picking up today: replacing your daily mesh setup with it. The right time to migrate is when FIPS leaves alpha and nostr-vpn settles a release cadence longer than days.

Capture-risk shape: zero by construction if the implementation is correct. Your identity is a keypair you generated; coordination happens over a set of Nostr relays you can choose, run, or rotate; routing happens through other FIPS nodes that authenticate each other via Nostr keys. No company exists in the protocol’s threat model.

Why it matters now even though it’s not deployable yet: the design is the proof-of-concept for the sovereignty frame applied to networking infrastructure, the same way Bitcoin was the proof-of-concept for the sovereignty frame applied to money. The architecture is what to learn from. The bits work today; the polish doesn’t.

Adjacent sovereignty stack: the Nostr-rooted ecosystem these tools sit in extends beyond networking. White Noise (choosing-communication-tools.md) is the messaging-layer counterpart, using MLS encryption over Nostr signaling via the Marmot protocol. Blossom is the Nostr-native file-hosting standard that Marmot uses for media transport. The shared infrastructure (Nostr relays, secp256k1 keypairs, Bitcoin-community-funded development) is one of the project’s sovereignty-frontier signals to watch.

Adjacent frontier: the Holepunch/Pear stack

Not Nostr-rooted, and not a transport. This note is parked in Tier 5 as a sovereignty-frontier signal to follow, alongside the Nostr-rooted stack above, until the project has a dedicated file-sharing doc to house it.

PearDrive is a peer-to-peer file-sharing and storage tool built on the Pear Runtime, the same Holepunch P2P stack that powers Keet.^{11 12} You create or join a “network” with a shared network key, then upload files and pull them directly from connected peers, with peer identities surfaced as public keys and network access shareable by QR code.¹³ An archive mode turns a node into a full replica that stores every file on the network, the project’s answer to availability when the original uploader goes offline.¹³

Substrate and funding shape: Pear Runtime is open-source infrastructure (the Hypercore family: Hyperswarm, HyperDHT, Hyperdrive) created by Holepunch, a company funded by Tether and Bitfinex, with Tether’s Paolo Ardoino as a co-founder and the project framed in explicitly Bitcoin terms.^{12 14} In this project’s lens that is a double signal: Bitcoin-aligned capital behind genuinely no-server infrastructure on one side, a single dominant corporate sponsor behind the runtime on the other, which is a concentration risk the volunteer-relay and grant-funded models elsewhere in this doc do not carry. PearDrive itself is published under its own AGPL-3.0 JavaScript repository (the peardrive GitHub org), separate from Holepunch’s own apps, so it inherits the substrate’s funding shape without being a Holepunch in-house product.¹³

Status: a pre-beta proof of concept by its authors’ own account.¹¹ The CLI carries a v4.x tag, but its own README still flags network deletion as unimplemented and file-change syncing as not working until a later PearDriveCore release, so this is a thing to read, run, and learn from, not to entrust files to.¹³ Name-collision caveat for anyone citing it: this is PearDrive (peardrive.com, Pear/Holepunch stack), distinct from the older and unrelated PeerDrive (peerdrive.org), an alpha-stage Erlang distributed filesystem.¹⁵ Revisit when it reaches a tagged beta and file-change sync lands.

Tier 6: Anonymity overlays

Different goal from everything above. Tier 1 through 5 are “encrypt the link”; this tier is “hide who you are.” Cross-referenced from security-overview.md’s “Tor versus VPN versus self-hosted mesh” section.

Tor

Three-hop volunteer-run onion network. Your guard node knows who you are but not what you do; the middle relay knows neither; the exit knows what you do but not who you are. Deanonymization requires controlling or observing both the guard and the exit, which is exponentially harder than compromising a single VPN.

This is the right answer for anonymity. Slow (three hops globally), incompatible with sites that block Tor exits (Cloudflare-protected sites in particular). Run it via Tor Browser for sensitive browsing or via Whonix for whole-system anonymity (see os.md and the Whonix VM-compartmentalization section of devuan-secure-workstation.md).

Tor is not a substitute for the mesh tiers above. Tor anonymizes you from destinations; it does not connect your laptop to your home server.

Onion services as a hosting primitive. Beyond using Tor as a client, you can publish a service on Tor that’s reachable only through Tor. Onion services (v3, ed25519-based, 56-character addresses ending in .onion) provide end-to-end encryption to the service plus location-hiding for the host. Use cases: SSH access to a home server without exposing a public IP; private file sharing (OnionShare); whistleblowing intake (SecureDrop, GlobaLeaks). The hosting primitive is the same architecture the anonymity uses, applied in reverse.

Bridges and pluggable transports for blocked networks. When your ISP or country blocks Tor directly, the project provides two layers of evasion. Bridges are unlisted Tor relays not published in the public directory; the censor doesn’t know to block them. Pluggable transports obfuscate Tor traffic to look like something else:

• obfs4. The workhorse. Disguises Tor traffic as random bytes, frustrating naive DPI.
• Meek. Tunnels Tor through major CDN services (currently meek-azure via Azure). The censor sees you talking to Azure; blocking Azure breaks half the internet. Slower than obfs4 due to CDN routing.
• Snowflake. Routes Tor traffic through ephemeral browser-based volunteer proxies (Chrome and Firefox users running the Snowflake extension act as proxies). Each connection looks like ordinary WebRTC traffic. The censor would have to block WebRTC to block Snowflake, which is collateral-damage-heavy.

Pick obfs4 by default; Snowflake when obfs4 is blocked too; Meek when both are blocked. The Tor Browser ships all three; selection is a settings choice.

I2P

Application-aware anonymity overlay. Where Tor exits to the regular internet, I2P is mostly an internal network with its own services (eepsites, IRC, BitTorrent over I2P). The threat model is similar to Tor’s but the use cases differ: Tor is “be anonymous on the regular web”; I2P is “participate in an anonymous internal network.”

Pick I2P if you specifically need to participate in I2P-internal services. For anonymous regular-web browsing, Tor is the practical answer.

Lokinet

Onion-routing overlay developed by the Loki Project (now Oxen). Architecturally similar to Tor in being a hidden-service network but using its own service-node infrastructure rather than Tor’s volunteer network. Used as the transport for Session messenger (see choosing-communication-tools.md) and as a standalone anonymity overlay.

Capture-risk shape: token-economic incentive structure for service nodes (Oxen token), a flag for some sovereignty-minded operators who prefer the volunteer-funding model Tor uses. Smaller network than Tor by orders of magnitude, which has both privacy and reliability implications.

Pick Lokinet if: you’re already on Session and want to use the same network for general traffic, or you specifically want an alternative anonymity overlay to Tor for diversity reasons. Otherwise Tor is the more practical answer.

NYM (NymVPN)

The mixnet entry, architecturally distinct from everything else in this tier. Tor, I2P, and Lokinet are onion-routing networks: they hide who-talks-to-whom by relaying through unrelated hops, but a global passive adversary who can observe the whole network’s traffic timing can still correlate flows. NYM defends against exactly that adversary by being a mixnet: it adds cover traffic (dummy messages that carry no payload) and mixes traffic timing at each hop, so the timing-correlation attack that works against Tor doesn’t work against NYM¹⁶.

Built by Nym Technologies S.A. (Swiss), based on the Loopix mixnet design; Chief Scientist Claudia Diaz was formerly a tenured professor at KU Leuven and is one of the better-known academic figures in mixnet and network-privacy research. The architecture descends from peer-reviewed mixnet literature rather than from a startup’s whiteboard, which is the durability signal worth naming.

NymVPN (the client) offers two modes. Anonymous Mode routes through a 5-hop Sphinx-based mixnet with continuous cover traffic (the strong-anonymity, higher-latency mode), recommended for messaging, crypto transactions, and email rather than streaming. Fast Mode is a 2-hop WireGuard path (running AmneziaWG, the same DPI-obfuscated WireGuard fork covered in Tier 7) for VPN-comparable speed when you want IP-hiding without the full mixnet latency cost. Rust, open source (GPL-3), clients for Linux, macOS, Windows, iOS, Android. Anonymous onboarding via 24-word access keys; payment in BTC or XMR. v2026.4 (March 2026) added desktop ad/tracker blocking.

Capture-risk shape: the mixnet itself is decentralized with no single operator that sees the whole topology. The flag worth naming is the NYM utility token, service nodes are incentivized via a token economy (bond-and-delegate-stake model), which is the same token-economic structure flagged for Lokinet and which some sovereignty-minded operators weigh against the volunteer-funded Tor model. Whether token incentives or volunteer incentives produce a more durable network is a genuinely open question; NYM is the most serious current bet on the token-incentivized side.

Pick NYM if: your threat model includes a global passive adversary (the nation-state-scale observer that can watch traffic across the whole network), which is the specific case Tor doesn’t fully defend against. Use Anonymous Mode for the high-value low-bandwidth traffic that case implies. For ordinary anonymity against a non-global adversary, Tor remains the larger, more battle-tested network; NYM’s advantage is specifically the metadata-timing defense against the strongest adversary class.

Tier 7: Censorship-evasion proxies

Not VPNs and not anonymity overlays, these are protocol-obfuscation tools designed to make encrypted traffic survive deep packet inspection. They stack on top of the other tiers: you might run WireGuard inside Shadowsocks inside a TLS tunnel to bypass a censor that blocks WireGuard’s UDP signature.

The threat model: a censor (state-level or corporate) running DPI on egress traffic to identify and block “circumvention tools.” The defense: make your traffic look like ordinary HTTPS or QUIC so the DPI signature doesn’t fire.

Shadowsocks

Originally developed inside China in 2012 to circumvent the Great Firewall. Lightweight SOCKS5 proxy with encrypted transport; later versions (Shadowsocks 2022, AEAD-based) much improved on the original cryptographic shortcomings. Deployable on any VPS with a 5-minute setup. The model: you (in a censored country) connect to a Shadowsocks server you operate (in a non-censored country); the server proxies your traffic out.

Pick Shadowsocks for: bridging into the open internet from a censored network where you control a VPS in a non-censored jurisdiction. The classic deployment is “VPS in Singapore for someone in mainland China.”

v2ray / Xray

Multi-protocol proxy framework that includes Shadowsocks plus several other transport protocols (VMess, VLess, Trojan-Go). More flexible than Shadowsocks alone; correspondingly more complex. Xray is the actively-maintained fork of v2ray. Used widely in the Chinese circumvention community.

Pick v2ray/Xray for: more advanced censorship cases where Shadowsocks alone is being detected; situations requiring TLS-mimicry transports like Trojan or VLess+Reality.

Trojan

A proxy protocol designed to be indistinguishable from ordinary HTTPS traffic. The Trojan server speaks valid TLS on port 443; if you don’t know the password it behaves like a normal HTTPS server (typically configured to proxy to a real website). DPI can’t distinguish Trojan traffic from ordinary HTTPS because it isn’t trying to: the cover traffic is genuine.

Pick Trojan for: high-stakes circumvention where active probing (the censor sending test traffic to your server to identify it) is part of the threat model. The “looks exactly like a web server” property is harder to detect than Shadowsocks-style obfuscation.

Hysteria

UDP-based proxy using QUIC, designed for hostile-network conditions where TCP performance collapses under high packet loss. Performs significantly better than Shadowsocks or v2ray over poor links. Trade-off: UDP is more often blocked outright than TCP, so Hysteria works best where UDP traffic is allowed but TCP is throttled.

Pick Hysteria for: poor-link or high-latency environments where the throughput improvement matters. Frequently paired with v2ray for the TCP fallback case.

Outline

Open-source Shadowsocks-based proxy from Jigsaw (Google’s research arm), with management tooling that simplifies VPS-based deployment¹⁷. Targets the journalist and activist use case specifically: the Outline Manager lets a less-technical user provision proxy servers for distribution to contacts in censored networks.

Capture-risk shape: Jigsaw is part of Alphabet, so the management tooling and the Outline Manager update channel are Google-operated. The proxy server itself is open source and runs on your own VPS; the censor cannot tell Outline traffic from ordinary Shadowsocks. Pick Outline if: the management UX is the gating factor for your deployment (you’re provisioning proxies for non-technical contacts) and you accept Google’s role in the tooling chain.

AmneziaWG

A WireGuard fork designed for the DPI-survival case¹⁸. Released by the Amnezia VPN team; the protocol implementation lives at amneziawg-go on GitHub. Preserves WireGuard’s cryptographic core (Curve25519, ChaCha20-Poly1305, Noise_IK) (the security argument doesn’t change) and modifies only the transport layer to evade DPI signature detection.

AmneziaWG 1.x (2024) replaced WireGuard’s fixed 32-bit message-type headers (the 1/2/3/4 values that make WireGuard trivially identifiable) with configurable magic values, padded handshake packets to vary their size, and added pre-handshake junk packets to confuse signature-based detection.

AmneziaWG 2.0 (March 2026) added active protocol mimicry. The transport layer now disguises traffic as one of several common UDP protocols (DNS queries, QUIC sessions, SIP calls) with packet sizes that vary during transmission rather than being fixed. The DPI box sees what looks like ordinary DNS or QUIC; the censor’s signature-based detection no longer fires.

Trade-off: like Shadowsocks and the other proxies in this tier, your traffic terminates at the AmneziaWG server, which knows your IP and your destinations. The privacy frame is single-trusted-party, same as commercial VPNs in Tier 1. The censorship-survival frame is what differs. Pick AmneziaWG if: you specifically want WireGuard’s performance and operational mental model (point-to-point tunnel between machines you control) plus DPI survival in a censored network. Self-host on a VPS in a non-censored jurisdiction; deploy via Amnezia’s official tooling or one of the community installers.

Adjacent commercial-VPN-side development: Mullvad rolled out QUIC obfuscation for WireGuard on its desktop clients in 2026, disguising WireGuard traffic as ordinary QUIC web traffic to bypass DPI in heavily-censored regions. Same convergence point (WireGuard plus DPI-survival obfuscation) from the commercial-VPN direction. The category to watch is “WireGuard with protocol obfuscation”; AmneziaWG and Mullvad’s QUIC obfuscation are two implementations of it.

Tier 8: Off-grid mesh and radio-grade

When the internet is unavailable, untrusted, or actively hostile. Different physical layer from everything above.

Reticulum

A network stack designed to work over any transport that can carry packets: LoRa radios, WiFi, Bluetooth, serial connections, I2P tunnels, the regular internet. Cryptographic-identity-routed (similar conceptual lineage to FIPS): your address is derived from your public key, and the network finds you without DNS or coordinators¹⁹. AES-128-CBC end-to-end encryption with ephemeral keys per packet; sender identity hidden from relays. Active development by Mark Qvist; v1.3.0 released May 21, 2026.

The use case Reticulum nails: a single logical network spanning LoRa radio + WiFi + serial + Tor simultaneously. A remote-forest LoRa node and an urban WiFi node and a serial-link emergency relay can all be on the same logical mesh and reach each other transparently. This is the design Yggdrasil and FIPS gesture at; Reticulum implements it for the post-internet case specifically.

Pair with LXMF (the Reticulum messaging format) and Nomadnet (terminal app) for off-grid messaging. RNode is the recommended hardware (open-source LoRa transceiver).

Pick this for off-grid messaging where the threat model includes “the internet is unavailable”, disaster scenarios, remote sites, jurisdictions where bulk internet surveillance is the default. Also pick this if you specifically want a single network that bridges radio and IP transports.

Meshtastic

LoRa-based mesh, open source, hardware-focused. Cheaper and simpler than Reticulum; the standard answer for community LoRa mesh networks. Floods messages across the mesh rather than routing intelligently like Reticulum, fine at small scale, degrades at large scale (>100 nodes).

Pick this for: a community mesh in a city neighborhood, hiking-group communication, hobbyist LoRa networking. Pair with Meshtastic-compatible hardware like the LilyGO T-Beam or the Heltec LoRa boards.

MeshCore

Newer entrant (2025) targeting embedded systems with custom routing requirements. Integration at the firmware level rather than through consumer apps. Picks this if you’re building embedded mesh systems and Meshtastic’s flood-routing doesn’t fit; otherwise Meshtastic or Reticulum.

B.A.T.M.A.N. (batman-adv)

The WiFi-mesh routing protocol that real-world community networks actually run on. Where Reticulum and Meshtastic are LoRa-and-radio mesh and bitchat/Briar are Bluetooth mesh, B.A.T.M.A.N. (Better Approach To Mobile Ad-hoc Networking) is the protocol that turns a fleet of ordinary WiFi nodes into a self-organizing mesh²⁰. Developed by Germany’s Freifunk community since 2006; the batman-adv kernel module has been part of the mainline Linux kernel since 2.6.38 (2011); current release batman-adv 2025.4 (October 2025); controlled via batctl; packaged in Debian and Devuan.

Architecture: batman-adv operates at Layer 2 (it routes Ethernet frames, not IP packets), which means it emulates one giant virtual network switch spanning every node in the mesh. Every node appears link-local to every other; higher-layer protocols (IPv4, IPv6, DHCP) run on top unaware of the mesh topology underneath. The routing intelligence is decentralized by design: no single node holds the full network map; each node knows only the best next-hop toward each destination, computed from link-quality metrics (the TQ, transmit-quality value). This is the distance-vector approach that scales where flat flooding (Meshtastic’s model) collapses.

Capture-risk shape: none. It’s a kernel routing protocol, not a service; there’s no operator, no account, no coordinator. The mesh is whoever’s running batman-adv on the same physical or VPN-bridged Layer-2 segment.

Pick B.A.T.M.A.N. for: building a community WiFi mesh (the Freifunk model, neighborhood-scale resilient internet that survives any single ISP or node), bridging multiple physical sites into one Layer-2 network over WiFi or VPN transports, or any case where you want a kernel-native mesh protocol rather than an application-layer overlay. This is the heaviest-infrastructure entry in Tier 8 and the most production-proven at community scale; Freifunk has run city-scale deployments on it for over a decade. Pair with OpenWrt on the node hardware for the standard community-mesh stack. For single-link or small-peer-count cases, the overlay-network options in Tier 4 (Yggdrasil especially) are simpler; B.A.T.M.A.N. earns its complexity at the scale of dozens-to-thousands of WiFi nodes.

Briar’s offline mesh

Briar is a P2P messenger (covered in choosing-communication-tools.md) that includes Bluetooth and WiFi-direct mesh as transports alongside Tor over the internet. This puts it at the messenger / networking boundary: it’s primarily a messenger, but the transport layer it ships is real mesh networking. Latest stable release 1.5.9 (January 2024), so development has slowed; the architecture remains sound but project velocity is a flag.

For the messenger landscape including Briar’s positioning, see choosing-communication-tools.md. For purely-networking off-grid use, prefer Reticulum or Meshtastic.

What to avoid

“VPN for privacy” as a frame. It was always a marketing frame, never a privacy frame. Encrypted DNS plus HTTPS-everywhere closes most of what commercial VPNs claim to solve, without any trusted third party. Use a commercial VPN for specific narrow reasons (geographic bypass, hostile-ISP bypass, hotel-network defense); don’t use one as a generic privacy upgrade.

Free commercial VPNs that aren’t free tiers of a paid product. The business model is logging-and-selling or malware. There are no exceptions.

WireGuard “VPN apps” with closed clients. WireGuard the protocol is fine; some commercial VPN apps claim to use WireGuard but ship closed clients with proprietary modifications. If the client isn’t open source, you’re back in Tier 1 even if the marketing says otherwise.

Browser-based “VPN” extensions. Browser proxies. Useful for specific bypass cases; not a VPN in any meaningful sense. Mozilla VPN, Brave’s built-in VPN, etc. are commercial VPN re-sells, not browser-side architecture.

Tailscale or any centralized-coordinator mesh as a “privacy” tool. Convenient mesh networking, real engineering, but the coordinator sees your metadata. The privacy frame collapses when one party knows the entire topology of who-talks-to-whom.

Censorship-evasion proxies as anonymity tools. Shadowsocks, v2ray, Trojan, Hysteria, Outline are great at evading DPI; they’re not anonymity tools. Your traffic still terminates at your proxy server, which knows your IP and your destinations. Don’t conflate these with Tor.

Where to start

A flowchart of common cases.

“I want my laptop on my home network when I travel.” Self-hosted WireGuard between laptop and home server (Tier 2). If the home server is behind a CGNAT or you have multiple devices, Headscale (Tier 3) is the next step up.

“I want a few machines to all see each other.” Same answer: Headscale plus WireGuard, or NetBird if you don’t already have Tailscale clients deployed. Self-host both.

“I want anonymity from my destination.” Tor (Tier 6). Use Tor Browser for browsing; use Whonix for whole-system Tor. If your adversary is nation-state-scale (can observe traffic across the whole network), NYM’s Anonymous Mode (Tier 6) adds the cover-traffic mixnet defense Tor lacks, at a latency cost.

“I want anonymity from my ISP.” Tor again. A commercial VPN gives you “trust one company instead of your ISP,” not anonymity. If you have a specific reason a VPN works better here (Tor blocked by your ISP, sites you need block Tor exits), Mullvad in Tier 1.

“I want to bypass a geographic block.” Commercial VPN, Tier 1. Mullvad or IVPN. Pay in cash or Bitcoin.

“Tor is blocked on my network.” Tor with pluggable transports. Try obfs4 first; Snowflake if obfs4 is blocked; Meek if both are blocked. Distribute Tor Browser ships all three (Settings → Connection → Bridges).

“Everything is blocked on my network.” Shadowsocks or v2ray to a VPS you control in a non-censored jurisdiction (Tier 7). Trojan if you’re being actively probed. Hysteria if the link quality is bad. AmneziaWG if you want WireGuard’s performance and operational mental model in the same DPI-survival posture.

“I want a sovereignty-aligned mesh that doesn’t depend on any company.” Today: Yggdrasil or Nebula (Tier 4), depending on whether you prefer IPv6-overlay simplicity (Yggdrasil) or PKI-based explicit-trust (Nebula). Tomorrow (when alpha lifts): nostr-vpn plus FIPS (Tier 5).

“I want communication that works when the internet doesn’t.” Reticulum with RNode hardware for the serious case; Meshtastic for the community-mesh case (Tier 8).

“I want to build a neighborhood-scale resilient WiFi network.” B.A.T.M.A.N. (batman-adv) on OpenWrt node hardware (Tier 8). This is the Freifunk model, city-scale mesh that survives any single ISP or node failure, proven over a decade.

“I want all of the above stacked.” Reticulum can transport over Yggdrasil which can transport over Tor; the layering is the design. This is the maximalist configuration and is real ops work to operate; don’t start here unless you’ve operated each layer individually first.

Maturity table

Read the entries above for the substance; this table is the at-a-glance map.

1. Martti Malmi, nostr-vpn, GitHub repository: https://github.com/mmalmi/nostr-vpn. README description: “nostr-vpn is a Tailscale-style private mesh VPN built around a FIPS-backed data plane. It includes the nvpn CLI/daemon, a shared native app core, and native shells for desktop and mobile platforms.” Malmi’s release announcement on X, 19 May 2026: https://x.com/marttimalmi/status/2056616263925854570. Malmi’s bio: “Bitcoin dev in 2009-2011.” Current release v4.0.47 on the GitHub releases page as of 28 May 2026; the project ships multiple releases per week, so the live version number drifts faster than this doc updates. ↩ ↩2

2. jcorgan, FIPS: The Free Internetworking Peering System, GitHub repository: https://github.com/jmcorgan/fips. Project website: https://fips.network/. Announcement on Nostr (jcorgan): “FIPS is a mesh networking protocol that makes a Nostr keypair your network identity. Nodes find each other and route traffic using npubs directly. No DNS registrars, no IP address allocation, no routing authorities. Just keypairs and encrypted links.” Protocol design documentation: docs/design/fips-intro.md. ↩ ↩2

3. Mullvad blog, Mullvad VPN was subject to a search warrant. Customer data not compromised, 20 April 2023: https://mullvad.net/en/blog/mullvad-vpn-was-subject-to-a-search-warrant-customer-data-not-compromised. Documents Swedish police executing a search warrant; the company demonstrated it held no customer activity data the warrant could compel production of. ↩

4. Tracxn company profile, Tailscale - 2026 Company Profile, Team, Funding & Competitors: https://tracxn.com/d/companies/tailscale/__HoO0OVaODdbZEsDLJ7_OTsp344lrcNrb7eGx_aw5Lrk. $275M total across four rounds; Series C of $160M on 8 April 2025; valuation $778M as of May 2022. PitchBook profile documents Tailscale’s acquisition of Border0 on 17 March 2026: https://pitchbook.com/profiles/company/268781-05. ↩

5. Tracxn (same profile as above): “As of Apr 30, 2026, the latest employee count at Tailscale is 318.” ↩

6. SiliconANGLE, Secure networking startup Tailscale launches identity-linked governance for AI tools and agents, 17 February 2026: https://siliconangle.com/2026/02/17/secure-networking-startup-tailscale-launches-identity-linked-governance-ai-tools-agents/. ↩

7. Juan Font, Headscale, GitHub repository: https://github.com/juanfont/headscale. Open-source reimplementation of the Tailscale control server. v0.26.1 as of early 2026. ↩

8. NetBird, NetBird - Connect your devices into a single secure private WireGuard®-based mesh network: https://netbird.io/. Apache 2.0 licensed; both client and management server open source. Berlin-based company; founded 2021. ↩

9. Nostr Compass #15 newsletter, 25 March 2026: https://nostrcompass.org/en/newsletters/2026-03-25-newsletter/. Documents FIPS v0.2.0 adding Tor transport support, reproducible builds, sidecar example connecting through a Nostr relay, and Nostr release publishing in the OpenWrt package workflow. ↩

10. Lobsters discussion of FIPS, February 2026: https://lobste.rs/s/fxljxx/fips_free_internetworking_peering. The acronym clashes with the U.S. National Institute of Standards and Technology’s Federal Information Processing Standards, which has been the dominant referent for “FIPS” in cryptography contexts for decades. ↩

11. PearDrive project site: https://peardrive.com/. Describes a peer-to-peer file-sharing system and states the software is currently a proof of concept, not yet production-ready, with a beta launch in progress. Retrieved June 2026. ↩ ↩2

12. Holepunch, Pear Runtime launch announcement, 14 February 2024 (Pears.com): https://pears.com/news/holepunch-unveils-groundbreaking-open-source-peer-to-peer-app-development-platform-pear-runtime/. Open-source peer-to-peer app platform created by Holepunch, described as Tether-backed, with Keet as the first flagship app. ↩ ↩2

13. PearDrive CLI, GitHub: https://github.com/peardrive/PearDriveCLI. AGPL-3.0, JavaScript; npm package @peardrive/cli; runs on Pear Runtime (pear run). README documents network keys, peer public keys, QR sharing, and archive mode, and flags that network deletion is unimplemented and that file-change syncing will not work as intended until a future PearDriveCore release. Latest release v4.0.0 (29 October 2025). ↩ ↩2 ↩3 ↩4

14. Tether, Tether, Bitfinex and Hypercore Launch Holepunch: https://tether.io/news/tether-bitfinex-and-hypercore-launch-holepunch-a-platform-for-building-fully-encrypted-peer-to-peer-applications/. Funding provided by Tether and Bitfinex; Paolo Ardoino appointed Chief Strategy Officer; Keet named as the first peer-to-peer application. ↩

15. PeerDrive: http://peerdrive.org/ and https://github.com/peerdrive/peerdrive. An unrelated, earlier project, an Erlang-based distributed filesystem its authors describe as early alpha. Named here only to prevent conflation with peardrive.com. ↩

16. Nym Technologies, Nym mixnet and NymVPN: https://nym.com/ and https://nym.com/mixnet. Loopix design documentation: https://nym.com/docs/network/concepts/loopix. FOSDEM 2026 talk, NymVPN: The First Real-World Decentralized Noise-Generating Mixnet for Anonymity: https://fosdem.org/2026/schedule/event/U3UCKS-nym-mixnet/ (Chief Scientist Claudia Diaz, formerly KU Leuven). Source: https://github.com/nymtech/nym and the NymVPN client https://github.com/nymtech/nym-vpn-client. GPL-3, Rust. Two-mode design (5-hop Sphinx mixnet Anonymous Mode; 2-hop AmneziaWG Fast Mode), 24-word access key onboarding, BTC/XMR payment, v2026.4 desktop ad-blocking per March 2026 release notes. ↩

17. Outline by Jigsaw, project website: https://getoutline.org/. Open-source Shadowsocks-based proxy with Outline Manager for VPS deployment. Source on GitHub: https://github.com/Jigsaw-Code/outline-server. ↩

18. Amnezia VPN, AmneziaWG 2.0 Is Here, blog post: https://amnezia.org/blog/amneziawg-2-0-available-for-self-hosted. Open-source implementation: https://github.com/amnezia-vpn/amneziawg-go. Documentation: https://docs.amnezia.org/documentation/amnezia-wg/. Technical analysis of v2.0’s transport-layer obfuscation: https://dev.to/bivlked/amneziawg-20-self-host-an-obfuscated-wireguard-vpn-that-bypasses-dpi-4692. Mullvad’s parallel QUIC-obfuscation-for-WireGuard rollout reported by TechRadar via Yahoo, 2026: https://tech.yahoo.com/vpn/articles/mullvad-deploys-quic-obfuscation-wireguard-150302808.html. ↩

19. Mark Qvist, Reticulum Network Stack, project site: https://reticulum.network/. Reference implementation v1.3.0 released 21 May 2026. Design summary from third-party 2026 coverage: “a single Reticulum network can run simultaneously over LoRa radios + WiFi networks + I2P tunnels + serial connections.” Reticulum manual: https://reticulum.network/manual/Reticulum%20Manual.pdf. ↩

20. B.A.T.M.A.N. (Better Approach To Mobile Ad-hoc Networking), Freifunk / Open Mesh project: https://www.open-mesh.org/. batman-adv has been part of the mainline Linux kernel since 2.6.38 (2011); current release batman-adv 2025.4 (24 October 2025) per the project version history. Layer-2 routing protocol controlled via batctl; packaged in Debian and Devuan. Developed by the German Freifunk community since 2006 to replace OLSR for large-scale community wireless mesh. Freifunk overview: https://en.wikipedia.org/wiki/Freifunk. ↩

Choosing Document Scanning Tools

A comparison of the open-source tools that vet PDF, EPUB, and other documents on a Devuan workstation before you open them. Parallel in structure to choosing-hids-tools.md and choosing-encryption-tools.md. Recommends a specific stack at the end; the rest of the document is the reasoning that defends the recommendation.

If you came here from security-overview.md looking for the input-vetting specialist, the short answer is: ClamAV plus Didier Stevens’ pdfid suite plus YARA, with VirusTotal as an optional consensus check and Firejail as the open-it-safely layer. Operational companion is doc-malware-scan.sh in this project. Read the rest if you want to understand why those and not the alternatives.

TL;DR

Document scanning is the layer between “this PDF arrived” and “I’m now reading it.” It does not prevent attacks; it catches known-bad and flags suspicious-structured documents before you open them. The tools are mostly mature, mostly in Debian and Devuan repositories, and require modest configuration to be useful.

The standard stack: ClamAV for signature matching, Didier Stevens’ pdfid plus pdf-parser for PDF structural triage, YARA for rule-based pattern matching, Firejail as the sandboxed-opening layer. Four tools, each doing one thing well, plus optional VirusTotal hash lookup for multi-engine consensus. Total configuration time on a fresh install: a one-time apt install, a small Didier Stevens download, and an optional YARA-rules clone.

The operational companion is doc-malware-scan.sh in this project, which wires all of these together with first-run auto-install and folder-recursion support.

Tools that compete with ClamAV in the open-source AV space (Linux Malware Detect / maldet) lose on coverage and freshness. Tools in the same category as pdfid for PDF analysis (peepdf, mutool) are complementary rather than replacements. Sandboxed-reader alternatives to Firejail (Bubblewrap, Flatpak’s portal model) are usable but require more setup.

What document scanning means and what it doesn’t

Document scanning is a category that sometimes gets confused with antivirus generally and with HIDS specifically. It’s neither. Distinguishing the categories before naming the tools makes the trade-offs clearer.

Structural analysis. Decompose a document into its constituent objects and report on suspicious structural features. For PDFs: embedded JavaScript, launch actions, font-decoder paths historically used for exploits, embedded files. pdfid lists; pdf-parser walks. PDFs only. Latency: seconds. Output: a feature inventory. This is forensic triage, not a verdict.

Signature scanning. Match the file against a database of known-bad hashes and byte patterns. ClamAV is the open-source default. Catches known malware families; weak on novel. Latency: seconds, dominated by signature load. Output: pass or a family name on hit.

Rule-based pattern matching. Write rules that describe malware features (specific byte sequences, structural conditions, magic-number tests, embedded-string tests) and scan files against the ruleset. YARA is the de facto standard. Catches malware families that haven’t been added to AV signatures yet but match documented patterns. Latency: sub-second. Output: list of rules matched.

Multi-engine consensus. Submit a file (or, privacy-preserving, just its hash) to a service that runs many AV engines and aggregate the results. VirusTotal is the public-facing default. Catches what your single AV missed because a different engine caught it. Latency: bounded by network. Output: engine-by-engine verdict plus aggregate counts.

A serious vetting workflow uses three of the four (structural for PDFs, signatures for any format, consensus for second opinion) and treats the fourth (YARA) as optional. They are not redundant. pdfid catches a malicious PDF whose hash isn’t yet in any AV database, by reading its structure. ClamAV catches the file whose name has changed but whose body matches a known sample. VirusTotal catches what your locally-fresh ClamAV missed because a different engine has the signature. The three overlap at the edges but each has a unique core.

Plus the sandbox tier: even with all of the above passing, the safest open is in a Firejail sandbox with no network, so any active content that does execute can’t phone home or reach the filesystem outside its sandbox.

Adjacent categories worth knowing but not the focus here:

• HIDS (host integrity monitoring). Watches the system for post-compromise changes. choosing-hids-tools.md covers this layer. Different category: HIDS catches an attack after success; document scanning catches it before success.
• Endpoint AV in the Windows sense. Continuously scans every file the OS touches. Linux versions exist (ClamAV’s clamd daemon, ClamWin, ESET for Linux) but the Linux threat model rarely justifies the resource cost; on-demand scanning of documents you receive covers most of the realistic threat.
• Network-level scanning. Mail gateways and proxy filters that scan attachments before they reach the workstation. Out of scope for single-workstation usage; relevant if you operate a mail server or HTTP proxy.

The tool comparison

The tools are organized by the categories above, plus auxiliary inspection tools and the sandboxing layer that sits after the scan.

Structural analysis: PDF

pdfid.py / pdf-parser.py (Didier Stevens)

Public-domain Python scripts by a long-running malware analyst at the SANS Internet Storm Center. Not packaged in Debian or Devuan; requires manual download from didierstevens.com to a script directory of your choosing (typically ~/bin).

License: effectively public domain (Didier Stevens releases his work for general use; check the specific release).

What they do: pdfid produces a one-page summary of suspicious-tag counts (/JavaScript, /JS, /OpenAction, /AA, /Launch, /JBIG2Decode, /RichMedia, /EmbeddedFile, /XFA) plus general object counts. pdf-parser walks the PDF object graph and lets you extract specific objects by ID, filter by content, or dump decoded streams.

Strengths:

• The forensic gold standard for PDF triage. CERTs and DFIR teams reach for these first.
• Fast (seconds even on large PDFs).
• Output is small, readable, and stable across versions.
• Each tool does one thing.

Weaknesses:

• Not packaged. Manual download means manual update tracking.
• Single-developer project (Didier Stevens). Stable over twenty years but vulnerable to bus factor.

When to pick: always, for PDF triage. There is no substitute that’s both as accurate and as widely deployed.

peepdf

Active community fork of the original Jose Miguel Esparza project (the original was Python 2 and unmaintained; the active fork is Python 3 and incorporates additional features). License: GPLv3.

What it does: interactive PDF analysis. Decodes streams, detects JavaScript, runs some pattern detection against known exploits, lets you navigate the PDF object tree.

Strengths:

• More featureful than pdf-parser for deep inspection.
• Interactive shell makes complex walks faster than pdf-parser’s command-line invocation pattern.

Weaknesses:

• Upstream maintenance is forky and harder to evaluate.
• More moving parts than pdfid/pdf-parser; more places for bugs.

When to pick: when pdfid plus pdf-parser have flagged a PDF as suspicious and you want to do a deeper interactive walk. Not a replacement for the Stevens tools.

mupdf-tools (mutool)

Part of the MuPDF project (Artifex Software, behind Ghostscript). Packaged in Debian and Devuan as mupdf-tools. License: AGPLv3 (commercial alternative available from Artifex).

What it does: structural inspection of PDFs, not security-focused. mutool show file.pdf trailer shows the trailer dictionary; mutool clean -d file.pdf out.pdf rewrites with streams decompressed and visible.

Strengths:

• Lets you look at what’s actually in a stream object without pdfid’s interpretation layer.
• Fast, in the repos.
• Useful for the “is this stream what it claims to be” question.

Weaknesses:

• Not a security tool. Won’t tell you if a stream is malicious; only what it is.

When to pick: when you’ve already established that a PDF object is suspicious and you want to see its decoded content with no analysis layer in between.

Structural analysis: EPUB

EPUBs are zip archives of XHTML, CSS, JavaScript, and images. There is no dedicated EPUB malware-analysis tool. The standard workflow is extract, inspect, and scan with ClamAV (which recurses into zips automatically).

epubcheck

Java tool from the W3C / DAISY consortium. Packaged in Debian and Devuan as epubcheck. License: BSD 3-Clause.

What it does: validates EPUB conformance against the EPUB 2 and EPUB 3 specifications.

Strengths:

• In the repos.
• Surfaces malformed EPUBs that often correlate with suspicious origin (legitimate publisher EPUBs almost always pass epubcheck).

Weaknesses:

• Not a malware tool. Validates structure, doesn’t detect malice.

When to pick: as a heuristic check on EPUBs from non-publisher sources. Run alongside, not instead of, the extract-and-scan workflow.

Signature scanning

ClamAV

The dominant open-source antivirus engine. Originally a small project (Tomasz Kojm, 2001), now developed under Cisco’s Talos security organization since the 2007 acquisition of Sourcefire. Packaged in Debian and Devuan as clamav (CLI) and clamav-daemon (clamd).

License: GPLv2.

What it does: scans files against a signature database for known malware. Recurses into archives (zip, rar, 7z, tar) and document containers (Office, PDF, EPUB) natively. Two main invocation modes: clamscan loads signatures on every invocation (slow); clamdscan queries a running clamd daemon (fast). freshclam updates signatures.

Strengths:

• In the repos.
• Native archive recursion (so EPUBs and zipped attachments get scanned automatically).
• Signature updates are regular and free, no paid subscription.
• Cisco Talos backing means the project is well-funded and unlikely to disappear.
• The de facto Linux AV.

Weaknesses:

• Weak on novel zero-day malware (the limitation of any signature-based approach).
• clamscan startup is slow (~5-10 seconds for signature load) which adds up over many files; clamdscan fixes this but requires a running daemon.
• Cisco ownership is a trust consideration for users wary of US-headquartered security vendors. The signature database is open and inspectable, mitigating this for paranoid users willing to audit.

When to pick: always, as the signature-scanning layer. No other open-source option comes close.

Rule-based pattern matching

YARA

Originally a VirusTotal project (Victor Manuel Alvarez, then at VirusTotal, now at Google). Packaged in Debian and Devuan as yara. License: BSD-3-Clause.

What it does: pattern-matching engine for malware research. You write rules (specific strings, byte patterns, structural conditions, file-size and magic-number tests); YARA scans files and reports which rules matched.

Strengths:

• In the repos.
• Fast (sub-second on most files).
• The de facto malware-pattern-matching standard; almost every malware report includes YARA rules.
• Community ruleset at github.com/Yara-Rules/rules covers most known families and is regularly updated.

Weaknesses:

• The ruleset is what does the work. Without good rules, YARA is just a string-matching engine.
• Community rulesets are uneven; some rules generate false positives. Pruning is part of operational use.
• VirusTotal / Google lineage is a trust consideration similar to ClamAV / Cisco. The engine is open and self-contained, mitigating this.

When to pick: when you want to catch novel-but-pattern-matching malware that ClamAV signatures don’t yet flag. Useful supplement to ClamAV, not a replacement.

Multi-engine consensus

VirusTotal

Public service started 2004, acquired by Google 2012, now operated as a Google subsidiary. Free public API with rate limits (4 requests per minute); paid tiers for higher rates and additional features.

What it does: maintains hashes and metadata for billions of files, plus runs roughly 70 AV engines against new submissions. Two interaction modes: upload a file (file is scanned and added to the database, shared with VT’s industry partners); look up a hash (returns the existing record if the file has been seen before, otherwise NotFound).

Strengths:

• 70-ish engines is broader coverage than any locally-runnable stack.
• Mature, well-funded, fast.
• Hash lookup is privacy-safe: the hash reveals which file but not its contents.

Weaknesses:

• Uploads leak file contents to Google and to VT’s AV-vendor partners. Anything you upload becomes part of an industry-shared database.
• Google ownership is a trust consideration. The privacy policy permits significant data sharing.
• Rate limits on the free tier mean it’s not viable as a primary scan layer; treat it as a confirmation step.

When to pick: hash lookup on every file (privacy-safe, fast). Full upload only when you’ve already established suspicion and you want the strongest available second opinion, and only for files whose contents you’re willing to share with the AV industry.

Auxiliary inspection

exiftool

Long-running Perl tool by Phil Harvey (since 2003). Packaged in Debian and Devuan as libimage-exiftool-perl. License: GPLv1 or Artistic (Perl’s dual license).

What it does: reads, writes, and edits metadata in a vast range of file formats. For document analysis: exposes authoring tools, creation timestamps, embedded thumbnails, EXIF and IPTC fields.

Strengths:

• In the repos.
• Mature, well-tested, broad format coverage.
• Useful for provenance triage (does this PDF’s authoring metadata match where it claims to come from?).

Weaknesses:

• Not a malware tool. Exposes metadata, doesn’t analyze threats.

When to pick: when you want to know who made a file and when, or when you suspect fabricated provenance.

binwalk

Originally created by Craig Heffner; the project’s lineage includes a period of funding from ReFirm Labs (acquired by Microsoft in 2021). Packaged in Debian and Devuan as binwalk. License: MIT.

What it does: scans files for embedded files of known formats. Originally built for firmware analysis (finding filesystems and bootloaders inside firmware blobs); useful generally for “is there something hidden inside this thing?”

Strengths:

• In the repos.
• Fast, mature, large signature database for embedded formats.

Weaknesses:

• Not document-focused. Useful for the specific question “is there a payload hidden inside this seemingly-innocent file” but doesn’t analyze the document content itself.
• Microsoft-adjacent lineage via the ReFirm acquisition is a trust consideration for some users; the project itself remains open-source and community-maintained.

When to pick: when other tools have flagged a document as suspicious and you want to check for embedded content (an executable hidden in an image, a zip appended to a PDF).

Sandboxing for actual opening

Firejail

SUID-based namespace sandbox. Packaged in Debian and Devuan as firejail. License: GPLv2.

What it does: runs a process in a namespace sandbox with restricted filesystem access, optional network isolation, seccomp filters, and capability dropping. Profile system covers most common applications.

Strengths:

• In the repos.
• Simple command-line usage (firejail --net=none atril doc.pdf).
• Default profiles for hundreds of applications.
• Lower friction than VM-level isolation.

Weaknesses:

• SUID-root binary, which means a sandbox-escape bug becomes a privilege-escalation bug. Firejail has had such bugs in the past; the project’s response has improved but the architectural concern remains.
• Profiles can be incomplete; a misconfigured profile gives less protection than expected.

When to pick: opening untrusted documents always. The SUID trade-off is acceptable for the use case (the alternative is opening the document without any sandbox, which is worse).

Bubblewrap

User-namespace-based sandbox without SUID. Packaged in Debian and Devuan as bubblewrap. License: LGPLv2.

What it does: same general purpose as Firejail (process isolation via Linux namespaces) but without requiring SUID. Used as the sandbox foundation by Flatpak.

Strengths:

• In the repos.
• No SUID, so no privilege-escalation risk from sandbox-escape bugs.
• The Flatpak ecosystem’s sandboxing is built on bubblewrap, which means it’s heavily exercised in production.

Weaknesses:

• No profile system. You construct each sandbox invocation manually with explicit --ro-bind, --bind, --unshare-* flags.
• The construction is fiddly enough that most users won’t bother for one-off document opening.

When to pick: if you’ve decided Firejail’s SUID is unacceptable for your threat model and you’re willing to write wrapper scripts. For most users, Firejail’s profile system wins on usability.

Political and lineage clustering

Less politically charged than encryption tools — no Snowden-era controversies to navigate. The tools fall into three lineages, distinguished by funding base and project culture:

Original-author / security-community lineage. Didier Stevens’ pdfid and pdf-parser, exiftool, ClamAV in its early years. Long-running solo or small-team projects with strong individual stewardship. Stable for decades. Vulnerable to bus factor but otherwise low-controversy.

Corporate-FOSS lineage. ClamAV (Cisco-funded since 2007), YARA (Google-funded via VirusTotal), Firejail (community plus occasional corporate contributions), Bubblewrap (Red Hat / Flatpak ecosystem). The corporate funding hasn’t measurably changed project direction or compromised the open-source nature; both projects’ code is inspectable and the funding has stabilized rather than co-opted them. Trust here is mostly an issue for users who object to the parent corporation on principle (US-headquartered, large, surveillance-adjacent for ClamAV / Cisco; advertising-funded for YARA / Google).

Multi-engine consensus with privacy cost. VirusTotal (Google subsidiary). The trade-off is explicit: roughly 70 engines of coverage in exchange for your files entering a Google-owned, AV-industry-shared database. Hash-only lookups avoid the file-disclosure cost; full uploads accept it.

There’s no equivalent here to the VeraCrypt-Microsoft situation in the encryption space or to the systemd-Lennart-Poettering controversies in the init space. The trade-offs are about coverage versus privacy and about project maturity versus complexity, not about competing ideological camps.

How to think about choosing

Three questions, in order:

1. What documents are you vetting? PDFs only → pdfid plus pdf-parser plus ClamAV. Mixed PDF and EPUB → add EPUB extract-and-grep plus epubcheck. Office documents (Word, Excel, PowerPoint) → add oletools (mentioned below). Academic papers and ebooks → the vanilla stack covers; oletools if you handle Office.

2. How much configuration time can you absorb? Zero → ClamAV alone with weekly cron of Downloads. Low → add pdfid for the PDFs that actually come through. Medium → the full stack via doc-malware-scan.sh. High (this is your job) → custom YARA rules tuned to your specific threat surface, plus a clamd daemon for fast scans.

3. What’s your response posture on a hit? If a hit will be ignored or routinely dismissed, none of this matters. The minimum posture: structural-suspicion hit → don’t open without sandbox; signature hit → don’t open at all without inspection; critical hit → don’t open, quarantine, investigate origin. Pre-decide before the first hit, because the first hit will arrive at an inconvenient moment.

The recommendation

ClamAV + Didier Stevens’ pdfid suite + YARA + Firejail. Plus VirusTotal hash lookup if you have an API key (free tier sufficient for casual use).

Operational companion: doc-malware-scan.sh in this project. First run auto-installs everything; subsequent runs scan files or folders.

One-time setup (also done automatically by the script on first invocation):

sudo apt install clamav yara epubcheck firejail file unzip curl git
sudo freshclam
mkdir -p ~/bin && cd ~/bin
wget https://didierstevens.com/files/software/pdf-tools_V0_2_8.zip
unzip pdf-tools_V0_2_8.zip && rm pdf-tools_V0_2_8.zip
chmod +x pdfid.py pdf-parser.py
echo 'export PATH="$HOME/bin:$PATH"' >> ~/.bashrc
git clone https://github.com/Yara-Rules/rules.git ~/yara-rules

Optional VirusTotal API key (free tier from virustotal.com after account creation):

echo 'export VT_API_KEY="your_key_here"' >> ~/.bashrc

Weekly scan of Downloads via cron (in root’s or your own crontab):

0 6 * * 0 /usr/local/bin/doc-malware-scan.sh -y "$HOME/Downloads" | logger -t doc-scan

After every legitimate apt upgrade that bumped ClamAV, the post-install hook usually runs freshclam; if signatures look stale (older than seven days), run sudo freshclam manually.

What about other document formats

The doc-malware-scan workflow above targets PDF and EPUB explicitly. Adjacent formats are worth knowing about even if they’re not the focus of this project.

Office documents (docx, xlsx, pptx, and the legacy doc, xls, ppt). The Linux user is less likely to encounter these but the threat is real for anyone who does. oletools (apt: python3-oletools) gives Office-format equivalents to pdfid: olevba extracts macros, oledump inspects OLE objects, mraptor flags suspicious VBA patterns. LibreOffice itself will warn before running macros from untrusted documents, but the warning is bypassable by user habituation. Treat Office documents like PDFs: pre-scan with ClamAV plus oletools; if they have macros, open in a VM with no network or don’t open at all.

Archives (zip, rar, 7z, tar, gz). ClamAV recurses into archives natively, which covers the contained files. The additional concern is decompression bombs: small archives that expand to terabytes and exhaust disk or memory. Check uncompressed size before extracting (unzip -l archive.zip, 7z l archive.7z); refuse anything implausibly large. Standard apt-installed tools include the size limits as defaults but the limits aren’t enforced by unzip itself.

Images (jpg, png, webp, svg). Generally low risk on Linux for direct exploitation (image-decoder bugs do exist and have been exploited historically, but Linux image libraries are well-fuzzed). The auxiliary concerns: EXIF metadata can leak location and authoring information (use exiftool to inspect and strip); steganographic payloads can hide data inside images (steghide, stegseek to detect, rarely justified for normal threat models); SVG files are XML and can contain JavaScript (treat SVG like HTML and only open in sandbox).

HTML files. Identical risk profile to opening a web page. Open in a sandboxed browser only; never let your default reader render HTML directly without isolation.

KFX (Amazon’s Kindle format) from non-publisher sources. Harder to inspect without Calibre or Amazon’s own tools; the practical guidance is to convert KFX to EPUB first and then scan, or to read KFX only on the Kindle device itself.

Out of scope, deliberately

Real-time on-access scanning. Continuously scanning every file the OS touches (the Windows-AV model) is possible on Linux via clamd’s on-access mode but rarely worth the resource cost on a single-user workstation. On-demand scanning of documents you actually open covers the realistic threat.

Mail-gateway scanning. Workstation-level. If you operate a mail server, the same tools (ClamAV plus pdfid via amavisd or rspamd) apply at the mail gateway. Different deployment pattern, same engines.

Sandbox-execution detonation (the cuckoo-sandbox model). Run the suspicious document in an instrumented VM, observe what it does. Real category, real value for security researchers, way too much infrastructure for a workstation user. If you need it, REMnux ships pre-configured.

Specialized formats (CAD files, scientific data formats). Format-specific. Most don’t have meaningful malware exposure; the ones that do (DWG, certain GIS formats) have their own niche analysis tools.

QR-code analysis. A QR code is a URL container; the threat is the URL it encodes, not the QR code itself. Scanning the URL with a reputation service (urlscan.io, VirusTotal URL endpoint) handles the actual question. Tooling like zbarimg decodes the QR; deciding whether the decoded URL is hostile is a different category of work than document scanning. Worth knowing about; deliberately not in the doc-malware-scan workflow.

HTML email with tracking pixels. Email tracking pixels (1x1 transparent images served from a tracking domain) leak read-receipts to the sender. Defense is at the mail-client layer (block remote image loading by default; Thunderbird and Geary both support this) rather than the scanning layer. Out of scope here because the question is mail-client configuration, not document scanning.

Downloaded-binary signature verification. Verifying a downloaded binary’s GPG signature or shasum is a separate workflow from scanning a document. The procedure (download .asc plus binary, gpg --verify, or compare sha256sum against the publisher’s published value) is mechanical and covered in privacy-setup.md and gpg-concepts.md. The document-scanning workflow can apply afterward (scanning the verified-as-authentic binary for completeness) but the primary defense for binaries is signature verification, not pdfid.

Choosing Supply Chain Tools

How to defend the code you install. This is the question the other input-vetting work does not answer: choosing-document-scanning-tools.md covers the files you open, this covers the code you install and build. It sits alongside the detection layer (choosing-hids-tools.md, which catches tampering after the fact) and the credential-isolation cross-cutting concern in the master overview (which keeps a compromised build environment from acting as you). The motivation, including the IronWorm npm worm that makes this concrete, is in why-secure-your-system.md. Recommends a baseline at the end; the rest is the reasoning that defends it.

TL;DR

The code you install runs on your machine, often the instant it installs, with your privileges and your secrets within reach. That is a different attack surface from the documents you open, and the document scanner does not touch it.

The baseline, in order of return per minute of effort:

1. Prefer the distribution. Install from apt main and backports, then Flatpak from Flathub, before reaching into language ecosystems. Fewer trust roots is the cheapest win.
2. Never pipe curl into a shell, especially as root. Read installers first.
3. Pin and verify by hash. Commit the lockfile for every project and install from it with hash checking: npm ci, pip install --require-hashes, cargo build --locked.
4. Contain install-time execution. Set npm to ignore install scripts by default; for ecosystems that cannot disable it (cargo, apt), run the install or build of any code you have not reviewed inside a disposable, network-restricted, credential-free sandbox.
5. Keep the credentials that publish, sign, or unlock value off the machine that installs and builds. The most common payload is an environment-variable sweep, so the build machine should not hold your cloud, npm, AI, or wallet secrets.

No single one of these stops every attack. Depth is the point, the same as everywhere else in this project. Detection after the fact is choosing-hids-tools.md; recovery is backups, in the overview.

The shape of the threat

Almost every package manager will run code supplied by a dependency at install or build time, before you have done anything deliberate with the package. npm runs lifecycle scripts (preinstall, postinstall). pip runs the build backend of a source distribution. cargo runs build scripts and procedural macros while compiling. apt runs maintainer scripts as root. That install-time execution is the foothold; everything else is what the foothold is used for.

The IronWorm npm worm in June 2026 is the worked example, and it is worth understanding mechanically because it exercised four structural weaknesses at once¹:

• Permissive install hooks. A Rust binary ran from a preinstall script the moment the package installed, before dependency resolution finished, with no build step and no confirmation.
• Trusted-publishing abuse. It propagated using stolen credentials, including the short-lived OIDC tokens of npm’s Trusted Publishing workflow, so it could publish from a victim’s own pipeline.
• Forged commit metadata. It planted back-dated commits across nine organizations under the names of trusted automation, dependabot, github-actions, and “claude,” to blend in with routine work.
• SemVer-tolerant updates. The malicious versions were minor or patch bumps, designed to be picked up automatically by lockfiles configured to accept them.

The result was self-propagating: it stole a developer’s publish credentials, republished itself into that developer’s packages, and infected the next person who installed them. The lesson for defense is that the package manager’s convenience features (run-on-install, automatic version updates, stored publish tokens) are the attack surface, and the defenses below are mostly about removing or containing each one.

The ecosystems, one at a time

The ecosystems differ enough that one rule does not fit all. Worst-to-best for your ability to disable install-time execution: cargo (cannot), apt (cannot, and it is root), pip (partly, by preferring wheels), npm (yes, with a flag and a cost), Flatpak (sandboxed regardless).

apt and Devuan repositories

The Debian and Devuan archives are cryptographically signed, and apt verifies the release signature against trusted keys before installing anything. The main archive is not the threat. The threat is what you add to it and what you run around it.

apt runs maintainer scripts (preinst, postinst) as root at install time, so a compromised package or repository is a root-level install-time execution problem, the highest-privilege version of the npm hook issue. You cannot easily sandbox dpkg, so the defense is trusting the source rather than containing it.

What to do:

• Minimize third-party repositories. Prefer Devuan and Debian main and backports, then Flatpak, over downloading upstream .deb files or adding vendor repositories.
• When you must add a repository, install its signing key to /etc/apt/keyrings/ and bind the repository to that key with a Signed-By: line in the .sources entry, so a different key cannot silently sign updates.
• Never pipe curl into sh, and never as root. These installers bypass the package manager, its signatures, and its uninstall path entirely. Download, read, then run if you trust it.
• apt-listchanges and apt-listbugs surface what an upgrade is about to do before it does it.

Flatpak

Flatpak is the one ecosystem here whose runtime model helps you. Apps run in a bubblewrap sandbox with access mediated by portals, so a malicious or compromised app is constrained by what it was granted. The weakness is the grant: an app with broad static permissions (filesystem=host, broad device access, or the ability to talk to the Flatpak control interface) has escaped the sandbox in practice.

What to do:

• Prefer Flathub “verified” apps, which are published by the upstream project rather than a third party.
• Inspect permissions before installing: flatpak info --show-permissions <app>.
• Tighten over-broad permissions with flatpak override or the Flatseal GUI. Never leave filesystem=host on an app that does not need it.
• Treat broad device access or session-bus access as a red flag worth understanding before granting.

npm

npm gives you the most direct control over install-time execution, at a cost.

Contain install hooks. Disable lifecycle scripts globally and re-enable per project only when a package genuinely needs a native build:

npm config set ignore-scripts true

For a single install in CI or on a project you do not fully trust:

npm ci --ignore-scripts

The mitigation that Microsoft recommended after the March 2026 Axios compromise was exactly this². Two honest caveats. First, the January 2026 “PackageGate” disclosure found zero-days in npm, pnpm, vlt, and Bun that undermined lifecycle-script disabling itself, so this is a layer, not a wall³. Second, some legitimate packages need their scripts to build native code, so blanket disabling has a real cost and you will re-enable selectively.

Pin and verify. Commit package-lock.json and install from it with npm ci, which installs exactly the locked versions and verifies each package against the integrity hash in the lockfile. Do not run npm install in CI, which can drift.

Detect downgraded trust. pnpm 10.21 and later ship a trustPolicy setting whose no-downgrade mode refuses to install a package whose publish-time trust level has dropped, for example a package that used to ship with provenance and now does not, which is an early signal of a compromised account⁴.

Screen before you add. Socket and npq screen packages for suspicious behavior before installation. Check maintainer history, download counts, and repository activity, and be especially careful with package names suggested by an AI assistant, which can hallucinate a name that an attacker has since registered.

Hardening what you publish is in the cross-cutting section below.

pip and PyPI

pip’s install-time execution depends on the artifact. A source distribution runs its build backend (historically setup.py) at install; a wheel does not run arbitrary install code, though it still executes on import. Prefer wheels where you can (--only-binary :all: for the dependencies that ship them).

Pin and verify by hash. Hash-checking mode forces every dependency, including transitive ones, to be pinned with a verified hash:

pip install --require-hashes -r requirements.txt

Generate the hashed lock file with uv or pip-tools rather than by hand⁵:

uv pip compile requirements.in --generate-hashes -o requirements.txt
# or, with pip-tools:
pip-compile --generate-hashes requirements.in

Audit. Run pip-audit in CI to catch dependencies with known advisories before they ship.

Dependency confusion. If you use private packages, pin the index and use a resolver strategy that will not silently pull a public package that happens to share an internal name; uv’s first-match index strategy is built for this.

Publishing hardening is in the cross-cutting section.

cargo and crates.io

cargo is the hard case, because by design it executes arbitrary code at build time through build scripts (build.rs) and procedural macros⁶. There is no clean equivalent of --ignore-scripts, because the code runs as part of compilation rather than as an optional hook.

It is worse than that. The RustSec maintainer’s guidance is to not run any cargo command on a project you have not reviewed, because every cargo command invokes Cargo, which can be made to execute arbitrary code, and that includes the supply-chain tools themselves: cargo audit, cargo deny, and cargo vet all go through Cargo and can be turned into code execution, for instance via a .cargo/config.toml alias in the repository⁷. The practical consequence is that the auditing tools are not a safe way to inspect untrusted code; they are a way to check code you already intend to build.

So for cargo the defense is review and sandboxing, not flags:

• Commit Cargo.lock and build with cargo build --locked (or --frozen offline), which pins exact versions and verifies registry checksums.
• Run the build, and any cargo command, on code you have not reviewed inside the same disposable, network-restricted sandbox you would use for an untrusted install.
• Inside that trusted-build context, the RustSec tooling earns its place: cargo audit and cargo deny (with the cargo-deny-action in CI) flag advisories, and let you also gate licenses and crate sources⁸. cargo-vet and cargo-crev record human audit attestations. cargo-supply-chain lists who you are trusting. cargo-auditable embeds the dependency tree into the compiled binary so the result stays auditable.

Tarballs and source you build yourself

The same principle applies to a tarball or a git clone you build with make: the build runs whatever the Makefile and configure scripts say, with your privileges. Verify the download against a signature or a published checksum from the project (not from the same place you got the file), read the build files if the source is unfamiliar, and build untrusted source in a sandbox.

Cross-cutting defenses

These apply across every ecosystem above.

Pin and verify by hash

A lockfile that records exact versions and integrity hashes, committed to the repository and verified on every install, is the baseline that turns “install whatever resolves today” into “install exactly what was reviewed.” npm ci, pip install --require-hashes, and cargo build --locked are the per-ecosystem forms. Lockfiles do not fully cover git-based dependencies, so pin those to a commit, not a branch.

Disable or sandbox install-time execution

Where the ecosystem lets you disable run-on-install (npm), do it by default. Where it does not (cargo, apt, source builds), contain it instead: run the install or build of any code you have not personally reviewed inside a disposable environment that has no network it does not need and none of your credentials. A throwaway container, a VM, or Firejail per the hardening doc (devuan-secure-workstation.md) all work; Qubes makes it the default. This is the single highest-value habit in this document, because it holds regardless of ecosystem and regardless of whether a specific control was bypassed: install-time code that runs in an empty, network-restricted sandbox cannot sweep secrets that are not there.

Keep publishing and signing credentials off the build machine

This is the credential-isolation cross-cutting concern in the overview, applied to packaging. The machine that installs and builds should not also hold the credentials that publish packages, sign releases, or unlock funds, so that a compromised build environment cannot act as you.

For publishing specifically:

• Use OIDC trusted publishing instead of a stored long-lived token. PyPI Trusted Publishers exchange a short-lived OIDC token for upload rights, so there is no PYPI_PUBLISH_PASSWORD secret to steal⁹; npm’s Trusted Publishing does the same and additionally produces provenance attestations.
• If you must use a token, scope it narrowly and rotate it.
• Treat the CI pipeline as part of the supply chain: pin GitHub Actions to commit SHAs rather than tags, start workflows with minimal permissions, and require manual approval for release workflows. OIDC is not a silver bullet, because a compromised workflow file or a malicious action can trigger a legitimate token exchange and publish malicious packages, which is how the LiteLLM and GhostAction incidents worked¹⁰.
• Protect the registry account with WebAuthn-based 2FA.

Secret hygiene on the machine that installs

Because the most common payload is an environment-variable and credential-file sweep, the machine that runs npm install, pip install, or cargo build should not have your AWS, GCP, Vault, npm, Anthropic, OpenAI, or wallet secrets sitting in environment variables, shell rc files, or unencrypted dotfiles. Keep them in a secrets manager that releases them per process, or on a separate user or VM (privacy-setup.md), so an install-time sweep finds nothing. A meaningful crypto seed should never be on the build machine at all; cold storage on an air-gapped machine is the credential-isolation pattern for funds.

Screen before you add, but know the limits

Screen new dependencies with the per-ecosystem scanners: Socket or npq for npm, pip-audit or osv-scanner for pip, cargo audit or cargo deny (sandboxed) for cargo. Check the human signals too: maintainer history, repository activity, and whether the name is one an AI assistant suggested and an attacker may have registered. The limits are real: these tools catch known-bad and some heuristic-bad, not a novel implant in an otherwise-trusted package, and as noted the cargo plugins are unsafe to point at untrusted code.

Recommendation

For a single-user Devuan workstation, in order:

1. Prefer apt main and backports, then Flathub verified apps, before language ecosystems.
2. Never pipe curl into a shell; read installers first.
3. For Flatpak, review and tighten permissions with Flatseal; no filesystem=host on apps that do not need it.
4. For every language project, commit the lockfile and install from it with hash verification (npm ci, pip install --require-hashes, cargo build --locked).
5. Set npm to ignore install scripts by default; re-enable per project only when a package needs a native build.
6. Run the install or build of any dependency set you have not reviewed inside a disposable, network-restricted, credential-free sandbox.
7. Keep cloud, npm, AI, and wallet credentials off the machine that installs, per the credential-isolation pattern.

Add if you publish packages: OIDC trusted publishing with no stored tokens, WebAuthn 2FA on the registry account, provenance attestations, and GitHub Actions pinned to commit SHAs with minimal permissions and manual release approval.

Add if you hold meaningful crypto: the seed never lives on the build machine; cold storage on an air-gapped machine.

The axis driving this order is security first, then the long term, per the project defaults. Item 6 is the load-bearing one: sandboxing the build of unreviewed code is the control that still works when ignore-scripts is bypassed, when a stolen OIDC token publishes a bad version, or when the registry serves a swapped release. The rest raise the cost of an attack; item 6 contains the blast radius when one gets through.

The honest limits

No single control stops every supply-chain attack, and the marketing around each one oversells it. Trusted publishing does not help if your package manager runs preinstall scripts from every dependency, and blocking those scripts does not help if an attacker replaces a version at the registry level. PackageGate showed that the lifecycle-script-disabling defense itself had bypasses. OIDC eliminates the stored token but not the compromised workflow that mints a legitimate one. Lockfiles do not fully cover git dependencies. cargo cannot disable build-time execution at all, which is why review and sandboxing carry the weight there.

These defenses shift the odds and contain the damage; they do not guarantee safety. Detecting a compromise after it lands is a different layer (choosing-hids-tools.md), and coming back from one is backups (the overview). Depth across all of these is the posture, not faith in any one of them.

1. JFrog Security Research, “IronWorm: Shai-Hulud’s rustier cousin,” 3 June 2026, https://research.jfrog.com/post/iron-worm-shai-hulud-rustier-cousin/. Corroborated by BleepingComputer, “New IronWorm malware hits 36 packages in npm supply-chain attack,” https://www.bleepingcomputer.com/news/security/new-ironworm-malware-hits-36-packages-in-npm-supply-chain-attack/. Disclosed via the compromised asteroiddao npm account in the Arweave/WeaveDB ecosystem; reported package count ranges from 36 to 43; 86 environment variables and 20 credential-file paths targeted; 57 back-dated commits across nine organizations under automation identities including dependabot, github-actions, and “claude.” ↩

2. Microsoft Security Blog, “Mitigating the Axios npm supply chain compromise,” 1 April 2026, https://www.microsoft.com/en-us/security/blog/2026/04/01/mitigating-the-axios-npm-supply-chain-compromise/. Recommends npm ci --ignore-scripts or npm config set ignore-scripts true, and adopting OIDC trusted publishing to eliminate stored credentials. ↩

3. Bastion, “npm Supply Chain Attacks 2026: Defense Guide,” 17 February 2026, https://bastion.tech/blog/npm-supply-chain-attacks-2026-saas-security-guide, reporting the January 2026 “PackageGate” zero-days (disclosed by Koi) affecting npm, pnpm, vlt, and Bun that undermined lifecycle-script disabling. ↩

4. pnpm documentation, “Mitigating supply chain attacks,” https://pnpm.io/supply-chain-security. The trustPolicy setting with no-downgrade blocks installation of a package whose publish-time trust level has decreased. ↩

5. pip documentation, hash-checking mode, https://pip.pypa.io/en/stable/cli/pip_hash/ and --require-hashes; uv and pip-tools --generate-hashes for producing the hashed lock file, per “Defense in Depth: A Practical Guide to Python Supply Chain Security,” bernat.tech, 10 March 2026, https://bernat.tech/posts/securing-python-supply-chain/. ↩

6. Rust Security Response WG advisories for the Cargo package, https://advisories.gitlab.com/pkg/cargo/cargo: “by design Cargo allows code execution at build time.” ↩

7. Sergey Davidoff, “Do not run any Cargo commands on untrusted projects,” https://shnatsel.medium.com/do-not-run-any-cargo-commands-on-untrusted-projects-4c31c89a78d6. Any command starting with cargo can execute arbitrary code, including the cargo-audit, cargo-deny, and cargo-vet plugins, because they invoke Cargo, which can be redirected via .cargo/config.toml. ↩

8. RustSec Advisory Database, https://rustsec.org/. cargo audit scans Cargo.lock against the database; cargo deny (with cargo-deny-action) adds advisory, license, and source policy enforcement; cargo-auditable embeds the dependency tree into compiled binaries. ↩

9. Python Packaging Authority, “Publishing to PyPI with a Trusted Publisher,” https://docs.pypi.org/trusted-publishers/. OIDC short-lived identity tokens replace stored API tokens for uploads. ↩

10. PyPI security best practices and the LiteLLM and GhostAction incidents, per https://github.com/lirantal/pypi-security-best-practices and the BerriAI/litellm Trusted Publishers migration issue, https://github.com/BerriAI/litellm/issues/24542: a compromised CI/CD pipeline can publish malicious versions even with trusted publishing enabled, so pin Actions to commit SHAs, use minimal permissions, and require manual release approval. ↩

Choosing Hardware for Linux in 2026

What to put Linux on. Three sovereignty tiers — desktop you built, Linux-friendly laptop, phone with a keyboard — with a clear nudge away from Macs in any tier. Companion to the OS guide; updated as things change.

TL;DR

If you already own a working PC that isn’t a Mac, install Linux on it and stop reading. This document is for the case where you are actually buying.

Three tiers, ordered by sovereignty and by my recommendation strength:

1. A desktop you built from parts. Most sovereign hardware available at consumer prices. AMD Ryzen, AMD Radeon GPU, Intel Wi-Fi card, ECC RAM if the budget allows. No vendor in the firmware chain you didn’t put there yourself. The boring-correct pick if you have a desk.
2. Framework 13/16, current ThinkPad (X1 Carbon Gen 14, T14 Gen 7, T16 Gen 5), or a refurbished X230/T480 with Libreboot. The practical sovereign-laptop tier. Framework for modularity (swap the mainboard to upgrade the CPU generation). Current ThinkPad T-series now matches Framework on iFixit’s repairability score (10/10 at MWC 2026)¹ while remaining the keyboard and battery-life champion. Refurbished X230/T480 if you want open boot firmware and don’t mind a 2012/2018 machine.
3. Pixel 8 or newer plus USB-C dock plus external display, running GrapheneOS with its experimental desktop mode. Android 16’s polished Desktop Mode (shipped with the March 2026 Pixel Feature Drop as part of QPR3, built on Samsung DeX’s foundations per Google’s I/O 2025 announcement)² is on stock Pixel today; GrapheneOS users get experimental desktop mode usable now with the same hardware path, and the stable version is expected via Android 17 release uplift³. PostmarketOS on a Pixel, plus the dedicated Linux phones (PinePhone Pro, Librem 5), are more radical paths and are not daily-driver-ready in 2026.

Do not buy a MacBook to install Linux on. Asahi Linux work covers M1/M2 well, but M3 boots without a working GPU, M4 development stalled after Apple’s architecture changes broke the project’s reverse-engineering tooling, M5 bring-up just started, and the project founder left in early 2025 over kernel-upstreaming politics^{4 5}. You’d be paying Apple’s hardware margin for a machine optimized for an OS you intend to replace, then chasing a target Apple’s silicon team does not want you to hit.

Do not buy a Mac for general computing if sovereignty is the question this document is trying to answer. The companion piece os.md covers when keeping a Mac for one specialized workflow (video, audio, Adobe) is a reasonable concession; that concession does not extend to “buy a new Mac for general computing.”

How this fits with the OS guide

The OS guide (os.md) is upstream of this one. Pick a distro first; pick hardware to match. Most readers already have a working PC and the question “what to put Linux on” is answered by “the machine you already own, unless it’s a Mac.” This document is for the case where you are actually buying.

The sovereignty axis used here is the same one the OS guide uses: the question is not “which vendor is best” but “which vendor is in the conversation forever, and on what terms.” Apple expects to be in the conversation forever and dictates the terms. Microsoft is increasingly putting itself in the conversation through firmware (Pluton’s integration into AMD Ryzen 6000+ and selected Intel chips, pushed through Windows-certified OEM hardware). Google, on Pixel hardware, sells you the device and then leaves the software conversation entirely if you install GrapheneOS. The vendors who sell explicitly for Linux (Framework, System76, Tuxedo, Star Labs, Purism) sit lower on the vendor-presence axis. The desktop you built yourself, from commodity parts, sits lowest.

Tier 1: A desktop you built from parts

A self-assembled desktop is the most sovereign general-purpose computer you can own in 2026 short of exotica. Every component is commoditized; you can replace any single piece without replacing the rest; nothing on the board expects an OEM telemetry pipeline.

CPU. AMD Ryzen (7000-series Zen 4, 9000-series Zen 5) over Intel for Linux in 2026. AMD’s PSP (Platform Security Processor) is no better than Intel’s ME at the firmware-blob layer — both are closed-source coprocessors with their own privileged execution environments — so don’t pick AMD for that reason. Pick it because the kernel-side support story is cleaner: Ryzen’s Linux performance, idle power, and scheduler interactions track upstream tightly, while Intel’s E-core/P-core hybrid scheduling has had multiple regressions on mainline kernels. AMD Ryzen also makes ECC RAM accessible at consumer prices (covered below).

GPU. AMD Radeon over Nvidia for any machine that runs a Linux desktop. The open-source amdgpu driver is in mainline, Wayland-native, handles HDR/VRR/FreeSync, suspends and resumes correctly, and has zero out-of-tree dependencies. Nvidia’s situation has improved (GSP firmware lets the open nouveau driver work for non-gaming use, and Nvidia’s proprietary driver has gained Wayland support), but you are still chasing a moving target where the open and closed drivers behave differently, Wayland sessions have subtle issues, and suspend remains unreliable. Exception: if you do ML/CUDA work, Nvidia remains the practical pick despite the friction, and a second small AMD machine becomes the general-computing answer.

Wi-Fi and Bluetooth. Intel AX2xx-series cards (AX210, AX211, BE200) work mainline-out-of-box with nothing beyond linux-firmware. MediaTek MT7921/MT7922 are the second pick. Avoid Realtek and Broadcom — driver support ranges from “binary blob from the vendor that breaks every kernel” to “no support at all.” On a desktop you can always swap the card; a single bad pick is not catastrophic.

RAM. ECC if the budget allows. The argument is not theoretical cosmic-ray bit-flips; the argument is that ECC modules are paired with more conservative validation and chip binning, so they fail less often even when ECC correction is not actively kicking in. AMD Ryzen plus an ASRock/ASUS Pro motherboard is the cheap path to ECC at consumer prices; on Intel you’re paying Xeon tax for the same feature.

Storage. NVMe over SATA. Boring brand names with mature Linux support — Samsung 9xx Pro, WD SN850X, Crucial T700 — over budget controllers with intermittent NVMe-disconnect issues under heavy I/O.

Case, PSU, motherboard. Boring. Fractal, Corsair, BeQuiet, Seasonic. The boring choice is the right choice.

What this tier dominates. No proprietary EC firmware running parallel to the OS. No fingerprint reader or webcam talking to a Windows-only driver you’ll have to fight. Every component is replaceable. The OS install is the only software that touches the hardware. That’s the whole pitch.

What this tier costs you: portability. If you cannot accept a desktop in your home or office, skip to Tier 2.

Tier 2: Laptops

In rough sovereignty order. One note before the picks: the “Linux-certified” Dell XPS / HP / Lenovo Linux SKUs from major OEMs exist, the certification is real, the hardware works — but you’re still buying from a vendor whose primary business is shipping Windows machines with vendor telemetry baked into the firmware, and the Linux SKU is a side product. The dedicated-Linux vendors below are competitive on price and better aligned on incentives.

Framework — primary recommendation

Framework 13 and Framework 16 are the most modular mainstream x86 laptops available in 2026. The motherboard is socketed; you can swap CPU generations without buying a new laptop. The ports are modular expansion cards; you choose USB-A/USB-C/HDMI/DisplayPort/MicroSD/storage per slot. Every internal part is sold individually with public service manuals. The April 2026 Panther Lake release shipped with explicit Linux support, including Fedora and Ubuntu OEM images.

What it does well: Linux compatibility is excellent and vendor-tested before release. Battery life on the current AMD Ryzen Framework 13 is competitive with current ThinkPads. The 16“ model accepts a discrete GPU expansion bay if you want gaming or ML on a portable.

What it does less well: per-unit cost is higher than a comparable ThinkPad. The hinge on the earliest Framework 13 generations was a weak point (fixed in current units; check before buying refurbished). The 16“ model is bulky for its screen size. The keyboard is good, not great — the ThinkPad X1 Carbon remains better.

On repairability scoring: iFixit gave the current Lenovo T14 Gen 7 and T16 Gen 5 the same 10/10 rating Framework has held for years¹. iFixit measures whether parts are accessible and replaceable. It does not measure whether you can swap the mainboard for a CPU generation upgrade or whether the ports themselves are user-configurable. Framework still wins on those two dimensions; the generic “most repairable” claim now has competition.

The political flag: GNOME-side commentators have called Framework everything from “supports Fascist and Racist s***heads” (GNOME OS Team, October 2025) to “Nazibook 13 pro” (GNOME contributor Jordan Petridis, April 2026, on Mastodon), after Framework’s continued partnership with DHH (David Heinemeier Hansson) on his Omarchy distribution and its sponsorship of Hyprland. The OS guide treats this pattern as a community-politics flag for GNOME, not for Framework; from the capture-risk frame the same pattern reads as inadvertent endorsement of Framework, since the actors attacking it are the ones most willing to weaponize CoC enforcement against engineering work they oppose politically. See os.md → “GNOME’s political turn and the Code of Conduct asymmetry” for the full chronology and primary sources.

Current ThinkPad T-series and X-series

The 2026 generation refresh landed at MWC 2026 in early March: ThinkPad X1 Carbon Gen 14 (Panther Lake)⁶, T14 Gen 7, T16 Gen 5, and T14s Gen 7¹. T14 Gen 7 and T16 Gen 5 both ship with Intel Core Ultra Series 3 (vPro) or AMD Ryzen AI Pro 400 Series options, larger speakers, an optional 5MP camera, and iFixit’s 10/10 repairability rating. Lenovo officially supports Linux on most current ThinkPad SKUs (Ubuntu and Fedora are tested upstream). The keyboard is the best in the laptop industry. Fingerprint readers, webcams, and Thunderbolt work mainline on most current generations.

The flag worth naming: Lenovo is Chinese-owned (Beijing-headquartered, listed on the Hong Kong exchange), and the supply-chain argument that applies to any vendor with a national-government overlay applies here. For a single-machine threat model where state-level interest in your hardware is implausible, this is moot. For a higher-stakes threat model, it is one of the inputs.

Refurbished ThinkPad X230 or T480 with Libreboot

The sovereignty-maximalist laptop path. Both the X230 (2012, Ivy Bridge) and the T480 (2018, Coffee Lake) have full Libreboot ports as of recent releases:

• X230 has been a Libreboot staple for years.
• T480/T480s support was implemented by Mate Kukri and landed in Libreboot 20241206 (December 2024)⁷; Libreboot 26.01 (30 January 2026) refined it further with headphone-jack detection fixes⁸. Thunderbolt support landed in 26.01 RC1 but was pulled before the stable release after S3-resume regressions on some units; expect that to return in a later release.

Both machines use me_cleaner to neuter Intel ME to the legal minimum; the X230 with Libreboot additionally uses deguard to handle ME configuration, and the T480 port uses the same deguard rewrite that now supports both boards.

What you get: a laptop whose boot firmware is open, auditable, and built from source you can read.

What you give up: it’s a 2012 or 2018 laptop with the hardware ceilings of those years. On the X230, the original BIOS whitelists which Wi-Fi cards will boot; Libreboot removes the whitelist, so you can drop in a current AX210 card. The T480 has no Wi-Fi whitelist to begin with. Batteries on units this old will need replacement; community vendors still sell fresh cells.

Pick this if your threat model specifically includes “the firmware on the laptop matters and I want to verify it.” For most readers, the current ThinkPad above does the same job at much higher performance for similar money.

System76

US-based hardware company in Denver, ships Pop!_OS preinstalled, runs system76-firmware on most current models, funds the COSMIC desktop. The hardware itself is rebranded Clevo/Sager whitebooks (true of most boutique Linux vendors), but the firmware customization is real.

What it does well: ships with Linux configured correctly, the company is actively in the Linux conversation, NVIDIA driver integration is the smoothest of any vendor.

What it does less well: Pop!_OS is Ubuntu-based and the OS guide currently recommends against a fresh Pop!_OS install while COSMIC is alpha/beta. The hardware ranges from “good” to “fine” but doesn’t reach Framework’s modularity or ThinkPad’s keyboard quality.

Tuxedo Computers and Star Labs

Tuxedo Computers (German) and Star Labs (British) ship Linux preinstalled. Tuxedo offers the broader range, including AMD configurations. Star Labs makes a smaller line — StarBook, StarLite, StarFighter — and has done public work on opening their firmware (partial coreboot ports on some models). Both are smaller companies; warranty and resale logistics favor EU buyers.

Pick if: you’re in the EU and want a vendor in your jurisdiction, or you specifically value Star Labs’ open-firmware work.

Purism Librem 14

The most libre laptop available, sold by the most politically explicit vendor. PureBoot firmware (Coreboot plus Heads), hardware kill switches for camera/microphone and Wi-Fi/Bluetooth, designed around the FSF’s free-as-in-freedom criteria.

What it does well: the political posture you’re paying for is real, not marketing — the kill switches break the circuit physically, the firmware is auditable, PureBoot uses a YubiKey for tamper-evident attestation.

What it does less well: performance is a generation behind, the company has had fulfillment delays (the Librem 5 phone took years to ship after the campaign closed), and the per-unit price is high relative to specs.

Pick this if sovereignty matters to you to the point that “every component must be replaceable and every blob removable” is the floor.

NitroPad (Heads-flashed ThinkPad from Nitrokey)

The pre-flashed-with-Heads option. Nitrokey (German, open-firmware hardware-token vendor; see privacy-setup.md) sells refurbished ThinkPads — currently the X230, T430, and X1 Carbon — with Heads coreboot pre-installed and a Nitrokey USB token paired to the laptop for verified-boot attestation. The pairing is the value-add: at each boot, Heads measures the boot chain, computes an HMAC, and lights a green-or-red LED on the paired Nitrokey indicating whether the measurement matches what was last enrolled. Tamper produces a red LED before you type your passphrase.

What it does well: gets you a working Heads/coreboot machine without flashing it yourself. Flashing Heads requires SOIC clips, an external programmer (Raspberry Pi or similar), and the time to follow the (long) procedure correctly the first time. NitroPad pays for that work plus the QA. Comes with the verified-boot-token integration ready to use.

What it does less well: you’re paying for a used ThinkPad plus Heads-flashing-as-a-service. The hardware is old (xx30 generation is 2012-2013); current Heads-compatible hardware doesn’t include modern chips. Indian shipping plus import duty roughly doubles the EU street price.

Pick this if you want Heads/coreboot working today, don’t want to do the flashing yourself, and the X230/T430/X1 Carbon performance envelope is enough for your work. Order at nitrokey.com/nitropad.

Dasharo-supported hardware (the modern coreboot frontier)

For users who want coreboot on current-generation hardware rather than xx30-generation ThinkPads, Dasharo (Polish, 3mdeb) is the active modern coreboot distribution. Supported hardware as of 2026: NovaCustom laptops (NV4x, NV41 series), MSI desktop boards (the PRO Z690-A series and selected newer models), Protectli vault hardware, several Raptor Computing boards.

Dasharo is positioned differently from Heads: Heads is a Coreboot payload focused on tamper-detection-via-TPM-attestation; Dasharo is a coreboot distribution-with-firmware-stack that may or may not use Heads as a payload. Dasharo-with-Heads is supported on some boards (the NovaCustom NV4x lineup specifically). For users who want the “open firmware on current hardware” property, Dasharo is the only meaningful answer; Purism Librem and System76 are the alternatives but more vertically integrated (you buy from them or you don’t).

Pick Dasharo-supported hardware if you’re at Tier 1 (built-it-yourself desktop) and want coreboot from a vendor whose only business is the firmware. Documentation at dasharo.com.

The MacBook trap

Don’t buy a MacBook to install Linux on. Asahi Linux’s situation in 2026 is roughly:

• M1 and M2 are well supported via Fedora Asahi Remix 43, which shipped 18 March 2026 with 120Hz display and M2 Pro/Max microphone fixes. Apple Silicon Mac Pro support landed in this release⁹.
• M3 boots but the GPU does not work and KDE runs in software rendering (LLVMpipe). One driver developer in January 2026 described the state as “ONLY the internal SSD, display, keyboard, and trackpad work”¹⁰.
• M4 development has stalled. Apple’s M4 architecture changes broke the project’s reverse-engineering tooling (the SPTM-at-GL2 plus MMU-enabled-EL2 changes that Sven Peter described as “rather painful”). A Linux 6.17 regression set the schedule back further⁴.
• M5 bring-up is in its earliest phase; no public confirmation of basic Linux boot as of early 2026⁴.
• Hector Martin, the project founder, left in early 2025 over kernel-upstreaming politics — specifically the Rust-in-kernel argument and the difficulty of getting 1000+ downstream patches accepted upstream. The seven-person successor team is focused on upstreaming, not new hardware⁵.

Translated: buying a MacBook with the intention of running Linux on it means either buying an M1/M2 used machine where the work is mostly done (the realistic path) or buying an M3+ machine where you’re chasing a target Apple is actively making harder to hit. Either way, you’ve paid Apple’s hardware margin for a machine optimized for an OS you’re going to replace. Buy a Framework or ThinkPad. They’re cheaper, modular, and supported on every distro the OS guide names.

Tier 3: A phone with a keyboard

The provocative tier. PostmarketOS sits at the OS layer; the Pixel and the dedicated Linux phones sit at the hardware layer. The rungs below are organized by hardware first, with OS options inside each.

Pixel 8 or newer — the practical hardware

Pixel 8/8a/9/9a/9 Pro/10/10 Pro/10a all support DisplayPort Alt Mode over USB-C; everything before Pixel 8 lacks the hardware. The Pixel 9a at roughly ₹38,000 is the value pick; see choosing-phone.md for the device argument.

Hardware setup, common to both OS options below:

• USB-C hub with DisplayPort Alt Mode and HDMI out plus PD passthrough for charging. Anker, Plugable, and UGreen sell working models. Cheap charge-only USB-C cables do not pass video; look for “DisplayPort Alt Mode” or “USB-C 4K” on the spec sheet¹¹.
• External monitor — 1080p or 1440p in practice; the Pixel can negotiate 4K 60Hz on some configurations but most setups land at 1920x1080 or 2560x1440 today.
• Bluetooth keyboard and mouse, or a USB keyboard/mouse via the hub.

Option A: GrapheneOS with experimental desktop mode (practical)

The stable polished Android 16 Desktop Mode (QPR3 baseline, taskbar, freeform windows, multi-monitor support) shipped on stock Pixel with the March 2026 Pixel Feature Drop, built on Samsung DeX’s foundations per Google’s I/O 2025 confirmation^{2 12}. GrapheneOS users do not get that polished build immediately: Google’s 2026 AOSP cadence means QPR3 features are not flowing downstream to custom AOSP forks the way they used to, and the stable Desktop Mode on GrapheneOS is widely expected to arrive via the Android 17 release uplift³. In the meantime, GrapheneOS ships an experimental desktop mode that is usable today with the same USB-C dock plus keyboard plus mouse hardware path³. It works; it’s not as polished as stock QPR3.

What you get: a real desktop with a taskbar, resizable windowed apps, and the phone screen functioning independently — calls and messages still arrive on the phone display while the desktop session runs on the monitor. Browser with many tabs works. Google Docs, Gmail, Lightroom, and spreadsheets work. Terminal-via-Termux works.

What you don’t get: heavy creative work, gaming, or anything that wants discrete GPU power. Some Android apps adapt cleanly to windowed mouse-driven use; some are barely usable in window mode. The trackpad-emulation story is still rough.

The Termux companion: install Termux from F-Droid (the Google Play build is deprecated and stale; F-Droid is the live channel), pkg install openssh tmux mosh, ssh out to a real machine, and run your real work there. This combination — Pixel plus GrapheneOS plus Termux plus mosh to a remote desktop or cloud machine — is, for some workflows, a complete substitute for a laptop. Code review, writing, server administration, light scripting, and anything browser-centric all fit comfortably.

This setup is sovereignty-maximal in a specific way: you carry one device that is also your phone; the OS on that device (GrapheneOS) is one of the most hardened consumer OSes shipping; and your actual work lives on a machine you control somewhere else. Loss or seizure of the device costs you the device, not the work.

Option B: PostmarketOS on the same Pixel (more radical)

PostmarketOS is Alpine Linux for phones. The current stable release is v25.12 (December 2025); v26.06 is in development¹³. As of February 2026, PostmarketOS supports an estimated 723 device models, including the Pixel 3a, OnePlus 6T, Fairphone 4/5, PinePhone, PinePhone Pro, and a long tail of older hardware¹⁴. The strategic shift in 2026 is toward generic mainline kernels — one kernel image bootable across many devices via Device Tree overlays — replacing the per-device kernel-fork model that previously throttled the project’s pace¹⁴.

What it gives you: real Linux on phone hardware. GNOME Mobile, Phosh, Plasma Mobile, or Sxmo as the desktop. No Android base, no Play Services, no advertising identifier, no vendor analytics. Alpine’s apk package manager.

What it costs you: app compatibility. Most Android apps run via Waydroid (a container) with caveats; banking apps that require device attestation typically do not work, even with microG. Camera support is uneven (the camera-stack on many mobile SoCs has no mainline driver). Cellular modem support varies by device. Independent coverage in 2026 characterizes the platform as “a credible development target and an increasingly functional enthusiast platform” rather than a daily-driver replacement for Android or iOS¹⁴.

Pick this over Option A if: you want full Linux on the phone hardware itself rather than hardened Android, you can tolerate the rough edges, and you’re prepared to roll back to GrapheneOS or stock if something doesn’t work. For most readers, Option A is the better trade.

Dedicated Linux phones — PinePhone Pro and Librem 5 (most radical)

The PinePhone Pro (Pine64, approximately $399 base) and Librem 5 (Purism, approximately $799 with current promos) are phones designed for Linux from the ground up. PinePhone Pro runs PostmarketOS, Mobian, or one of several other distros; Librem 5 runs PureOS by default. Both have hardware kill switches for the radios and cameras — a feature no Pixel offers.

What you get: an open Linux mobile device with hardware designed around the open Linux mobile stack. The kill switches are physical circuit breaks, not software toggles.

What you give up: battery life is measured in hours, not days (PinePhone Pro especially is known for 3–5 hours of active use). Camera quality is below 2019 Android-flagship level. Cellular reliability ranges from “fine on common bands” to “intermittent.” Purism’s fulfillment history is poor (multi-year delays on the original Librem 5 deliveries). Software maturity remains rough enough that long-time users describe both phones with affection-tinged frustration.

Pick this if: hardware kill switches are non-negotiable, the phone is primarily a Linux mobile device rather than a working daily phone, and you have patience for the maturity gap.

For most readers asking “phone as workstation,” the Pixel hardware path above is the answer.

What to avoid

• Microsoft Surface devices. Linux runs (the linux-surface kernel-fork project exists and is competent), but Microsoft’s hardware roadmap does not target you, touch and pen integration are half-supported, and you’re paying the Surface premium to fight the platform.
• Laptops with Nvidia Optimus (discrete GPU plus Intel iGPU switching). Suspend-resume is unreliable on most Linux configurations, switching the active GPU is fiddly, and battery life suffers. If you need Nvidia, prefer a discrete-only laptop or a desktop.
• Anything with a vendor-locked bootloader you can’t disable. Most Chromebooks (though the Crostini path exists for those who commit to it), Microsoft Surface laptops with Pluton enforced, and “AI PC” hardware with vendor-mandated NPU firmware running continuously.
• 2-in-1 / tablet hybrids. Touch support on Linux is competent on GNOME and KDE but worse than on Android, iPadOS, or Windows. If you want a tablet, get an iPad and accept it for what it is, or get a phone with desktop mode (Tier 3 above).

The MacBook case has its own subsection inside Tier 2 (#the-macbook-trap) and is not repeated here.

1. Engadget, Lenovo’s ThinkPads get a spec bump at MWC 2026, 1 March 2026: https://www.engadget.com/computing/laptops/lenovos-thinkpads-get-a-spec-bump-at-mwc-2026-230100419.html. Confirms ThinkPad T14 Gen 7 and T16 Gen 5 starting at $1,799 with Intel Core Ultra Series 3 (vPro) or AMD Ryzen AI Pro 400 Series CPUs, optional 5MP camera with computer vision/vHDR, larger speakers, and an iFixit 10/10 repairability score. T14s Gen 7 starts at $1,899. Most devices shipping Q2 2026. ↩ ↩2 ↩3

2. Techlicious, Turn your Pixel into a PC with the new Desktop Mode, 26 March 2026: https://www.techlicious.com/tip/pixel-desktop-mode/. Plugable, How to Use Android 16 Desktop Mode with a Pixel Phone and USB-C Hub or Adapter, knowledge-base article current as of 1 April 2026: https://kb.plugable.com/how-to-use-android-16-desktop-mode-with-a-pixel-phone-and-usb-c-display-or-hub. Confirms Desktop Mode requires a Pixel device that supports DisplayPort Alt Mode over USB-C (Pixel 8 series and newer); shipped on stable Android 16 (March 2026 Feature Drop, Android 16 QPR3 baseline); Google confirmed at I/O 2025 the implementation is built on Samsung DeX’s foundations. ↩ ↩2

3. PiunikaWeb, Why GrapheneOS users will have to wait for stable Android 17 to get Google’s new desktop mode, 16 April 2026: https://piunikaweb.com/2026/04/16/grapheneos-android-17-desktop-mode/. Explains that QPR3 made the polished desktop session generally available on supported Pixel and Samsung devices, but that Google’s 2026 AOSP cadence has slowed downstream propagation to custom OS projects; GrapheneOS already has an experimental desktop mode usable today with USB-C to HDMI/DP setup plus keyboard and mouse, with the stable user-facing option expected when GrapheneOS moves to Android 17. ↩ ↩2 ↩3

4. Phoronix, Asahi Linux Has Experimental Code For DisplayPort, Apple M3/M4/M5 Bring-Up Still Ongoing, 31 December 2025: https://www.phoronix.com/news/Asahi-Linux-EOY-2025-CCC. Coverage of Sven Peter’s 39C3 presentation. Documents that M4/M5 changes have broken the existing Asahi Linux reverse-engineering tools, that the display controller and GPU driver remain the biggest pieces not yet upstreamed for M1/M2, and that M4/M5 Linux support is expected to take significant additional time. Original Sven Peter Mastodon post from April 2025 characterizing M4 support as “rather painful” referenced in earlier Phoronix and Apple Insider coverage; SPTM-at-GL2 / MMU-at-EL2 detail from the same post. ↩ ↩2 ↩3

5. How-To Geek, Asahi Linux Gets a Reboot, Still Working On M3 & M4 Mac Support, 13 February 2025: https://www.howtogeek.com/asahi-linux-reorganization-m3-m4-mac-support/. Documents Hector Martin’s departure from the Asahi Linux project, the seven-person successor team’s organizational restructure, and the focus on upstreaming the 1000+ downstream patches before prioritizing new hardware. Quotes Martin on the difficulty of being “in a position to have to upstream code across practically every Linux subsystem, touching drivers of all categories as well as some common code”; also covers the Rust-in-kernel argument as a contributing factor. ↩ ↩2

6. NotebookCheck, Now available to order in many countries: Panther Lake powered Lenovo ThinkPad X1 Carbon Gen 14 releases, 9 March 2026: https://www.notebookcheck.net/Lenovo-releases-new-14-inch-ThinkPad-globally-with-120-Hz-VRR-OLED.1246063.0.html. X1 Carbon Gen 14 with Intel Panther Lake replaces the Lunar Lake-based Gen 13. Three Thunderbolt 4 ports, 58 Wh battery, optional 120 Hz VRR OLED display. Configurable without an OS at a £50/€60 discount. ↩

7. Libreboot release notes, Libreboot 20241206 released! ThinkPad T480 added. Plus U-Boot UEFI on x86. Fixes for OptiPlex 3050 Micro., 6 December 2024: https://libreboot.org/news/libreboot20241206.html. T480/T480s support implemented by Mate Kukri with testing and hardware logs provided by Leah Rowe; uses Mate’s rewritten deguard for ME configuration, which now supports the 3050, T480, and T480S with machine-specific configurations. Install procedure: https://libreboot.org/docs/install/t480.html — both me_cleaner and deguard applied; Intel graphics, internal screen, ethernet, USB, WLAN, HDA verbs working. ↩

8. Libreboot release notes, Libreboot 26.01 RC1 “Tenacious Tomato” released!, 25 December 2025: https://libreboot.org/news/libreboot2601rc1.html. Refinements for T480/T480s including headphone-jack detection (pavucontrol no longer required to switch the port manually). Linuxadictos coverage, Libreboot 26.01 expands support to HP Pro 3500, Topton X2E N150, ThinkPad T580 and Dell Latitude E7240, 5 February 2026: https://en.linuxadictos.com/Libreboot-26.01-expands-support-to-HP-Pro-3500--Topton-X2e-N150--ThinkPad-T580--and-Dell-Latitude-E7240.html. Documents 26.01 stable on 30 January 2026 and notes that T480/T480s Thunderbolt support was added in RC1 then removed before the final 26.01 release due to S3-resume regressions on some units. ↩

9. LinuxTeck, Fedora Asahi Remix 43 Arrives — And It’s The Most Complete Apple Silicon Linux Release To Date, 19 March 2026: https://www.linuxteck.com/fedora-asahi-remix-43-apple-silicon/. Documents 18 March 2026 joint release of Fedora Asahi Remix 43 by the Asahi Linux project and the Fedora community: Mac Pro with Apple Silicon support added; 120Hz display refresh on MacBook Pro; M2 Pro/Max internal microphone fix; RPM 6.0 plus DNF5 packaging. ↩

10. AppleInsider, It’s not usable yet but Asahi Linux runs on M3 Macs now, 27 January 2026: https://appleinsider.com/articles/26/01/27/its-not-usable-yet-but-asahi-linux-runs-on-m3-macs-now. Reports Fedora 43 Asahi Remix running on an M3 Mac with KDE Plasma in software rendering (LLVMpipe); contributor IntegralPilot posted the photo. Quote on M3 state (“Basically ONLY the internal SSD, display, keyboard, and trackpad work”) attributed to one of the driver creators. ↩

11. Plugable, How to Use Android 16 Desktop Mode with a Pixel Phone and USB-C Hub or Adapter: https://kb.plugable.com/how-to-use-android-16-desktop-mode-with-a-pixel-phone-and-usb-c-display-or-hub. Names DisplayPort Alt Mode as the required cable/hub feature and confirms that charge-only USB-C cables will not pass video. ↩

12. Droid Life, 2026 March Pixel Update is here, what’s in it?, 3 March 2026: https://www.droid-life.com/2026/03/03/2026-march-pixel-update-download/. Records the March 2026 Pixel Feature Drop as both a Feature Drop and the quarterly Android 16 QPR3 update. ↩

13. postmarketOS, Install postmarketOS (current stable version): https://postmarketos.org/install/. Confirms v25.12 as current stable; v26.06 in development per the 10 May 2026 update post: https://postmarketos.org/blog/2026/05/10/pmOS-update-2026-04/. ↩

14. SitePoint, State of Linux Mobile 2026: PostmarketOS & F-Droid Updates, 27 February 2026: https://www.sitepoint.com/postmarketos-fdroid-2026-status/. Source for the “credible development target and an increasingly functional enthusiast platform” characterization and the estimate of ~723 supported device models as of February 2026. Also documents the generic mainline kernel strategy and the postmarketOS 25.06 reorganization of device categories. ↩ ↩2 ↩3

Devuan Secure Workstation

A complete install-plus-hardening procedure for a single-user Devuan workstation with full-disk encryption. Covers everything from the USB stick to a hardened daily driver. Written for someone who already chose Devuan per the OS guide and wants the secure default rather than the default default.

The companion executable devuan-luks2-install.sh performs the partitioning, LUKS setup, debootstrap, and base configuration. This document explains what that script does, what to do before running it, and the post-install steps the script deliberately does not perform.

TL;DR

The script gets you a working LUKS-LVM Devuan install with a single passphrase prompt at boot. After first login you do this, in order:

1. Back up the LUKS header off the disk before anything else.
2. Set a GRUB superuser password.
3. Set a firmware (BIOS/UEFI) supervisor password.
4. Set up off-machine encrypted backups with Borg, before adding any data you care about.
5. Install CPU microcode for your vendor.
6. Update vendor firmware (BIOS, Thunderbolt, NVMe) via fwupd where available.
7. Set up unattended security-only updates.
8. Apply kernel hardening sysctls and boot parameters.
9. Install and enable AppArmor profiles.
10. Set up MAC address randomization for WiFi and Ethernet.
11. Switch to encrypted DNS via dnscrypt-proxy.
12. Install USBGuard with a whitelist of your existing devices.
13. Authorize Thunderbolt devices explicitly via boltctl and tighten IOMMU settings (if your hardware has Thunderbolt).
14. Set up a default-deny host firewall with nftables.
15. (Optional, breaks things) Enable kernel lockdown mode.

Items 1 through 14 are mandatory for any threat model that includes a stolen laptop or a network you don’t fully control. Item 15 trades capability for reduction in post-compromise damage; enable it only if you understand what it costs.

What this produces

A Devuan workstation with the following properties when complete:

• Full-disk encryption with a single passphrase prompt before GRUB
• Encrypted swap (the swap LV sits inside the LUKS container)
• Encrypted /tmp and /home (likewise inside LUKS) plus tmpfs /tmp for runtime
• No systemd (sysvinit, OpenRC, or runit as the init system; default sysvinit)
• A signed bootloader chain you can verify against the LUKS header backup
• AppArmor mandatory access control on top of standard Unix permissions
• Kernel hardened against the common local-privilege-escalation paths
• Off-machine encrypted backups via Borg
• Encrypted DNS resolution
• MAC randomization on network interfaces
• USB device whitelist enforcement
• Thunderbolt devices require explicit authorization on first connect (where applicable)
• Off-machine alerts on host-integrity changes (see the companion HIDS guide)

What this is not. Not a Tor-routed system by default (use Tails or Whonix for that — Part 4.5 covers the Whonix-on-this-workstation option). Not amnesic (use Tails). Not the maximalist VM compartmentalization model (use Qubes for that). Not signed-boot-enforced from firmware (Secure Boot is off; see the trade-off in the next section). The threat model assumes a competent attacker with physical access to a powered-off laptop, network-level observers, and ordinary malware — not a nation-state with hardware implants.

Architectural costs

Three deliberate choices the install makes against the most-paranoid alternative, so you know what you’re trading.

LUKS2 with PBKDF2, not Argon2id. Argon2id is the modern key derivation function (memory-hard, side-channel-resistant). PBKDF2 is older and weaker against GPU attacks. The install uses PBKDF2 because GRUB’s LUKS2 support does not yet handle Argon2id; if you format with Argon2id, GRUB cannot read /boot and the system does not boot. Two ways out: put /boot on a separate USB stick that you don’t unlock via GRUB (covered in Part 4.2, full Argon2id becomes possible), or accept PBKDF2 with a long, high-entropy passphrase. The script picks the second, with PBKDF2 hardened by LUKS_ITER_TIME=5000 (default) — five seconds of derivation per guess, ~2.5× the cryptsetup default. This is the strongest PBKDF2 setting that doesn’t pessimize boot.

This constraint has an expiry date. GRUB 2.14, released 13 January 2026, adds native Argon2 KDF support. Devuan Excalibur (and Debian Trixie behind it) ships GRUB 2.12 and will keep PBKDF2 for the lifetime of this release. The next Devuan stable (Freia, based on Debian Forky, no fixed release date yet) is expected to ship GRUB 2.14 or later, at which point the PBKDF2 workaround stops being necessary and a re-encrypt to Argon2id (cryptsetup luksConvertKey --pbkdf argon2id) becomes the cleanup task.

Secure Boot off. Devuan does not ship signed shim/grub binaries for Secure Boot the way Ubuntu and Fedora do. Leaving Secure Boot on would mean either signing your own bootloader chain (real work, breaks on every kernel update unless automated) or running an unsigned bootloader that Secure Boot then refuses. The script assumes Secure Boot is off. The cost: a sufficiently determined evil-maid attacker can install a malicious bootloader. Mitigations covered in Part 4.

No TPM-sealed unlock by default. A TPM2-sealed LUKS key with PCR binding gives unlock-without-typing-the-passphrase, with the unlock conditional on the firmware and bootloader being unmodified. It’s a real security improvement but it interacts badly with firmware updates and kernel updates, and the recovery path is harder. The default install uses passphrase prompt. TPM2-sealed unlock is covered as an optional upgrade in Part 4.1.

GRUB 2.14 also adds a native TPM2 key protector. On the current Excalibur GRUB 2.12, the TPM2 enrollment path is clevis-luks-bind plus clevis-tpm2 (Part 4.1). Once Devuan Freia ships GRUB 2.14, native TPM2 unlock at the bootloader level becomes the cleaner path and clevis-on-Devuan becomes the legacy approach.

What’s outside encryption

Exactly one partition: the EFI System Partition (ESP), /dev/<disk>1, 1 GiB, FAT32, mounted at /boot/efi. Everything else — the kernel, the initramfs, /boot proper, /, /home, swap — sits inside the LUKS2 container on /dev/<disk>2. The LUKS header (the first ~16 MiB of <disk>2) is also on disk but is encrypted-metadata, not plaintext.

The ESP cannot be encrypted in this architecture because UEFI firmware needs to read it before any decryption could happen. This is the structural reason “evil maid” is the residual threat — an attacker with physical access to a powered-off machine can modify grubx64.efi on the ESP to inject a passphrase-capturing payload, and your next boot types the LUKS passphrase straight into their malware. Secure Boot would defend against this by refusing to load a tampered grubx64.efi, but Devuan ships unsigned GRUB so Secure Boot is off, so this defense is unavailable in the stock setup.

The genuinely strongest configurations for “nothing sensitive in cleartext on the laptop’s internal storage”:

Genuine “everything inside encryption including the ESP” is impossible because something has to boot first. The closest practical answer for “nothing sensitive in cleartext on the laptop’s internal storage” is /boot on a separate USB key (Part 4.2); the closest answer for “even the firmware loading the bootloader is integrity-protected” is Heads/coreboot (Part 4.3). The Knots-lens / cold-storage operator practice converges on running one or both.

A note on the script’s GRUB install behavior. The script writes the bootloader to /EFI/devuan/grubx64.efi and creates an NVRAM boot entry pointing at it. With INSTALL_REMOVABLE_PATH=true it also writes /EFI/BOOT/BOOTX64.EFI (firmware fallback path), which adds reliability (NVRAM-clear by firmware update or motherboard battery failure still boots) at the cost of one more ESP file an attacker with physical access could swap. Default is false for minimum attack surface. The reliability cost is asymmetric: if NVRAM gets cleared without the fallback, you boot from the install USB and run grub-install again, ~5 minutes’ work. The attack-surface cost is permanent until the file is removed. Set the toggle to true only when your firmware is known to ignore NVRAM (some older Macs, some buggy UEFI), or when the disk needs to boot on multiple machines.

Order of operations

This document is structured so that you can read it once, top to bottom, on a fresh install. The order matters: later parts assume the earlier parts are done. If you skip ahead, check the section preamble for prerequisites.

Part 1: Install. Pre-install preparation, then run devuan-luks2-install.sh.

Part 2: First-boot essentials. Three things you must do before anything else (LUKS header backup, GRUB password, /tmp tightening that the install script already configured but you should verify).

Part 3: Runtime hardening, in priority order. Each part is independently applicable.

Part 4: Architectural upgrades. Optional, more disruptive, do these last.

Part 5: Recovery from failed boot. Reference material for when something breaks.

Part 1: Install

1.1 What you need before running the script

• A target machine that boots UEFI (the script refuses non-UEFI). Disable Secure Boot in firmware setup before booting the installer.
• A spare USB stick of at least 4 GB to write the Devuan installer to.
• A second USB stick of at least 1 GB for the LUKS header backup (Part 2.1). Can be smaller; a 1 GB stick is just what you’ll find lying around.
• A passphrase plan. The LUKS passphrase needs to be long and memorable and you must not lose it. See the Encryption Tools companion guide for guidance on passphrase construction and backup.

1.2 Downloading and verifying the installer ISO

Get the Devuan netinst ISO from https://www.devuan.org/os/download for the current stable release. As of this writing that’s Excalibur 6.1 (point release dated 2 January 2026), based on Debian Trixie, shipping Linux 6.12 LTS, cryptsetup 2.7.x, and GRUB 2.12. Devuan publishes SHA256SUMS and SHA256SUMS.asc alongside the ISO. Verify both.

sha256sum -c SHA256SUMS --ignore-missing
gpg --verify SHA256SUMS.asc SHA256SUMS

If gpg complains it doesn’t know the signing key, fetch it from the Devuan keyring page (linked from the download page) and verify the key fingerprint against multiple independent sources before trusting it. The fingerprint of the Devuan release key has been stable; cross-check it against the project’s official channels (devuan.org, the Devuan announce mailing list archive, the dev1galaxy forum) rather than a single source.

Write the ISO to USB:

sudo dd if=devuan_excalibur_amd64_netinst.iso of=/dev/sdX bs=4M status=progress conv=fsync

Replace sdX with the actual device. Get this wrong and you overwrite a different disk. lsblk before, lsblk after.

1.3 Boot the installer in live mode

Boot the target machine from the USB stick. At the Devuan installer menu, choose “Live with Xfce” (or another live option). You want to get to a shell, not run the graphical installer; the script handles partitioning and base install in one pass without the installer’s interactive prompts.

Once at the desktop, open a terminal. The user is devuan with sudo. Confirm internet works (ping -c 2 deb.devuan.org) and that you booted in UEFI mode:

ls /sys/firmware/efi

The directory should exist. If it doesn’t, reboot into firmware setup and switch to UEFI mode.

1.4 Get the install script onto the live system