What server maintenance actually covers

The work you only notice when it's missing

Nearly all of maintenance exists so that nothing happens, which is exactly why it is the first thing to get cut. Go six months without a scare and it starts to feel like the server looks after itself. Usually it is the other way round: it has been quiet for six months because someone has been on it.

Real trouble rarely arrives as a dramatic break-in. It arrives because a disk filled up with logs and a service could no longer write, because a certificate expired with nobody watching, or because a patch published three months ago still hasn't been applied and somebody found the hole before you did. None of that gives you warning if nobody is watching.

Maintaining a server is, in large part, staying ahead of that dull little list before any item on it turns into an out-of-hours call.

A backup is only proven when it restores

Everyone takes backups. The trouble shows up the day you have to use one. That is when people find the dump had been quietly truncating for weeks, that the scheduled job was failing in silence, or that the key to decrypt the copy lived on the very machine that just died.

Nobody has a backup problem. Everybody has a restore problem.

Maintenance that takes this seriously does three things. It automates the copies so they don't hang on anyone remembering. It keeps at least one off the server, somewhere else, because a copy sitting on the same disk is no help when that disk is the thing that fails. And it restores one now and then into a separate environment: that is the only thing that confirms the data is whole and the process works end to end.

Updating without the drama

Software changes for two very different reasons, and they deserve different handling. There are security patches, which close holes that are already known and public — those go in quickly, because time is on the side of whoever is scanning for machines that haven't caught up. And there are new releases with big features or changes, which can break what already worked and call for more care.

The sane way to apply a risky change is to try it first on a copy of the environment, keep a snapshot or a way back in case it goes wrong, and roll it out when fewer people are using the service. When a component is fragile, pinning a known-good version and not chasing the latest is often the right call: a point behind and running beats bang up to date and down.

One case gets forgotten a lot: updates to the system core only take full effect after a reboot. Leaving that reboot on automatic can drop the service with no warning at all; leaving it pending forever means carrying a hole that was already patched. Maintenance decides when that reboot happens rather than leaving it to chance.

Who holds a key

Given time, the list of who can log into a server grows on its own. A contractor's key goes in for one job, a colleague is let in for an emergency, a “temporary” account is opened and never closed. Months later half a dozen keys are still there and nobody remembers whose they are.

Reviewing access is part of the job: who can get in, with what rights, and why they still can. The sensible baseline is logging in by key rather than password, direct root login switched off, and every key pulled the moment it stops being needed rather than kept “just in case”.

The firewall is the other half. The rule that ages well is plain: open only what has to be open, closed everything else. Every exposed port is a door somebody will try; if a service is only for you, it has no business listening to the whole internet. On top of that sits automatic blocking of addresses that keep hammering with failed logins, so brute force meets a wall long before it lands.

Every task has its own rhythm

Maintenance isn't a yearly review you do once and forget. It is a set of small tasks, each on its own clock: some want attention daily, others are fine once a month. Here is the shape of a reasonable maintenance list, where an empty box is something still to check, not something already handled:

• Daily Monitoring with alerts — an alert fires when the server stops answering, instead of you hearing it from an annoyed user.
• Daily Disk space — a disk at 95% is an outage with a date on it; the warning has to land well before you get there.
• Weekly Log review — reading the logs for errors that repeat, logins that don't add up, and odd spikes that haven't surfaced yet.
• Weekly Security patches — applying the critical system updates before they pile up.
• Monthly Test restore — having the backup isn't enough; you restore it for real and check the data is intact.
• Monthly Access and firewall — going over who can get in and through which ports, and pulling keys and rules that no longer belong.
• Quarterly Credential rotation — changing service passwords, tokens and keys that have sat untouched too long.
• Quarterly Response drill — walking the “it's down” plan while everything is calm, so nobody is improvising on the day it matters.

The frequencies aren't sacred: a busy service will want eyes on the logs every day, a quiet one can stretch further. What doesn't change is that each task has an owner and a slot on the calendar. Anything that isn't scheduled doesn't get done.

Seeing trouble coming

Good maintenance hears about things before the users do. That means continuously tracking a handful of numbers that tell most of the story: how much CPU and memory are in use, how much disk is left, how long the service takes to answer, and whether it is even up. When one drifts out of its normal range, an alert goes to a person, not to a dashboard nobody is looking at.

The textbook case is the disk. It fills slowly and silently until, one night, one more log line won't fit and the service falls over in a chain. With a warning at 80% that is a calm reminder on a Tuesday afternoon; without it, it is a 3 a.m. outage. The same goes for certificates: they expire on a known date, so there is no excuse for one to catch anyone off guard.

Logs are the other early-warning system, and the most underrated. A trickle of login attempts from the far side of the world, an error that starts repeating, a scheduled job that has been failing quietly for days — it is all written down there before it becomes something anyone can see. Reading them regularly is what turns a nasty surprise into an early heads-up.

A plan for the bad day

However well kept it is, a server can still go down: a hardware fault, an outage at the provider, plain human error. The difference between a ten-minute scare and a lost day isn't luck; it is having decided in advance what you do.

That plan answers concrete questions before they are urgent. How often is a backup taken, and therefore how much data could you lose at worst? Where is the off-site copy, and how, step by step, is it restored? Who finds out something is wrong, and who decides what to do about it? When those answers are written down, recovery is following a script; when they aren't, it is improvising under pressure, which is exactly when the expensive mistakes get made.

This is where the dull work of the previous months earns its keep. The off-site copy exists and has been restored before, so it works. The procedure is written down, so it doesn't hinge on one particular person picking up the phone. Getting back on your feet becomes a matter of time — and not much of it.

How to tell if yours is looked after

You don't have to be technical to sense whether a server is being cared for. A couple of questions usually do it. When was a backup last restored to prove it works? Who gets the alert if the service goes down overnight, and how quickly do they act? Is there a list of who has access, and has anyone looked at it this year? If those questions land on an awkward silence, the server isn't broken — not yet — but it is running on borrowed time.

The good news is that catching up is almost never starting from scratch. It is usually tidying what is already there: automating the copies that were being done by hand, adding the alerts that are missing, closing the old accounts, and writing down the plan that until now lived in somebody's head.