Rules
Part of Reading security compliance training with a sceptical eye
Why a security compliance training programme quietly stops working
Why security training programmes stop working, what the completion figure was hiding in each case, and the change that fixes it without buying anything new.
A training program almost never fails loudly. It reports high completion, produces a certificate, and the business finds out it achieved nothing on the morning somebody says they thought the message looked odd three days ago and did not want to bother anyone.
Each failure below is described by what the numbers said at the time, because that is what makes these hard to spot. The fixes are mostly free.
What to take away
- Completion is the figure that hides every failure on this page.
- Silence is not safety. A program producing no reports is a program that has failed.
- Almost every fix here is a change to friction or to tone, not to content.
Everyone completes it and nobody reports anything
What the numbers said: attendance near perfect.
What was happening: people finished the module and had no route, or had one that required remembering an address.
The fix: a reporting action that takes one click from where people already are, and a session that consists mostly of using it once. Measure reports per month from then on and treat completion as an administrative detail.
Reports stopped after somebody was made an example
What the numbers said: an early rise, then a steady decline that looked like fatigue.
What was happening: one person was named, corrected publicly, or included in a list of who clicked something. Everybody watched.
The fix: say plainly that reporting is welcome including for things you did yourself, have the most senior person say it, and honor it visibly the next time. Recovery from this takes months, which is why the punitive approach is never worth trying once, as phishing defense argues about blameless reporting.
The annual module is clicked through at speed
What the numbers said: completed, in under the expected time.
What was happening: a ninety-minute obligation competing with actual work.
The fix: ten minutes a month on one specific thing, ideally something that happened to your own business. Keep the annual module only where an outside party insists on a quantity, and stop expecting it to change behavior.
The material is about a company nobody recognizes
What the numbers said: good scores on the quiz.
What was happening: generic scenarios about a fictional organization, which people process as a test rather than as a warning.
The fix: one real story a quarter, anonymized where necessary. People remember the supplier and the invoice, and they repeat the story to each other, which no module achieves.
The people who most need it are excluded
What the numbers said: completion measured across staff accounts.
What was happening: contractors, temporary staff, the bookkeeper who comes in on Thursdays, and often the owner were never in the list. Several of them hold access that matters, and the owner is the person most often impersonated.
The fix: build the list from who has access rather than from who is on payroll, which is the same list the reviews in identity and access work from.
Simulations became a game
What the numbers said: falling click rates, celebrated.
What was happening: people learned the pattern of the simulations rather than the behavior, and some of them stopped opening legitimate messages from the finance team.
The fix: announce that simulations happen, measure reports rather than clicks, and stop publishing anything resembling a scoreboard. If click rate is the only number available, it is the wrong program.
The safe way is harder than the unsafe way
What the numbers said: knowledge scores fine, behavior unchanged.
What was happening: the policy asked for something more difficult than the alternative. A password rule with no password manager. A reporting route with three steps. An approval process people route around because it takes two days.
The fix: treat it as a friction problem rather than a knowledge one and change the configuration. Most training failures in small businesses are this, wearing a different name, and it is the argument the sequence in security foundations makes for configuration before education.
Nobody owns it
What the numbers said: the program existed last year.
What was happening: the person who set it up moved on or got busy, and nothing decayed visibly because completion is recorded once.
The fix: a named owner with an hour a month, and one calendar entry a quarter to check whether reports are still arriving.
The content is three years old
What the numbers said: unchanged, which is exactly the problem.
What was happening: the same material recycled, so people recognize it and disengage on sight.
The fix: replace one item a quarter with something current, and use the free public material rather than writing from scratch. The Cybersecurity and Infrastructure Security Agency's Secure Our World resources are written for general audiences and are a reasonable source for the everyone-gets-this half of a program.
The record cannot be produced when asked
What the numbers said: everything was fine, in a system nobody exported from.
What was happening: an insurer or a customer asked for evidence and the answer took a fortnight to assemble.
The fix: export the record once, now, and look at what comes out. What your business is actually obliged to hold depends on your industry, contracts and jurisdiction and belongs with your own advisers, but being able to produce what you have is universal.
The pattern behind all of them
| Symptom | The real cause | The check |
|---|---|---|
| High completion, no reports | No route, or a route with friction | Reports per month |
| Reports declining | Somebody was blamed | Ask people, do not read a dashboard |
| Clicked through fast | The session is too long | Session length in minutes |
| Good scores, no change | Generic material | Can anyone repeat a story from it |
| Gaps in coverage | The list came from payroll | Build it from access |
| Falling click rate only | Measuring the wrong thing | Are reports rising too |
| Behavior unchanged | The safe way is harder | Time both ways and compare |
| Quiet decay | No owner | Whose name is on it |
Six of those eight checks take under five minutes. The United States National Institute of Standards and Technology's guide to security awareness programs treats measurement as part of the program rather than as an afterthought, which is the habit that prevents most of this page.
Common questions
We get no reports at all. Is that good?
No. It means either the route is unusable or people are not willing to use it, and both are worth more attention than any module. Send one announced simulation and see whether anybody says anything.
Reports are mostly false alarms. Should we discourage them?
Never. Two minutes to dismiss a false alarm is the cheapest thing in security, and the first discouraged report is the last honest one. The measurement that matters is set out in security compliance training.
How do we restart a program that has clearly died?
Stop the annual module, appoint an owner, fix the reporting route, and run one ten-minute session about something that actually happened to you. That sequence takes a fortnight and rebuilds more than a new platform would.
Is any of this different for a business of five people?
The mechanics are smaller and the tone matters more, because in a business of five one badly handled report ends reporting entirely. The rest is the same list with less chasing.
