
Major incident management: who calls whom, and the escalation matrix you agree in advance
Major incident management is what happens when half the business stops and everyone rings someone at once. What fails in that moment is rarely the technology: it is that nobody knows who decides, who calls the supplier and when the board hears anything. This is how you settle that on a quiet Tuesday, so it is in place when you need it.
A major incident is an outage with so much impact that the normal process is no longer enough. You agree the criteria beforehand, so nobody argues about them during the outage.
One person leads, one ticket is the master record, and updates go out on a fixed rhythm, even when there is nothing new to say.
The escalation matrix says who calls whom and after how long, other suppliers included. Decisions about your operations and what your customers hear stay with you.
When an outage becomes a major incident
A major incident is not a dramatic name for an annoying fault. It is an outage where the normal route, a ticket in the queue and an engineer working on it, no longer works because the impact is too large or growing too fast. At that point you switch to a different way of working: one lead, one open bridge call, updates on a rhythm.
Agree the criteria in advance, as a list. Nobody wants to debate during an outage whether this is “bad enough”. Workable criteria:
- a business-critical process has stopped (tills, order entry, the patient record, the production line) and there is no usable workaround;
- a whole site or a large share of users cannot work;
- customers can see the outage, or an external commitment is at risk;
- there is doubt about whether data is complete or available.
If it is an attack, or you suspect one, different rules apply: there it matters more what you do not touch. That is covered in our article on the first hour after an incident. This article is about outages.
Who declares it, and who may stand it down
Agree who may declare a major incident, and keep that threshold low. An incident declared too early and downgraded after twenty minutes costs a few people half an hour. One declared an hour too late costs everyone a morning.
Then there are a handful of roles, and in a smaller organisation two people sometimes share them:
- The incident lead (in ITIL, the major incident manager). Coordinates, watches the clock, decides the next step. Fixes nothing personally, which is the point: whoever has their hands in the system cannot see the time.
- The technical lead. Directs the recovery work and makes sure two people are not changing the same system at once.
- Communications. Writes the updates and keeps questions away from the team.
- The decision-maker inside the organisation. Someone who may say: we close the shop, we delay the delivery, we call the customer.
Also agree who closes the incident. That is not when the engineer thinks it works, but when a user confirms it.
The escalation matrix: who calls whom, and after how long
An escalation matrix is a table you can print on one page. One axis lists the priorities, the other the time since the report, and each cell says who is informed or brought in at that point. There are two kinds of escalation and both belong in it:
- Functional: towards more knowledge. First line brings in the specialist, the specialist brings in the system’s supplier.
- Hierarchical: towards more authority. After an agreed time the IT manager hears about it, then the board, because decisions may be needed that the technical team cannot take.
What makes a matrix usable is the dull detail: a name and a deputy in every cell, phone numbers that can be found outside the affected systems, and the hours when each person is reachable. A matrix that points to an address book in the email system that has just gone down is not a matrix.
How priorities are set on impact, and which response times go with them, is in our article on SLAs you can actually meet. The escalation matrix builds on that.
The first half hour of major incident management
The first thirty minutes set the course for the rest. What needs to happen in that time is short:
- One master ticket. Every user report is linked to it, so you can see how wide the impact is without reading fifty separate tickets.
- One bridge call that stays open until the incident is closed. Whoever is needed dials in; whoever is not, leaves.
- A timeline with every observation and every action against its time. You will need it for the review, and during the incident it stops someone repeating a step.
- Workaround first, root cause second. If people can work again through another route, a backup line or a previous version, that comes first. The cause can be found calmly afterwards.
And one rule that prolongs more outages than any other: do not change two things at once. Try three fixes together and afterwards you will not know which one worked, or what to roll back if it fails again.
Updates on a fixed rhythm, even when nothing has changed
In a major incident, silence is the most expensive thing you can communicate. People who hear nothing start phoning, and every caller pulls someone off the recovery. Agree a rhythm and stick to it, even when the update is that nothing has changed yet.
A useful update answers the same four questions every time: what is not working, for whom, what we are doing now, and when you will hear from us next. Write a template for it in advance, in plain language and without system names only the IT team knows.
Keep two channels apart: the technical bridge for the people fixing things, and a separate channel for everyone who needs to be informed. Choose that second channel so it does not depend on whatever is broken. If Microsoft 365 is down, an email about the outage reaches nobody.
When the cause sits with another supplier
Many major incidents start outside your own environment: with the internet provider, a cloud platform, the till system or the supplier of a line-of-business application. The organisation that wins is the one that knows beforehand how to get through to them.
- Put every critical supplier in the escalation matrix, with their fault line, your customer or contract number and who may call on your behalf.
- Agree whether your IT partner may log faults for you. Many suppliers only accept a report from a known contact, and you would rather not find that out mid-outage.
- Note what that supplier’s contract promises. Then you know when you can escalate further and when you can only wait.
Keep leading in the meantime. Even when the cause is elsewhere, control over workarounds, updates and decisions stays with you.
Closing, reviewing and what is left behind
A major incident is closed only when users are working again and have said so. Two things follow, and they are often skipped because the pressure is off.
The review, within a few working days, while everyone still remembers. Lay the timeline next to it and ask three questions: when did we know, when did we act, and what held us up. Look for the point in the process that cost time, not for someone to blame. A matrix with a wrong number in it is a finding, not someone’s failure.
The problem that remains. The workaround stopped the outage, but the cause is often still there. It carries on as a separate problem with an owner and a date, so the same outage does not return next month.
Finally, update the escalation matrix with whatever was wrong, and rehearse it once a year with an invented outage. A matrix that has never been used always contains someone who left long ago.
Where our work stops
What we do. When a major incident hits an environment we manage, we lead the technical side: we declare it on the agreed criteria, keep the master ticket and the timeline, coordinate recovery and escalation to other suppliers, and send updates on the IT status at the agreed rhythm. With our own local engineers in every country, we come over when it cannot be done remotely, within a few hours for an outage. Afterwards we deliver the review and follow the problem until the cause is gone.
What we do not do. Decisions about your operations, such as closing a site or delaying a delivery, are yours. What your customers, the press or a regulator hear is for you to decide and send; we are not a communications agency and we do not assess reporting obligations. And if the cause lies in a third party’s platform, we escalate to them on your behalf, but we do not repair that platform ourselves: only its supplier can.
Questions we get about this
The five that come up most often.
What is major incident management?
Major incident management is the separate way of working you switch to when an outage has so much impact that the normal process is not enough, for example because a business-critical process has stopped or a whole site cannot work. It means one lead, one master ticket, an open bridge call and updates on a fixed rhythm. The criteria for declaring one are agreed in advance.
What is the difference between a major incident and a P1?
A priority 1 is a classification on impact and urgency, and it sets the response time. A major incident is a decision to change how you work, because one engineer and a ticket are no longer enough. Most major incidents are P1s, but not every P1 needs to become a major incident.
What goes into an escalation matrix?
An escalation matrix shows, per priority, who is informed or brought in after how long. It covers two kinds of escalation: functional, towards more knowledge, and hierarchical, towards more authority. Each cell holds a name, a deputy and a phone number that still works when the affected systems are down, and the other suppliers are in it with their fault lines.
What does a major incident manager do?
The major incident manager leads the incident: coordinates, watches the clock and decides the next step, but does not do the fixing. It is deliberately someone other than the engineer doing the recovery, because that person cannot see the time. With an outsourced service desk the role usually sits with the IT partner, with a decision-maker on the organisation’s own side.
Do you handle communication to our customers?
No. We send updates on the IT status at the agreed rhythm to the people you nominate. What your customers, the press or a regulator hear is for you to decide and send, and we do not assess reporting obligations. We do supply the facts and the timeline for it.
Further reading
What major incident management connects to.
Shall we hold your escalation matrix up against practice?
Send us what you have now, even if it is a list of phone numbers. We will tell you which criteria, roles and suppliers are missing, and what it looks like when we run the service desk.
Practical IT knowledge in your inbox
New guides on management, security and the workplace, written by the people doing the work. No sales talk, and you can unsubscribe in one click.