Each year, Saepio conducts several hundred incident response workshops and tabletop exercises across our client base. These draw out and highlight good practices that exist, but also highlight areas of improvement or causes of weakness. We recently sat down and went through some of the common findings and areas that caused confusion in an effort to draw up strategies and tips to improve things. Common challenges were;

• Several different channels of communication (Teams, Email, Whatsapp) with different threads with overlapping stakeholders, creating a lot of missed updates and duplication of investigation findings.
• Delay of incidents being declared due to manual monitoring of key platforms or services.
• Lack of a single source of truth of incident management policies and procedures with different policies existing in different systems and sometimes being unavailable due to downtime.
• Members of incident management teams that joined at a later stage of an incident had to spend a significant amount of time catching up with the current status.
• Communications to customers and other third parties principally rely on email and disparate systems manage SMS/Phone alerts.
• Post mortem/lessons learned washups needed to be created manually and were time-intensive.
• Lack of integration with on-call management tooling led to confusion or miscommunication of incident management responsibilities.

Quite a few of these challenges were the result of policy and procedural weaknesses, compounded by a lack of practice and rehearsal. We also found that most incident management tooling to help with structure, communication and post- mortems were owned by development teams and didn’t cover different incident types across the business. As a result, we have developed our recommendations in a few ways, but have a few core recommendations;

• Build incident management processes around realistic risks to business – what incidents are the most likely to actually happen?
• Simulate incidents and account for unknown unknowns – what if your primary communication method is offline?
• Work closely with an IR provider or cyber insurer if you have one to leverage what you’re entitled to.
• Ensure you have a resilient way to communicate updates externally to clients/suppliers/media that can be accessed without any existing IT system.
• Consider using incident management tooling to help structure and drive excellence in incident management.

The solutions architecture team is here to discuss any of these findings or recommendations. We recently conducted a market review of incident management tooling and have found tooling integrated into Teams/Slack are lowering the barrier to entry and making resilient, scalable, and easily usable tooling available to all, something which we are adopting ourselves.

Incorporating these recommendations into your incident management framework can streamline communication, reduce delays, and strengthen your response to incidents. By focusing on realistic risks, ensuring resilient communication channels, and adopting dedicated incident management tooling, businesses can address weaknesses uncovered in our workshops and tabletop exercises.

To discuss how these insights might apply specifically to your organisation, or for guidance on selecting and implementing the right incident management tools, reach out to our team. They're ready to support you in building a more resilient and responsive incident management process.