The Go Bag You Cannot Pack Mid-Incident: Critical Rebuild Materials

The materials you need first can be stranded behind the services you lost

Most recovery registers list critical rebuild materials. Fewer answer the question that matters during an incident: where does each one live, and what must already be working before the team can reach it? A firewall configuration may sit behind the failed firewall. Rebuild automation may sit in a cloud repository the team can no longer access. An architecture diagram may live in a wiki that won't start. Foundational here means Tier 0, which includes critical IT infrastructure and core IT services such as identity, trust, name resolution, time and control planes.

We're assuming you already have immutable or isolated copies to recover from. The harder part is everything needed to turn those copies back into working services. If the team can't reach a configuration, key or image it needs, the outage lasts longer and every decision gets harder.

How we chose these fifteen

This list spans backups, credentials, security and forensic tools, documentation, build tools and software. NIST and CISA tell organizations to identify, protect, validate and test recovery resources, but they don't prescribe these fifteen materials or rank which ones fail first.

One issue keeps showing up in WWT recovery assessments and exercises: the material exists, but the team can't reach it, trust it or use it once normal services are gone. Three failure modes shaped this list:

1. Most commonly missing: everyone assumed someone had it, and nobody checked until the incident.
2. Most complex to reconstitute: recoverable on paper, but in practice it's slow and waits on a specialist or a vendor.
3. Caught in a dependency loop: the material is intact, but you can't get to it or use it until another service is back. Figure 2 shows how that stalls a recovery.

Why the labels matter

The fifteen materials don't all serve the same purpose during recovery. MEANS are the sources and tools the team rebuilds from. KEYS are the credentials and trust material needed to access systems, use recovery sources or re-establish trust. PROOFS are the evidence and independent tools used to determine whether the recovered environment is trustworthy enough to move forward.

• [MEANS] - Rebuild sources and tools
• [KEYS] - Access and trust
• [PROOFS] - Evidence and validation

The labels make different recovery dependencies visible. An organization can have current backups, images and automation and still stall if the team can't access them, decrypt them, restore trust or validate the recovered environment. Categorizing each material by its role helps expose an inventory that is strong in one area but weak in another. In WWT's experience, organizations tend to focus most heavily on the MEANS because backups, images and automation are easy to see and fund. KEYS and PROOFS work quietly during normal operations, so it's easy to assume they'll be there. In a destructive incident, that assumption can stop the recovery.

Backups: available does not always mean reachable

1. Network and infrastructure device configuration backups

Keep current copies of switch, router, firewall, load balancer, storage and compute configurations somewhere outside the infrastructure they describe. The classic loop is a firewall configuration sitting on a file share that can only be reached through the firewall that just failed. If that's the only copy, engineers end up rebuilding VLANs, routes and rule bases from memory while every downstream recovery wave waits.

Test: Take the affected infrastructure and network paths offline and try to pull the configurations.

2. Foundational identity, name-resolution and network-service recovery state

For foundational services, recoverable state includes Active Directory and LDAP, cloud identity where supported, DNS and DHCP. That may mean system state, zone files, scopes, identity policies, privileged roles and trust settings. A trusted identity recovery point is less common than a data backup. Conventional Active Directory forest recovery still assumes a trusted recovery point. If there isn't one, the organization may have to reconstruct or re-establish identity rather than simply restore the forest.

Test: Assume the worst your threat model allows. Can you still bring identity, DNS and DHCP back to a state you trust, on premises and in the cloud?

3. Backup system configuration, backup catalog and recovery boot media

Backup data isn't much use if the team can't bring up the platform that reads it. Keep the backup platform's configuration, backup catalog and recovery media outside the failure domain. Rebuilding a backup catalog by rescanning media can be slow. Some platforms also protect their own configuration only inside the platform, so the backup system ends up waiting on itself while every recovery wave behind it waits too.

Test: Can the backup platform be rebuilt without relying on its own management plane or backup catalog?

Documentation has to survive the environment it describes

4. Tier 0 and Tier 1 rebuild procedures and incident recovery runbooks

These are step-by-step procedures for reconstructing identity, name resolution, time, privileged access and the first business services. They need to be written for the engineer on shift, not just the architect. If someone who did not write the runbook cannot follow it at 2 a.m., it isn't ready. Runbooks also drift. Once the environment is several upgrades past the procedure, engineers are forced to improvise where mistakes can propagate upward.

Test: Can current personnel execute foundational recovery without the architects who designed the environment?

5. Architecture and dependency diagrams

The team needs current estate and dependency diagrams, including cloud organizations and accounts, landing zones, identity relationships, DNS control planes, SaaS administrative dependencies and external connectivity. Those diagrams often live in a wiki hosted on the estate that was just lost, so the team ends up rediscovering its own architecture by interviewing whoever is awake.

Test: Can the team understand sequencing and external dependencies without the production documentation platforms?

6. Vendor, contract, license entitlement and support escalation records

A severity-one support call is a bad time to discover that nobody can prove the entitlement. The team needs to know who to call, which contract applies and what proof is required. If those records, or the vendor portal itself, sit behind SSO to the identity provider you're rebuilding, you can be locked out of support you already pay for.

Test: Can the team open a severity-one case and prove entitlement without production SSO, email or the usual vendor-portal path?

Software and build tools make recovery repeatable

7. Configuration baselines and gold images

Gold images and hardened templates give the team a known-good place to start: OS and VM templates, OVA/OVF files and container base images. But 'gold' doesn't mean current. Building an image during an incident costs time, but an old image can be just as dangerous if it reintroduces the vulnerability the team is trying to remove.

Test: Do the known-good build images still match current platforms, patch levels and hardening expectations?

8. Configuration management, infrastructure as code, recovery orchestration and offline package repositories

Keep the code and automation that make recovery repeatable: configuration-management content, infrastructure-as-code modules, landing-zone code, recovery plans and mirrored package sources. One that's easy to miss is on-premises rebuild material stored in a cloud repository. The files may be intact, but if federation, managed endpoints, DNS, proxy, VPN or tenant controls block the path, nobody can pull them, and the team goes back to building by hand.

Test: Can the team retrieve automation, recovery-orchestration definitions, code and package sources without depending exclusively on normal cloud access or production connectivity?

9. Vendor installation media, drivers, firmware, license keys and recovery vault software

Operating system media, drivers, firmware, license files, cloud marketplace entitlements and recovery vault software are easy to treat as things you can download later. During an incident, later may mean no portal access, no entitlement or no internet egress. Firmware is especially easy to overlook until a bare-metal rebuild stops on a version mismatch and the one file the team needs isn't there.

Test: Can the team obtain the required software, firmware, drivers and entitlements under incident constraints?

Credentials are the keys, and keys fail quietly

10. Break-glass administrative access and recovery credentials

Emergency access is broader than one domain administrator account. Keep emergency accounts, DSRM credentials and elevation procedures for directory, cloud, hypervisor, storage, network, out-of-band interfaces, public DNS and the domain registrar. Cloud emergency access should work without normal federation. An account can exist and still be useless because the password rotated, the authenticator is unavailable or conditional access blocks the path.

Test: With normal identity, federation, MFA and network paths all down, can the recovery team still get the privileges it needs without weakening authentication to do it?

11. Independent secrets-platform access and multifactor authentication recovery methods

Recovery access has to work when production identity, federation, MFA and networks are unavailable or untrusted. A robust on-premises recovery vault uses recovery-specific credentials, such as isolated AD or independent local accounts, plus a known-good administrative workstation. Cloud recovery needs the same principle: an emergency path that doesn't depend on the service being recovered. Problems surface when that path was never exercised or still reaches outside the recovery environment.

Test: Do the on-premises and cloud emergency-access paths work under realistic isolation, from the recovery workstation rather than a production laptop that's already signed in?

12. Cryptographic keys, certificates and tokens

This includes CA private keys and databases, Kerberos and federation secrets, disk-encryption recovery keys, and cryptographic material used by MFA, PAM, TLS and cloud KMS/HSM platforms. A CA can be restored when its database, private key and trusted configuration are recoverable; otherwise, trust may have to be re-established. Some HSM/KMS keys are intentionally non-exportable, so recovery may depend on replicated state, activation material, protected backup blobs or platform-native recovery.

Test: Can cryptographic trust be restored or re-established, including when key material is intentionally non-exportable, without introducing unknown or compromised trust material?

Security and forensic tools provide the proof

7.  Offline endpoint security installers, malware-analysis tools and detection content

Do not assume the endpoint security agent will simply be there after a rebuild. Keep installers, onboarding material, malware-analysis utilities and current detection content somewhere production systems can't block access to them. The sensor may normally come from a management platform that is down or untrusted, and some agents still depend on cloud services or enrollment tokens. Account for those dependencies before a rebuilt host is cleared to return to service.

Test: Can rebuilt systems be instrumented and produce enough evidence to meet the organization's defined recovery trust criteria?

8. Portable disk, memory, network and log analysis tooling

The forensic kit needs to work without production. Imaging, memory capture, packet capture and log-analysis tools should be available on a dedicated, isolated workstation built from a known-good image. Too often the kit everyone counts on is licensed to one person, sits on a laptop nobody can vouch for or depends on a service that's down. The assessment then waits on procurement or an external retainer.

Test: Can the compromise boundary be assessed with independent tools and a known-good analysis environment?

9. Integrity validation baselines and evidence-handling materials

Hashing utilities, hardened baselines, known-good manifests, chain-of-custody forms and organizational indicators give the team something objective to compare against. Without a baseline, "clean" is a feeling. Recovery needs evidence that a rebuilt system meets the organization's trust criteria, and legal, regulatory or insurance requirements may set a higher bar.

Test: Can the organization produce defensible evidence that rebuilt systems meet its trust criteria?

How critical rebuild materials fit into the recovery plan

The recovery register should identify the materials teams need at each stage of recovery. Those materials also inform damage assessment and the decision to restore, remediate or rebuild. For each material, record who owns it, when it is needed, where it is stored and how authorized personnel can retrieve it during an incident.

A vault and an isolated recovery environment may be integrated or separate. Either way, authorized personnel need a known-good retrieval path that doesn't depend on production identity, on premises or in the cloud.

Keep the recovery register current as configurations, access controls and dependencies change. During an exercise, test the retrieval paths under the identity, network and cross-environment access restrictions the team could face after containment.

Treat these fifteen as a starting point. Your architecture, threat scenarios, recovery sequence and regulatory obligations will add to the list and set the bar for proving readiness.

Use the matrix to spot likely gaps in your own environment.

Three questions to test your recovery readiness

1. Do we know where each critical rebuild material is, and what has to be working before the recovery team can reach it?
2. Could the team still retrieve those materials if normal identity, network access, managed endpoints and collaboration tools were unavailable?
3. Have we actually demonstrated that the team can retrieve, trust and use them in the required recovery sequence?

Anything short of a confident yes is a gap. Give it an owner, fix it, then rerun the retrieval test to prove the fix worked.

For help assessing these gaps, ask WWT about Our Tier 0/Critical Business Recovery Dependency Mapping services.