Always Have a Plan---and Don't Throw it Out the Window

In the early days of the COVID-19 pandemic, I wrote a post for the AdvantageCS blog about the continuity plans that we had in place at the time. We had developed, and continually updated, our plans and capabilities for how to function in the face of various challenges that might occur. How could we prepare for a situation where our corporate headquarters in Ann Arbor burned to the ground? If a natural disaster struck that prevented staff from getting to the building, whether for a short period or extended period, how could we ensure that we didn’t miss a beat in providing support to our clients? Looking at the possibility of a key staff member being sick or disabled, what advance preparation was necessary to keep our R&D targets and product roadmap on track?

It’s been said that no plan of battle survives first contact with the enemy. This gets at the heart of the fact that as much as we plan for contingencies, critical events that affect our operations rarely play out exactly as expected. There are an infinite number of variables that affect how events transpire in real life. No matter how often you test your disaster recovery plans, when a disaster strikes in real life, it rarely takes the shape that you expected.

This isn’t to say that having a plan is useless. In fact, having a good plan for critical events allows you to adapt to the evolving circumstances as they occur. In the absence of a good plan, and the preparation needed to execute that plan, it’s all too likely that important decisions won’t be made quickly enough and implementing those decisions will take far too long. The fact that your plan may not survive first contact with the crisis intact doesn’t mean it should be thrown out the window. It just means your plan needs to provide for the needed flexibility and rapid adjustment to the actual conditions on the ground.

In the initial days of the pandemic, when AdvantageCS moved from a 25% remote workforce to a 98% remote workforce in the span of 18 hours, we certainly faced challenges that stretched our contingency plans. Late on the morning of March 23, 2020 Michigan’s governor issued an order requiring all residents of the state to remain home for the next three weeks, except for critical needs such as food and medical care. The order took effect at midnight that day, leaving us half a day to implement our plans. We established a Microsoft Teams chat between key cross-divisional leadership to coordinate how to meet any challenges implementing the plan company-wide, while not overly constraining the team members as they executed the plan for their division. By the end of that afternoon, we felt prepared to face the next day when virtually all ACS staff would be working remotely.  

At the start of business the following day, reports came in that the poor performance of some Remote Desktop servers was causing problems with a number of employees getting work done efficiently. Other staff reported that their home internet connections were operating at a crawling pace, due to the sudden high demand on broadband access with the entire state now working and attending school from home.

By noon, we had spun up additional Azure resources to handle the inadequate Remote Desktop server performance. People were quickly back to efficiently developing, checking-in, testing, and deploying changes to the Advantage software. To handle the issues with home internet connections, affected staff members coordinated alternate working hours that allowed them to work while the public broadband infrastructure was under less strain. Within a few days, this became unnecessary as the broadband companies addressed their own issues.

While we encountered issues that affected our operations at the start of the March 2020 lockdown, our plan and preparation allowed us to quickly handle them. For example, having moved some critical infrastructure to Azure well in advance enabled us to agilely spin up resources when needed. An adaptable workforce that was highly motivated to provide our clients with excellent service despite any personal hardships they experienced was also key.

Through the pandemic, we’ve had to continually adapt. The three-week lockdown announced in March 2020 stretched on for a few months. Today, our employees are still predominantly working from home, although we have a good percentage in the office now and are preparing for a full return in the coming months. 

Despite all the challenges, the planning and preparation in the years prior to the crisis has allowed us to provide our clients with uninterrupted top-level service, which is the core of our mission as a company.

That said, we certainly hope we don’t have to execute another crisis plan anytime soon.