Cisco Systems Inc. is supposed to be the company that gets everything right, and for the most part it does. For decades, Cisco has been the benchmark for countless high-tech manufacturers looking to tune up their supply chains. Occasionally, though, its best-laid plans don't yield the desired result. Take that $2.2bn in excess inventory that Cisco had to write off back in 2001, after the dotcom bubble burst. The company had relied too heavily on internal forecasting software that couldn't adjust to actual demand patterns. Cisco's application had assumed a level of growth that simply wasn't present in the real world.

Flash forward 10 years later, to a chastened Cisco - one that has spent the time implementing a comprehensive risk-management strategy designed to combat any number of disasters or supply-chain glitches. The plan got a heavy workout during Japan's earthquake and tsunami back in March of this year. And this time, Cisco passed the test.

It did so by acknowledging at the outset that trying to predict exactly what kind of catastrophe will happen next is a fool's game. No one could have anticipated the awe-inspiring parade of disasters and business interruptions we've witnessed over the past decade - the hurricanes, the epidemics, the erupting volcano, the Great Recession, the West Coast port shutdown, floods in the U.S. and Asia, political upheaval in the Middle East - the list goes on.

For Cisco, figuring out what its global supply chain looks like on a good day is a challenge. The company outsources nearly 99 percent of its manufacturing, creating an army of upstream suppliers and partners that can be difficult to identify, let alone monitor on a regular basis. What's more, it has been on an acquisition tear, scooping up some 160 companies over the last 10 years. "In any year, there are at least 10 companies we're trying to integrate," said James B. Steele, program director of value chain risk management for global business operations. (One thing Cisco ought to consider streamlining is its titles.)

To get a handle on its exposure to risk, Cisco first tried an actuarial approach. It built a simulation engine that attempted to categorize "all the risk in the world" through the gathering of extensive historical data, said Steele. (He spoke at the recent Executive Summit North America of the Supply Chain Council in Palm Springs, Calif.)

The applicable buzzphrase is "predictive analytics," which Cisco employed to create a series of "heat maps" that would show the riskiest parts of the world and the company's supply chain, said John O'Connor, senior director of Cisco's Business Transformation Organization for Customer Value Chain Management (again, those titles!). He gave a similar presentation at Eyefortransport's Hi-Tech & Electronics Supply Chain Summit in San Jose, Calif.

All well and good - except that Cisco found its sophisticated tool to be virtually useless in predicting what was going to happen next. You can't be prepared for every eventuality, but the model was of little help in identify where the company should be directing its limited resources for responding to a disaster. "The conclusion: we don't invest in predictive analytics anymore," said O'Connor.

Instead, Cisco focused on building "agnostic resilience" into its supply chain - the ability to cope with anything that comes along. The idea was to develop an index that quantified resilience in four areas: components, suppliers, manufacturing and product testing.

Steele said the risk-management effort was broken into three stages over the past seven years: reactive, where it deployed business continuity planning with a limited view of external partners; proactive, where it took a global view and worked to centralize its scattered incident-response group, and innovative, where resiliency was embedded into the end-to-end supply chain, to include such elements as product design and launch. In that last phase, which began in 2010 and is still ongoing, engineers and designers are brought to the table and every product's bill of materials is scrutinized for vulnerability. Their input is crucial, said O'Connor, because of Cisco's high number of new-product introductions - 400 a year, or better than one a day. Traditionally, "engineers are not thinking about how to make a more resilient product."

Out of that effort came a "playbook" which outlines the steps to be taken in any given disaster. It was tested to the maximum when the 9.0 earthquake and tsunami rocked Japan.

The quake occurred at around 9:45 p.m. Pacific time on March 10. Cisco knew about it within 15 minutes. Within 45 minutes it had confirmed the seriousness of the event and informed senior management. Steele mobilized a team of 10 who worked through the night to understand exactly where the company's Japanese-sourced suppliers and components were. By 7:00 the next morning, Cisco had assembled a 100-person "war room" to attend to the details and figure out which orders were affected.

What they found wasn't encouraging. Cisco had about 250 Tier-1 suppliers in Japan, many of them the sole source for highly engineered components. The response team was given an "open checkbook" by chairman and chief executive officer John Chambers to seek out new sources of product. "Sub-war rooms" were set up to locate secondary sources for specific commodities and suppliers. In the end, said Steele, the company spent around $100m on mitigation efforts in the wake of the incident.

In those crucial weeks, Cisco found itself dealing with three separate disasters: the quake and consequent tsunami, the impact on nuclear power plants in the immediate vicinity, and a widespread power shortage. All threatened to disrupt the company's ability to get product to market on time. One of the key aspects of its playbook was creation of a "customer experience team" to field hundreds of enquiries over the next few months.

Things didn't go perfectly. Cisco had underestimated the importance of constantly revising its parts site maps in line with the ever-changing landscape. "It's your biggest investment," O'Connor said, "and the most necessary." It also needed to do a better job of collaborating with manufacturing partners, a lesson that has proved valuable in coping with the more recent floods in Thailand. Risk assessment and mitigation related to sub-tier partners can be another "Achilles' heel" if not managed properly, O'Connor said.

Cisco's risk-mitigation team was in management's good graces for months after the incident. The bottom-line savings that it achieved by minimizing the impact on customers was incalculable. But here's the shocker: that warm glow doesn't last forever. An effective risk-management program isn't cheap, and Cisco's return on investment in that area "is under question right now, especially for my team," said Steele. "In Japan, we had to put a lot of money into building out second sources."

Steele doesn't expect to lose his budget any time soon, but he's fully aware of the need to keep justifying his existence to senior management. That's a fear to which any supply-chain executive can relate. It's the point when the notion of "resilience" becomes a personal matter.

Comment on This Article