A few years ago, making a business case for major investment in supply-chain technology was easy. Just include a few key buzzwords such as competitive advantage, end-to-end supply-chain visibility and productivity improvement in a presentation to the boss, and the project was bound to win approval.

"Managers at all levels were looking for something easy to solve complex supply-chain problems," says John Fontana, a principal with New York City-based Tigris Consulting, which specializes in supply-chain management and technology, primarily for Fortune 500 companies. "Software salesmen presented business cases that were pure fantasy, but it was what the companies wanted to hear. They just nodded their heads. Worst of all, no one in the organization was made accountable."

According to Fontana, there was a lot of "fluff, sloppy thinking and downright bad management" when it came to evaluating supply-chain technology investments. While Fontana and many others think that this problem has yet to be adequately addressed, there is no doubt that the carnage of such poor decision-making and return-on-investment evaluation has taken its toll.

For all IT projects, about 23 percent are canceled before completion. Roughly half turn out to be a failure in terms of just meeting budget, functionality and schedule. In terms of overall success rate in meeting all three objectives, only 28 percent of projects make the grade. Less than 5 percent of companies go back and measure whether IT projects worked as planned on a financial basis.

With such a dismal history, it is not surprising that CEOs and CFOs have clamped down on big supply-chain projects and increasingly demand that all managers develop much better business cases and financial plans, even for relatively small, seven-figure projects.

"No longer are companies buying into the notion that they have to buy solutions so they don't get 'out-Amazoned'," says Tom Pisello, CEO of Alinean, an Orlando, Fla.-based provider of solutions that measure IT project value and help manage their success.

Although Pisello provided the above statistics, he sees clear signs that companies are again willing to invest in e-business and supply-chain applications - but only if such projects have solid ROI built on hard savings, within a short time frame and with little risk of disruptive failure. Such a case is not easy to make, especially with today's tight budgets that are shared by many competing departments.

The size of the proposed project is a key factor in determining its chances of approval. Large, multi-million-dollar supply-chain planning and optimization projects that dominated the industry from 1998 to about 2001 have hit hard times. According to AMR Research, sales were down 11 percent last year and at best will grow at 1 percent this year. The outlook is even worse for expensive, complex enterprise resource management projects that likely will have a 5 percent sales slide this year.

"IT investment is actually increasing, but you are seeing far fewer big, boil-the-ocean projects that promise returns well into the future," says Pisello. "Projects are now divided into more manageable pieces. That is good for the small and mid-tier supply-chain management vendors. It also makes sense for the end-user who needs to capture near-term gains to fund the next stage of IT investment."

Starting Small
For example, AMR Research says that sales for transportation management systems (TMS) and warehouse management systems (WMS) are going to grow at least 8 percent this year because they have manageable implementation times, tangible results and "not the negative war stories" that larger supply-chain applications have.

"These execution application projects are being approved because they solve very obvious cost or tactical problems," says Gerald McNerney, senior research analyst for AMR Research. "There really is no complex ROI case to be made, since they can often pay for themselves in less than a year. When companies learn how to use more of the functionality and apply them strategically, the benefits get even greater."

Even with these supply-chain execution applications, McNerney says companies take a very cautious approach and start off with a small pilot project.

"Management is very reluctant to accept assumptions about ROI or to predict total value of a project based on some formula," he says. "Just about every company today will do a pilot with one division or maybe even one product line. Based on the real benefits and real ROI produced, management will apply this payback to a wider application, perhaps across the whole organization. It makes for a more convincing case."

While companies are becoming more cautious and methodical about their IT investments, Pisello says that many technology buyers still can't get past the planning stage. For example, CIOs and business unit managers do not know where to look for value at the planning stage of a new project, so they rely heavily on vendors to provide the business case, at least at the initial phases of the analysis.

"Most of the time, the CIOs are taking the opinions of the three or four vendors who have software of this type," he says. "They are pitting the vendors against one another to see what value proposition each offers. They take bits and pieces from each presentation and figure out a business case they think is realistic. Then they look at what the differences are among them competitively to look for unique value."

Using vendor input to help plan the business case for technology purchases is controversial. The general consensus from consultants and other industry experts is that even considering the vendors' ROI projections is naïve at best.

"Of course the guy across the table wants to make the sale," says Fontana. "God help the guy who buys into the number the vendor provides, but it happens all the time. If a buyer is not capable of having an objective point of view, and managers are not willing to thoroughly vet the numbers, then that company is fair game for any slick salesman that has a convincing story."

Nucleus Research, a technology analysis firm based in Wellesley, Mass., has found that the most important difference between companies that achieved targeted ROI and those that did not was how diligently they investigated the application and planned the deployment.

"Companies that succeed simply do not accept the vendor's ROI claims," says Rebecca Wettmann, vice president of research for Nucleus Research. "They do their homework. They may get outside help to evaluate and plan the project. They make sure that the software they were buying is actually suitable for their needs. They may work with the vendors to share the responsibilities for making sure the deployment works, but they know success is their ultimate responsibility from the start."

Pisello understands the suspicion that buyers have with vendor ROI figures, but he says many vendors today use second-generation business case tools, such as those from Alinean, called "ROI calculators." These applications focus on the needs of CIOs, consultants and vendors looking for a formal analysis to make sure the ROI will deliver an attractive payback substantial enough to overcome any possible risks of failure or cost overruns. For buyers willing to pay for their own application evaluation tools, Pisello says that Alinean has an application called ValueIT designed specifically for the end user.

CFOs Get Tough
Whether or not a company uses an ROI calculator, AMR's McNerney says that CFO's are now fully in charge of the final decision on even the most modest IT project, and they demand a business case based on hard numbers.

"Financial people want a quantitative value proposition and a definite time horizon," says McNerney. "No proposal gets any consideration unless these two factors are clear right up front."

According to Fontana, whose background includes stints in capital budgeting at such companies as Frito-Lay, no major company would dream of investing in a new $100m factory or even a $10m machine tool without thoroughly analyzing its return on investment and establishing accountability. Such detailed analysis is rarely applied to software purchases.

"Companies should treat a $10m supply-chain software purchase the same way they treat a factory expansion or a new machine," he says. "The standard tools of capital budgeting work exactly the same."

At the most basic level, a capital budgeting analysis begins by enumerating the cash flow inputs and outputs for the planned investment. The most important part is realistically predicting the outputs, or returns, from a software investment. Some items, such as reduced labor, discontinuance of old software licenses or being able to shut down an entire manufacturing location, are so-called "hard" benefits because they are easily quantifiable. Everything else is a so-called "intangible" or "soft" benefit.

For example, one such intangible benefit that Fontana has seen in an ROI calculation is decreased employee turnover and lower hiring costs because workers are happier using better software.

"Any savings applied to this benefit is a stretch," says Fontana. "In any case, you have to discount this kind of intangible much more aggressively than hard benefits because there is a strong risk that the benefits will never be realized. Companies get themselves into trouble because they apply the same risk factor, discount rate and cost of capital for things that are soft and squishy as they do for hard benefits that are absolutely predictable. That is a big mistake."

Apples and Oranges
Even calculating straightforward hard benefits is often done incorrectly, according to Fontana. For example, companies have to be able to separate "expense reductions" from "reductions in capital deployed." A company that spends $5m on supply-chain software may determine that this capital investment will allow them to save $7m in inventory costs. This may appear to be a saving of $2m or 140 percent payback the first year, but it is not that simple. One is capital, the other is an expense. They can't be treated in the same way.

This distinction is important because savings occur over time, not all at once. If a software investment is supposed to result in reduced head count, that "savings" cannot be recognized all at once because head count reduction doesn't occur at the instant the software is being implemented.

How should a company evaluate optimization software that is supposed to increase output from 80 to 100 units per hour on each of its four lines? Assuming it works, the company will get 400 units of production, which is the same output it would have taken five lines to produce without the software. If a new production line costs $10m, has that company avoided $10m capital cost? Is the $10m a legitimate saving?

"Companies do this all the time, but it is fuzzy math," says Fontana, who explains that the company has really just reduced fixed cost as a percentage of total unit cost. To determine the benefit, the correct approach is to discount those savings over several years as the gross margin per unit improves.

"You need to calculate the benefit when the cash actually comes into the company as sales take place," he says. "The cash doesn't come in all at once in the form of an avoided cost, so you can't just credit yourself with a big lump of capital that never really existed."

While the CFO will look very closely at ROI for any IT investment, financial measures alone do not tell the whole story. The value of many supply-chain applications is found in process and operational improvements made during and after the implementation phase. End users rarely have set up key performance indicators (KPIs) or put in place any accountability metrics to see if KPIs have been favorably impacted. According to Pisello, only about 10 percent of projects have formal accountability systems in place.

"Companies do not think past the initial budgeting phase, and they often get themselves in trouble for this reason," says Pisello. "They need to plan and budget for accountability systems."

Since supply-chain applications impact big budget items that are already tracked, such as material acquisition costs, inventory carrying costs, accounts receivable days outstanding, accounts payable for production materials, and total inventory days of supply, financial managers often assume that the ROI for technology investment will just automatically show up on financial statements. They don't.

Lack of Communications
Such ROI measurements have to be established in the planning process when the company should have formally established net benefit goals. If no KPIs or goals are established early on, measurements are impossible, and no causal ties to any improvement can be attributed to the software. Essentially, there is no way to really determine if the software was a good investment, or how best to leverage any benefits it produces.

An underlying problem for making the ROI case for any technology project, according to Pisello, is lack of communication among the key decision-makers. Financial executives, IT managers, functional managers, consultants, and everybody involved in planning and implementing technology projects don't speak the same language in terms of what value a proposed solution is supposed to deliver. Corporate-level managers are looking for hard financial benefits on the income statement or balance sheet. IT managers and vendors want to implement the solution on time and on budget. Functional managers want the on-going business process improvement. Rarely are these goals properly aligned.

The advantage of an ROI calculator is that it analyzes projects both at the top level and at the business process level. For corporate and financial managers, it provides a risk-adjusted, discounted cash flow analysis. At a functional level, it provides details of each business process that is impacted by a new investment, both in an "as-is" and the "to-be" scenario.

"We look at all the costs, all the benefits, roll them up, and then you apply the risk of the project and discount whatever benefits and increase whatever costs the risks can affect to determine whether the project is worthy or not," says Pisello.

Financial people like "macro indicators" such as gross margin improvement because they create a common denominator for evaluating all projects. The problem with these financial indicators is that they are not very useful at the functional level. Supply-chain analysis requires looking at specific business processes.

For example, a technology investment involving fulfillment requires measurements such as how many times a task takes place, how long activities take, costs involved, etc. A company that uses this as a KPI will take the "as-is" and the "to-be" and come up with a projected cost savings. Most supply-chain technology projects will cover at least 20 such processes - from accounts payable for production materials, to finished goods delivery. Each one of these supply-chain processes is important to project success for the functional-level managers.

Corporate level managers don't care about these KPIs, but they are the building blocks that produce the results they want to see. It is a two-step process, according to Pisello.

"You first take a detailed look at specific benchmarks for the way things are and the target level you want to reach," he says. "You then add up all of those KPI improvements to show that the impact is going to be on the overall corporate financials."

"Just knowing the overall ROI is no longer enough," says Pisello. "End users want tools that help them leverage new technology to make sure they are getting the business results they expected. By knowing at both the top level and the functional level what your goals are, at both micro and macro level, you can make the necessary adjustments."

This KPI analysis at the business process level also allows companies to consider factors that are usually called "intangible benefits" because they are very difficult to quantify. Pisello says that it is very important to factor intangible benefits into the ROI calculation if they have strategic impact on the organization. For example, will the supply-chain application help re-position the company as a technical leader? Can it help improve quality and customer experience? Not all of these can be quantified in dollars and cents, but they should be measured with customer surveys or other techniques.

Hard to Quantify
Examples include fewer purchase order changes to suppliers, reduced order errors, complete order fill, damage-free delivery, and other metrics that are hard to quantify. Each of these will lead to a financial benefit, so they should be tracked with the same ROI software that is used to track other KPIs.

"No project can be justified by these intangibles, but they are supporting proof that a project is a success or a failure," says Pisello.

One of the most difficult but important parts of a thorough ROI analysis is to factor in risk of failure. Supply-chain management projects are riskier than many other IT projects because they are so complex and present so many opportunities for failure. These projects involve many functions within the organization to work in concert internally and externally with trading partners. This collaboration must happen at the planning phase as well as the delivery phase of the project. Even if the technology is implemented flawlessly, the project can fail. Supply-chain solutions do not come out of the box ready to work with any organization. There is always significant customization that has to be done before the technology and the processes work smoothly together. Risk is real and it must be measured on a probability basis in the calculation.

"Discounted cash flow calculations based on time are nice, but you also need to discount based on risk and track it independently because of the inherent issues," says Pisello.

An even more important aspect of building the business case for technology investment is clearly establishing accountability. Goals need to be set, milestones established and measurement systems put in place. Finally, someone has to be accountable for delivering the expected benefits over the life the project.

"By accountable, I mean the managers need to be willing to put their careers on the line by standing by the projected ROI numbers and delivering the return," says Fontana. "That is rare today."

This accountability has little to do with the mechanics of the ROI calculation. It has to do with the hard work of analyzing business processes, identifying the meaningful numbers, and then managing the delivery of those numbers.

"Top management does not arbitrarily make up numbers before the start of the project, and then wait until the end to calculate whether or not they were achieved," says Fontana. "The functional-level managers have to set the expectations and then deliver those benefits throughout the course of the whole project."

Who Takes the Heat?
Such accountability is the exception, not the rule, according to Fontana. Perhaps because companies lacked genuine faith in their own ROI projections, few ever set up mechanisms to determine if the returns ever materialized. With no auditing system, no one is responsible, which means no one is at fault if projects fail.

Fontana says that countless operations and IT managers sheepishly come back to their bosses six months into an implementation and explain that the software had glitches, the vendor oversold the product, or some other excuse.

"It's always 'blame them, not me'," says Fontana. "The truth is that that these managers did not rigorously evaluate the product or the expected benefit. They should be the ones to take the heat."

Another aspect of the accountability issue is that key members of the team often have little if any responsibility for delivering the benefits of the software investment. The IT manager, who often makes the final decision on which application to select, is usually done when the software goes live. The consultant who makes the application recommendation is gone by then as well. Contingent payments and success fees to vendor are usually paid shortly thereafter. Only the supply-chain operations manager is left with all the responsibility for delivering the benefits.

"The process is really just getting started when the software goes live," says Fontana. "All the milestones, all the process diagnostics, all the adjustments and all the expected benefits occur weeks and months after the initial implementation, yet the key players are long gone. The whole process needs to be managed by all people who have an accountable role."