ERP Evaluation Criteria for an Operationally Complex Company: The Six That Decide It, and the Twenty That Do Not.
Updated: 3 days ago
An ERP evaluation, as it is usually run, is a scorecard. Someone assembles a list of criteria, forty or fifty rows long, weights each one, watches three demonstrations, scores each vendor on each row, and adds the columns. The vendor with the biggest number wins. Eighteen months later, the company is working around the system, and nobody can find the row on the scorecard that would have predicted it.
That is because the rows that predict the outcome are not the rows that fill a scorecard. For an operationally complex company, meaning one with inventory, or manufacturing, or projects, or several entities, or an unusual way of doing business that is the reason it makes money, there are six criteria that decide whether the implementation works, and they are hard to score in a demonstration. This post is the six, and then a word about the twenty.
Why the scorecard fails.
A scorecard is built from features, because features are what vendors publish and what demonstrations show. Features are also the things every serious vendor has. Multi-currency, approval workflows, a mobile app, dashboards, an API, role-based security: the top eight products in the mid-market all score four or five out of five on all of them, and the sum of forty near-ties is a random number with a confident face.
The things that decide the outcome are not features. They are fits: between the product's model and your business, between its edges and your other systems, between its customisation model and your ability to sustain it. Fits are hard to score because they need your data and your processes in the room, and vendors do not demonstrate with your data. So the scorecard weights what is easy to see and the decision is made on the rest by feel.
The six that decide it.
One: does the data model fit how the business actually works.
Every ERP has an opinion about what a customer is, what an order is, what a product is, how a job is costed and how an entity relates to another. If your business matches the opinion, configuration is short and the system is used. If your business does not, every process becomes a workaround, and the workarounds are the failure. The test is not a demonstration. It is a walkthrough of your five most awkward real transactions, with your records, in the vendor's sandbox, run by your people. Any vendor who will not do this is scoring themselves out.
Two: what is the integration surface, and who owns each join.
On the day the ERP goes live, the shop, the warehouse, the bank, the payroll, the industry-specific tool and probably the CRM will still be outside it. Each needs a join, with a master named for every fact and a failure path. The criterion is whether the product's interface can carry each of those joins in the direction you need, in near real time, without an export, and whether the vendor or a partner will own each one. A product with a strong feature list and a weak interface is a product you will re-key around; we wrote about the decision that sits under this one, and it does not go away when the suite arrives.
Three: what is the customisation model, and can you sustain it.
You will customise. The question is in what language, by whom, at what rate, and what happens at each release. A product customised in a proprietary language by a small pool of expensive specialists is sustainable for a company that can afford a permanent one and not otherwise. A product customised in an open, widely-known model by a large pool is sustainable for most, and easier to overdo. Score this on your ability to sustain the customisations five years from now, not on the vendor's demonstration of how easy the first one is.
Four: which module can you not replace, and does this product have it.
Every operationally complex company has one or two functions that are the reason it needs an ERP at all: multi-entity consolidation, revenue recognition on schedules, MRP, lot and serial tracking, landed cost, project accounting. Name yours before the evaluation, and test each product against that function with your actual cases. This is the criterion that eliminates products, and it should, early. A product that scores forty near-ties and fails the one module you cannot replace has failed.
Five: who will implement it, and are they accountable for the number.
The product is a third of the outcome. The implementer is the rest. Score the implementer on whether they count before they quote, whether the estimate is guaranteed or time and materials against a scope they wrote, whether they will make the process decisions or configure to defaults, and whether they can recommend against the product. We wrote a companion piece, ERP Implementation Cost Is Decided Before You Choose a Vendor, on why the counting matters; the short version is that an implementer who does not count cannot hold a number, and the overrun will be yours.
Six: what does the exit look like.
You will leave this product one day, or at least you should be able to. Score the exit: can the data be extracted in full, in an open format, including history; what does the contract say about renewal increases; how much of what you build is portable. A product that is cheap to enter and expensive to leave has priced the exit into the renewal, and the renewal letter in year three is where you will find out.
The twenty that do not.
The features. Multi-currency, dashboards, mobile, workflow, approvals, document management, reporting, the API's existence, role security, audit trails, and the rest of the standard list. Every product you are seriously considering has them. Use them to break ties between products that passed the six, and to eliminate a product that is genuinely missing one you need. Do not weight them, because weighting them drowns the six under forty rows of near-ties, which is how the scorecard produces a confident wrong answer.
How to run the evaluation.
Before any vendor is in the room, do the readiness work: fix what has to be fixed before a single vendor is evaluated, name the module you cannot replace, write down the five awkward transactions, and inventory the joins. Then evaluate on the six, with your data, in the vendors' sandboxes, run by your people, and let the twenty break ties. The vendor with the biggest feature score will object. The vendor who passes the six will not need to.
How we approach it.
We run the six as an ERP Readiness Review, a few days of work that ends with a written recommendation and, where it recommends a product, a guaranteed estimate for the implementation. The recommendation can be to proceed, to fix something first, or to do something other than an ERP, and we have given all three. We do not own your evaluation; you do. We are accountable for the six being scored on your business rather than on a demonstration, and for the number holding when the implementation starts.
The short version.
An ERP scorecard with fifty rows produces a confident random number. Six criteria decide the outcome for an operationally complex company: data model fit, integration surface, customisation model, the module you cannot replace, the implementer's accountability, and the exit. Score those with your data and your people. Let the twenty features break ties. Never let them vote.
Know whether your business is ready for an ERP before you sign for one.
The ERP Readiness Review is a 90-minute working session plus a written scorecard across master data quality, documented exceptions, integration scope and data ownership. Fixed scope, no obligation. Or see how we approach it.
More on the same problem:
Comments