An ERP evaluation, as it is usually run, is a scorecard. Someone assembles a list of criteria, forty or fifty rows long, weights each one, watches three demonstrations, scores each vendor on each row, and adds the columns. The vendor with the biggest number wins. Eighteen months later, the company is working around the system, and nobody can find the row on the scorecard that would have predicted it.
That is because the rows that predict the outcome are not the rows that fill a scorecard. For an operationally complex company, meaning one with inventory, or manufacturing, or projects, or several entities, or an unusual way of doing business that is the reason it makes money, there are six criteria that decide whether the implementation works, and they are hard to score in a demonstration. This post is the six, and then a word about the twenty.
A scorecard is built from features, because features are what vendors publish and what demonstrations show. Features are also the things every serious vendor has. Multi-currency, approval workflows, a mobile app, dashboards, an API, role-based security: the top eight products in the mid-market all score four or five out of five on all of them, and the sum of forty near-ties is a random number with a confident face.
The things that decide the outcome are not features. They are fits: between the product's model and your business, between its edges and your other systems, between its customisation model and your ability to sustain it. Fits are hard to score because they need your data and your processes in the room, and vendors do not demonstrate with your data. So the scorecard weights what is easy to see and the decision is made on the rest by feel.
Every ERP has an opinion about what a customer is, what an order is, what a product is, how a job is costed and how an entity relates to another. If your business matches the opinion, configuration is short and the system is used. If your business does not, every process becomes a workaround, and the workarounds are the failure. The test is not a demonstration. It is a walkthrough of your five most awkward real transactions, with your records, in the vendor's sandbox, run by your people. Any vendor who will not do this is scoring themselves out.
On the day the ERP goes live, the shop, the warehouse, the bank, the payroll, the industry-specific tool and probably the CRM will still be outside it. Each needs a join, with a master named for every fact and a failure path. The criterion is whether the product's interface can carry each of those joins in the direction you need, in near real time, without an export, and whether the vendor or a partner will own each one. A product with a strong feature list and a weak interface is a product you will re-key around; we wrote about the decision that sits under this one, and it does not go away when the suite arrives.
You will customise. The question is in what language, by whom, at what rate, and what happens at each release. A product customised in a proprietary language by a small pool of expensive specialists is sustainable for a company that can afford a permanent one and not otherwise. A product customised in an open, widely-known model by a large pool is sustainable for most, and easier to overdo. Score this on your ability to sustain the customisations five years from now, not on the vendor's demonstration of how easy the first one is.
Every operationally complex company has one or two functions that are the reason it needs an ERP at all: multi-entity consolidation, revenue recognition on schedules, MRP, lot and serial tracking, landed cost, project accounting. Name yours before the evaluation, and test each product against that function with your actual cases. This is the criterion that eliminates products, and it should, early. A product that scores forty near-ties and fails the one module you cannot replace has failed.
The product is a third of the outcome. The implementer is the rest. Score the implementer on whether they count before they quote, whether the estimate is guaranteed or time and materials against a scope they wrote, whether they will make the process decisions or configure to defaults, and whether they can recommend against the product. We wrote a companion piece, ERP Implementation Cost Is Decided Before You Choose a Vendor, on why the counting matters; the short version is that an implementer who does not count cannot hold a number, and the overrun will be yours.
You will leave this product one day, or at least you should be able to. Score the exit: can the data be extracted in full, in an open format, including history; what does the contract say about renewal increases; how much of what you build is portable. A product that is cheap to enter and expensive to leave has priced the exit into the renewal, and the renewal letter in year three is where you will find out.
The features. Multi-currency, dashboards, mobile, workflow, approvals, document management, reporting, the API's existence, role security, audit trails, and the rest of the standard list. Every product you are seriously considering has them. Use them to break ties between products that passed the six, and to eliminate a product that is genuinely missing one you need. Do not weight them, because weighting them drowns the six under forty rows of near-ties, which is how the scorecard produces a confident wrong answer.
Before any vendor is in the room, do the readiness work: fix what has to be fixed before a single vendor is evaluated, name the module you cannot replace, write down the five awkward transactions, and inventory the joins. Then evaluate on the six, with your data, in the vendors' sandboxes, run by your people, and let the twenty break ties. The vendor with the biggest feature score will object. The vendor who passes the six will not need to.
We run the six as an ERP Readiness Review, a few days of work that ends with a written recommendation and, where it recommends a product, a guaranteed estimate for the implementation. The recommendation can be to proceed, to fix something first, or to do something other than an ERP, and we have given all three. We do not own your evaluation; you do. We are accountable for the six being scored on your business rather than on a demonstration, and for the number holding when the implementation starts.
An ERP scorecard with fifty rows produces a confident random number. Six criteria decide the outcome for an operationally complex company: data model fit, integration surface, customisation model, the module you cannot replace, the implementer's accountability, and the exit. Score those with your data and your people. Let the twenty features break ties. Never let them vote.
The ERP Readiness Review is a 90-minute working session plus a written scorecard across master data quality, documented exceptions, integration scope and data ownership. Fixed scope, no obligation. Or see how we approach it.
More on the same problem:
CodeStringers Becomes a Zoho Consulting Company for Complex Operations (article)
ERP Total Cost of Ownership: The Five-Year View a Vendor Quote Never Shows. (article)
Three at most once the hard constraints have been applied. Each vendor gets the same scenarios with your data in its sandbox, and each walkthrough takes a half day of your operations people's time plus the scoring. Five or six vendors doubles that burden without improving the choice, because the products that survive the one module you cannot replace are usually a short list already. Use the constraint list to shortlist and the six criteria to decide.
A fit gap analysis records, for each of your real processes, whether the product handles it as standard, with configuration, with customization, or not at all. In a scorecard-driven evaluation it happens after signing, as the implementer's first deliverable, which is too late. Run on your five most awkward transactions in the vendor's sandbox during the evaluation, it is the evidence behind the first criterion and the start of the implementation plan.
Bring your records and your awkward cases rather than a data dictionary. Ask the vendor to set up your customer with two bill-to entities and five ship-to addresses, your product with a lot number and a unit conversion, your job with a change order after invoicing. Watch whether each is a field, a workaround or a refusal. The data model is what you are seeing when the presenter says the product does not quite do that.
Whether all data, including history and attachments, can be extracted in an open format at any time without a professional services engagement. What the contract says about renewal increases and about the price of adding users and modules. How much of your configuration and customization is portable. And what happens to your data when the subscription ends. A product that is cheap to enter and silent on these has priced the exit into year three.
Ask whether they count your systems, data and exceptions before quoting, whether the estimate is fixed or time and materials against a scope they wrote, whether they will make process decisions or configure to defaults, who owns data cleanup and integrations in their scope, and whether they have ever recommended against their own product. The product is a third of the outcome; the implementer's answers to those five questions predict the rest.
"Something isn't working." "We're outgrowing our tools." "We're afraid to make the wrong move." No-Risk Discovery is a short, practical conversation that gets you clarity before you commit to anything big. We'll tell you if we're a fit. If we're not, we'll tell you that too.
Say who you are and ask. A person replies within one business day.
