How to Choose a Data Science Consulting Partner: 12 Due-Diligence Questions
Choosing a data science consulting partner is difficult when you are not technical yourself, because slides, demos and credentials say little about whether a project will work in your business. This guide gives you 12 due-diligence questions to put to any data science consultancy, a printable scoring brief, sample acceptance criteria and a red-flag checklist.
What matters most when choosing a data science consulting partner
- A convincing model demo shows that a provider can build a model. It does not show that the provider can get a working result into your operations and keep it working.
- The strongest evidence is a project that reached production, a named team that did the work, and a client willing to discuss what went wrong.
- Ownership of code, trained models and derived data, plus a tested handover, should be settled in the contract before work starts, not at the end.
- Some situations call for no consultancy at all, and a good provider will say so. The no-fit criteria below cover both sides.
The things you can see in a sales process are only loosely related to the things that decide the outcome. Those are scoping discipline, data realities, handover quality and who does the work. This guide is written for executives who must judge technical suppliers without being technical themselves.
Start with the outcome, not the technology
Before you speak to any supplier, write down the business change you want in one sentence that contains no technical terms. “Reduce stock-outs on our fastest-moving products” is a goal. “Build a machine learning model” is a method. Providers who are handed a method will happily quote for it. Providers who are handed an outcome have to think about whether the method is right.
Four things should be written down before the first call:
- The decision or process that will change. Who will do something differently on a Tuesday morning because of this work?
- The current way of doing it. This becomes the baseline any new solution must beat.
- The data you believe exists, and where it lives. Be honest about its state. Providers can work with messy data, but they cannot price it fairly if you hide it.
- The internal owner. One named person on your side who can answer questions, approve scope changes and accept the result. Projects with no internal owner stall regardless of the supplier.
These four items turn the first two questions below into a two-way conversation rather than a pitch.
The 12 due-diligence questions to ask a data science consultancy

The questions follow the order in which a project can go wrong: goals, delivery capability, ownership, governance, references, scope and exit. Each is followed by what a good answer sounds like. The printable brief later in the article condenses them into a single table.
Goals and fit
1. “What would you need to know before you could quote, and what would make you decline?” A reliable provider asks about the decision being supported, the data, the users and how success will be judged. A provider who quotes immediately from a one-paragraph brief is pricing a guess. Willingness to decline or to recommend a smaller first step is a good sign.
2. “How do you tell whether our data is good enough, and what happens if it isn’t?” Good answers describe a short data assessment before the main build, with a clear outcome: proceed, fix data first, or stop. Weak answers promise results regardless of data quality.
Delivery expertise
3. “Show me a project that is running in production today, and tell me what has changed since launch.” Models degrade as the world changes, so a supplier with real production experience will have stories about monitoring, retraining and data pipeline breakages. Getting a model into daily use usually also means building the software around it, so ask whether the provider can deliver custom software development for business workflows as well as the model itself. If every example ends at the demo or the pilot, you are buying a prototype.
4. “Who exactly will work on our project, in which roles, and for how much of their time?” Ask for roles and seniority, not just a team photo. Sales-stage experts who disappear after signing are a common complaint in consulting generally. A good answer names the lead, the roles involved and the expected time commitment, and says how replacements are handled.
5. “How will you prove the result is better than what we do today?” Look for a baseline comparison, testing on data the model has not seen, and an agreed measure of success before building starts. Be wary of accuracy figures quoted without a comparison. A model that is “95% accurate” may be no better than always predicting the most common outcome.
Ownership and handover
6. “Who will own the code, the trained models and the data products made from our data?” The usual buyer-friendly position is that you own what you pay for, while the supplier keeps its pre-existing tools and general know-how. The boundary between the two must be written down. Ownership terms depend on the contract and applicable law, so have your own legal adviser review them. Do not rely on a verbal assurance.
7. “What exactly do we receive at the end, and how will you show that our team can run it?” A strong handover includes source code in a repository you control, documented setup steps, a data dictionary, a list of assumptions and known limitations, and at least one working session where your staff make a change themselves. The sample acceptance criteria below show how to turn this into something testable.
Governance and data handling
8. “How will you handle our data: who can access it, where it is stored, which subcontractors see it, and is it ever used to train anything beyond our project?” You want specific answers: named systems, access controls, data location, and a written statement on reuse. Vague reassurance about “enterprise-grade security” is not an answer.
9. “How do you manage model risk: bias, explainability, monitoring and who is accountable when it goes wrong?” The question matters most where models influence decisions about people or money. The NIST AI Risk Management Framework is a useful vocabulary for the conversation. NIST published it on 26 January 2023 as a framework for voluntary use, intended to help organizations build trustworthiness into the design, development, use and evaluation of AI products and systems. It is commonly described as organizing AI risk work into four functions: govern, map, measure and manage. NIST has also published a separate generative AI profile (July 2024), which is worth raising if the supplier plans to use generative AI, and it notes that version 1.0 of the framework is being revised, so check which version a supplier is working from. You do not need the supplier to certify against it. What you are testing is whether they have a repeatable process for these concerns. The question is less relevant if the project is conventional reporting with no predictive component, and a good supplier will say so.
Evidence from other buyers
10. “Can I speak to two clients with similar projects, including one where something did not go to plan?” Ask to speak to the client directly rather than reading a testimonial. Useful reference questions include: Was the result delivered in the form promised? How were problems handled? Did the supplier’s team change during the project? Could your own staff run the result afterwards? Would you hire them again for something harder? A supplier who has only perfect references is either very new, very selective, or not being straight.
Scope and exit
11. “What is in scope, what is out, and how are changes handled?” Prefer a first phase with a defined deliverable and an explicit decision point before committing to the full programme. Check how unexpected data problems are treated. Are they a change request, a shared risk or the supplier’s problem?
12. “If we stop after the first phase, or part-way through, what do we keep, and what does leaving cost?” Check termination terms, notice periods, who holds the environments and credentials, and whether any fees apply for taking the work in-house. A supplier confident in the value they provide will make it easy to leave.
Printable brief for comparing data science vendors
Print this table and use one copy per supplier. Score each answer 0 (evasive), 1 (generic) or 2 (specific and evidenced), and write the evidence beside it. Compare suppliers on the evidence, not just the total. If you want a wider side-by-side view of your shortlist, our compare data science vendors checklist covers it in more detail.
| # | Ask | A specific answer includes | Warning sign | Score (0–2) and notes |
|---|---|---|---|---|
| 1 | What do you need before quoting, and what would make you decline? | Questions about decisions, users, data and success measures; willingness to suggest a smaller start | Fixed quote from a one-page brief | |
| 2 | How do you judge whether our data is good enough? | A short assessment with a proceed, fix-first or stop outcome | Guaranteed results regardless of data | |
| 3 | Which project is in production, and what changed after launch? | Monitoring, retraining or pipeline stories | All examples end at demo or pilot | |
| 4 | Who works on our project, in which roles, and how much time? | Named lead, roles, time commitment, replacement process | “Our experts” with no names or hours | |
| 5 | How will you prove it beats our current approach? | Baseline, unseen test data, success measure agreed up front | Accuracy figures with no comparison | |
| 6 | Who owns code, models and derived data? | Written split between your assets and their reusable tools | “We’ll sort that out later” | |
| 7 | What do we receive, and how will you show we can run it? | Repository, documentation, working session with your staff | Handover is a final slide deck | |
| 8 | How is our data accessed, stored and reused? | Named systems, location, subcontractors, no-reuse statement | Generic security language only | |
| 9 | How do you manage bias, explainability and monitoring? | A repeatable process and clear accountability | “The model is unbiased” | |
| 10 | Can we speak to two similar clients, one where it went badly? | Direct contact details and honest account of a difficulty | Only written testimonials; no difficulties ever | |
| 11 | What is in and out of scope, and how are changes handled? | Phased scope, decision points, change process | Open-ended scope or large single phase | |
| 12 | What do we keep if we stop early? | Clear exit terms, transferable assets, no lock-in fees | Penalties or lock-in for leaving |
Sample acceptance criteria: define “done” before work starts

The most valuable protection in any data science contract is a definition of “done” that you can test without technical help. The example below is an illustrative example. It assumes a hypothetical distributor commissioning a weekly demand forecast for its product range, to be used by the purchasing team. Adapt the wording and numbers to your own project, and agree on the figures with the supplier before signing.
| Criterion | Sample wording | How you check it |
|---|---|---|
| Beats the baseline | The forecast has lower error than the current method (for example, last year’s sales for the same week) on a recent period the model was not trained on. The required improvement is set in writing once the baseline has been measured. | Your analyst reruns the comparison on held-back data using the agreed error measure. |
| Runs without hand-holding | The forecast is produced automatically each week and delivered to the purchasing team’s existing tool by an agreed time. | Observe three consecutive weekly runs with no manual intervention from the supplier. |
| Fails visibly | If input data is missing or forecast quality drifts beyond an agreed range, an alert goes to a named person on your side. | The supplier demonstrates the alert by deliberately removing a data feed in a test environment. |
| Reproducible | Your staff can rebuild and redeploy the solution from the repository using the written instructions. | A named engineer on your side does this, with the supplier observing but not typing. |
| Documented | Delivered with a data dictionary, assumptions, known limitations and a plain-language description of how outputs should and should not be used. | A non-technical colleague reads the document and explains back when not to trust the forecast. |
| Owned | All project code, trained models and project-specific data products transfer to you on payment, as set out in the contract. | Your legal adviser confirms the contract language; your team confirms access to the repository and environment. |
Notice that none of these criteria says “the model is accurate.” Each describes something a buyer can observe. That is the standard to hold every proposal to. If a supplier resists measurable acceptance criteria, ask why.
What to put in a data science request for proposal (RFP)
A short, clear request for proposal improves proposals and makes them comparable. Ask every respondent to cover the same items: their understanding of your outcome; a phased plan with deliverables and decision points; the named team and time commitment; how success will be measured against your baseline; data handling and security arrangements; ownership and handover terms; two relevant references; pricing by phase with what is and is not included; and what they would need from you. Provide your goal, the current process and a realistic description of your data, so proposals respond to your situation and not to a generic brief.
Red-flag checklist for data science suppliers
None of these is proof of a bad supplier on its own. Several together should slow you down.
- Every example project ends at a demo or pilot, with none in production.
- The people on the sales call cannot be told apart from the people who will do the work, and no one will name the delivery team.
- A firm price and timeline arrive before anyone has looked at your data.
- Success is described in terms of the model (“high accuracy”) rather than your business outcome.
- Ownership, data reuse or exit terms are “covered in the standard contract” and not shared in advance.
- References are available only as written quotes, or are anonymous.
- Nobody on the supplier’s side ever tells you that something is a bad idea or that a simpler tool might do.
- The proposal is large and indivisible, with no earlier point at which you can stop.
When a custom data science consultancy is not the right choice
Honest no-fit criteria save time for both sides. Consider another route if any of these apply:
- An off-the-shelf tool already does the job. If your need is standard reporting, a configured business intelligence or analytics product is usually cheaper and quicker than a custom build.
- You have no usable data and no way to collect it. A consultancy cannot create signal that is not there. Fixing data collection first may be the real project.
- Nobody on your side can own the result. Without an internal owner, even a well-built solution tends to be unused or to decay.
- You need extra hands rather than outcomes. If you already know what to build and want people to build it under your direction, you are looking for staff augmentation, which is a different contract.
- The decision is not repeated or measurable. A one-off question can often be answered with analyst time rather than an automated system.
The same logic applies to suppliers. A provider whose experience is mainly in a different sector, scale or technology to yours is not necessarily a bad provider, but it is not a fit for you, and the best ones will say so on the first call.
Next step: shortlist and compare suppliers
Write your one-sentence outcome and the four preparation items from the start of this article. Then send the same questions to two or three suppliers, print the brief for each call, and score them on evidence. If you would like to put these questions to DataColab as one potential supplier, describe your project and your data, and we will tell you plainly whether we think we are the right fit. You can also see what we build in our custom AI and ML software services.