An AI implementation partner builds your production AI system in your own repositories and stays on it until it runs, and it is the right choice when the system has to reach production before you can hire for it. Build in-house instead when the system is part of what you sell and someone on staff can review AI work, and use an AI development agency when the work can be finished and scored against a test you wrote.
Key facts
Definition
An in-house AI team, an AI development agency and an AI implementation partner differ most in who can still change the system a year after launch.
How Bigcircle does it
- Build in-house when the system is part of what you sell and someone on staff can review AI work.
- Use an agency when the work can be finished and scored against a written test.
- Bring in a partner such as Bigcircle when the system has to reach production before you can hire, with its code and eval set in your repositories.
Comparison
A day rate does not say who changes the system after the model is updated, the documents change, or users ask what nobody tested.
Key number
MIT NANDA's July 2025 report found about 67% of externally built pilots reached deployment against about 33% of internal builds, self-reported, from interviews with 52 organizations.
Two questions decide most of it: will the system keep changing every month as part of what you sell, and can someone on your staff review AI work today? Build in-house when both answers are yes. If you start with a partner, keep its code and eval set in your repositories so your own team can take it on.
| Criterion | In-house team | Agency | Implementation partner |
|---|---|---|---|
| What you are buying | People on your payroll who will own AI for years | A deliverable built to a specification | A working system taken into production, with the evals that keep it there |
| Fits when | The system is part of the product, and someone on staff can review AI work | The work can be finished, and you can write the acceptance test | The system has to run in production before you can hire, or your first hire needs a working system to learn on |
| Fails when | Nobody on staff can tell a strong AI hire from a convincing one | The system keeps changing after acceptance and nobody inside learned it | You expect the partner to decide what the product should do |
| Who can change it a year later | Your team | Whoever the contract names, often nobody who built it | Your team, if the code and the eval set live in your repositories |
| What to check first | Who will review the first hire's work | Who maintains the system after acceptance | Whether the team that scopes the work builds it, and whether any of it is subcontracted |
| How Bigcircle handles this | Bigcircle does not place engineers. If AI is core and you can review the work, building in-house is the better long-term answer | A fixed-scope build is our version of building to a written test, and the evals ship in your repository with the code | The team that scopes the system builds it in your repositories. The founder hired and trained each engineer, they stay on one product, and nothing is subcontracted |
Most of the table follows from one row: who can change the system a year from now. This guide explains why that row matters more for AI than for ordinary software. It then reads the most quoted build-versus-buy number and the timelines that come with it, and ends with how to choose an implementation partner when an agency uses the same words.
Why does the year after launch decide this?
A production AI system keeps changing after it ships, even when nobody touches its code. The base model is updated or retired, the documents it reads change, and users ask questions nobody tested. An acceptance test proves the system passed on the day it shipped, and someone still has to check it against real questions every month after that. Decide who does that checking before you decide who builds.
Our trademark drafting platform has been in production since 2023, and it shows what that upkeep looks like. Production logging runs the base model and the fine-tuned model on the same inputs, so a regression from a model change shows up as a score that moved. Whoever owns the system after launch has to read that comparison and act on it, and an acceptance clause written at handover does not price that work.
So the build-or-partner choice is two choices made at different times: who builds the first version, and who can still read the evals, change the code and defend the result a year later. A price comparison made before kickoff only covers the first.
What does MIT's build vs buy AI figure measure?
Read the most quoted build vs buy number as a warning that first internal builds often stall, and not as evidence that any particular partner will get your system into production. In The GenAI Divide: State of AI in Business 2025, published by MIT NANDA in July 2025, pilots built through external partnerships "reached deployment ~67% of the time, compared to ~33% for internally built tools", and its authors call partnerships twice as likely to succeed. Check three things in its methods before you lean on that.
The build and buy percentages rest on interviews with 52 organizations, so the 67 and the 33 are shares of a small sample. Success means deployment beyond the pilot with measurable KPIs, and the outcomes are the interviewees' own. The authors add that the gap "may reflect organizational capabilities rather than implementation approach alone", and that is where we start.
The buy column is also not what most readers assume. It means procuring external tools and co-developing with vendors, which is closer to buying software than to hiring a team to build a custom system. The arrangement many teams end up with, an in-house team working alongside an outside builder, had too little data in the study to score.
Deployment is also a harder bar than a finished build. Each of our case studies gives the system's status next to its numbers, built and tested or running in production, so ask any partner for the same split between built and running.
How long does AI implementation take?
Plan for somewhere between three and nine months or more, and decide first what counts as done, because that moves the number more than anything else. In MIT NANDA's sample the top-performing mid-market companies took an average of 90 days between pilot and full implementation, and enterprises, firms with over $100 million in annual revenue, took nine months or longer.
Those timelines come from the interviewees themselves, in the same sample as the 67 percent comparison, and they start at the pilot, so the time to build the pilot is not in them. Why AI pilots fail covers what stalls a pilot before that clock starts.
Our cloud agent for Skionis reached a working product in about four months, by our internal records. That is time to a working product, and a finished build and a deployed one are different milestones. When a firm quotes a timeline, ask which of the two it means, and whether the eval set that shows the system works is inside it.
When is building in-house the right answer?
Often, and for a core capability it is usually the right long-term answer. If the AI system is how your product works, the backlog will keep producing new failures for years. The people fixing them should sit in your planning meetings and learn your domain in a way an outside team rarely does. The first of them can also hire the second when the team grows.
The condition is review: someone on your staff has to read an eval result and tell a retrieval failure from a prompt failure. They also have to say whether a change made the system better or only made the demo smoother. Without that person, you cannot judge the hire.
Our guide to LLM evaluation covers that review in detail. A failing score has to point at a cause, or the manager is approving anecdotes and a weak hire can drift for months before anyone sees it.
The founding argument behind Bigcircle explains why this hire is hard to judge. Our founder hired and trained every engineer individually, for depth in one or two areas and working breadth across the rest.
His reason is that AI engineering asks one person to reason across retrieval, evaluation, latency, cost and failure modes at once. A candidate can be strong in two of those and untested on the rest, and only a reviewer who has run such a system will notice.
Timing is the other limit, because sometimes the system has to be in production before you can find and review that person. The long-term answer can still be in-house while someone else builds the first version. In that case, write the handover into the first contract rather than hoping for one at the end.
When is an AI development agency the right answer?
An AI development agency fits work that is allowed to be finished: a migration, a one-off extraction pipeline, or a proof with a kill date. Agencies are good at delivery against a specification. The risk sits in the specification itself, because for an AI system done has to be a test you can rerun rather than a demo that answered the easy questions.
Write that test before you sign. For a retrieval system, done means the right passage comes back and every citation resolves, and our guide to RAG in production lists the failures to test for.
For an agent, done means a check that the action happened, made independently of what the model claims. On our executive-operations agent we compare the claimed completion against the actual state of the application, because an agent can claim success on work that did not land.
For anything a customer will see, the test also covers what the system must refuse. If you cannot write these tests, you are not ready to buy a fixed deliverable, and a week spent on the test is worth more than a week of vendor selection.
The failure mode is the year after: an agency team learns your edge cases and your data quirks, and then the statement of work ends. If the system will keep changing, ask before signing who will maintain it after acceptance and where its evals will run. A repository without its eval set gives your team code it cannot safely change.
How do you choose an AI implementation partner?
Not by the label, since agency and partner are both used loosely and a firm can call itself either. The difference shows up in three places: how the work is staffed, what counts as done, and what stays with you. Each of those can be checked in a proposal or a contract.
| Question to ask | Why it matters | Bigcircle's answer |
|---|---|---|
| Does the team that scopes the work also build it? | A scope written by one group and built by another loses the reasons behind each decision | Yes. The team that scopes a system builds it and stays on it until it runs in production |
| Are the engineers on your system only? | A person split across three or four accounts is measured in hours, and nobody owns the whole system | Yes. Engineers stay on one product and are not rotated between accounts |
| Is any of the work subcontracted? | You cannot vet people you never meet, and the contract may not bind them | No. Every engineer was hired and trained by the founder |
| Is acceptance an eval threshold or a demo? | A demo passes on the inputs someone chose. An eval also counts the failures | Evals are part of every fixed-scope build, alongside discovery, build and deployment |
| Where do the code and the eval set live? | If the evals stay in the vendor's tooling, your team inherits code it cannot safely change | In your repositories, under your IP and confidentiality terms, in every engagement |
| How were their published numbers measured? | A figure without its method cannot be checked | Every figure in our case studies carries its measurement note, and we withheld the trademark build's internal figures because they were never independently measured |
The row about splitting people across accounts comes from why Bigcircle exists. The founder's argument is that service companies spread engineers across three or four accounts and measure them in hours. Nobody then owns a whole system, and nobody has room to learn it. Bigcircle is built the other way round, and the question is worth asking of any firm you consider.
When does a partner fit, and when does it not?
A partner fits when the system has to work in production before you can build a team for it. It fits best where a wrong answer costs money, a customer or a regulator's attention. It also fits when your first AI hire will learn more by reviewing a running system than by starting from an empty repository.
On our cloud agent for Skionis, adoption came from what an engineer could review rather than from the model. Engineers went from refusing to let an agent near production to letting it own routine cloud changes. What changed their minds was seeing each change as an architecture diagram they could approve before anything was applied. Template errors before deploy fell from around 10% to 4% as the validation loop went in, in our internal measurement.
A partner does not fit when you need someone to decide what the product should be. On a studio-led programme, one of the engagements in our guide to AI pilot to production, we make the hard technical calls on a roadmap, but product judgement stays with you. A partner also does not fit when AI is core and you can already hire and review the people, because then the knowledge belongs inside from the start.
If you plan to start with a partner and move the system in-house, set three conditions in the first contract:
- The code lives in your repositories from the first commit.
- The eval set runs in your own CI, so a later change by your team is measured the same way ours were.
- The person you hire reviews pull requests while the partner is still there, so the handover happens through real changes and not a final meeting.
The flowchart puts the guide in one picture. On every branch, settle two things before you sign: who will change the system a year after launch, and where its evals will run when they do.
Common questions
Should we build an AI team in-house or use a partner?#
Build in-house when the AI system is part of what you sell and someone on your staff can already review AI work. Use a partner when the system has to reach production before you can hire for it. A partner also helps when your first hire needs a working system to learn on. The two often run in sequence: a partner builds the first system in your repositories while you hire the person who will own it.
When is an agency the better buy?#
When the work can be finished and you can write the acceptance test before kickoff. A migration, a one-off extraction pipeline and a proof with a kill date all qualify. An agency is the wrong buy when the system will keep changing after acceptance and nobody on your side learned how it works. The people who knew its edge cases leave when the statement of work ends.
What is the difference between an agency and an implementation partner?#
The labels do not tell you. Check whether the team that scopes the work also builds it, and whether the engineers work only on your system. Ask whether any of it is subcontracted, whether acceptance is an eval threshold or a demo, and where the code and the eval set will live. A firm that answers those well is acting as a partner, whatever it calls itself.
Does the MIT report prove that partners succeed more often?#
No. MIT NANDA's July 2025 report found that about 67 percent of externally built pilots reached deployment, against about 33 percent of internal builds. That comes from an interview sample of 52 organizations. The report says the outcomes were self-reported, and that the gap may reflect the companies rather than the approach. It also lacked data on in-house teams working with an outside builder.
How long does AI implementation take?#
It depends on whether done means built or deployed. In MIT NANDA's 2025 report, top-performing mid-market companies reported about 90 days between pilot and full implementation, and enterprises took nine months or longer, both self-reported and not counting the pilot itself. Our Skionis cloud agent reached a working product in about four months. Ask any firm which milestone its timeline means.
Who are the best AI implementation partners?#
A list of names will not answer that, because the same label covers firms that work very differently. Judge any shortlist, Bigcircle included, with the same questions. Does the team that scopes the work build it, and are its engineers on your system only? Is any of it subcontracted, is acceptance an eval threshold, and where will the code and evals live? Ask too which of its published builds are running in production.
Who owns the code and the IP when a partner builds the system?#
With Bigcircle, you do, in every engagement. The work happens in your repositories under your IP and confidentiality terms. With any other firm, read the contract for who owns the repository and for whether the eval set and the test data are delivered with it. Code without its evals is hard for your team to change safely.
Can a partner build the first system and our team take it over?#
Yes, if you set it up that way in the first contract. Keep the code in your repositories from the first commit, and run the eval set in your own CI. Have the person you hire review pull requests while the partner is still working. A handover that happens only at the end leaves your team with a system they watched being demonstrated but never changed.
Further reading
- AI pilot to production. Once you have decided to work with a partner, which of our three engagements fits.
- LLM evaluation. The review an in-house lead needs to run, and the acceptance test an agency scope depends on.
- Why AI pilots fail. Build, buy or partner, one level up from who builds it.
- Production AI engineering. The layer between a working demo and a system in production.
- Case studies: trademark drafting, executive-operations agent, Skionis cloud agent.
- MIT NANDA, The GenAI Divide: State of AI in Business 2025, July 2025, as reported by Fortune. Read section 6.1 and the limitations in section 8.2 of the report before quoting the build-versus-buy figures.