CRO Strategy

In-House CRO vs Agency: The Revenue Framework

The honest answer to "in-house CRO vs agency" isn't "it depends on your goals." It depends on your monthly revenue, your test velocity needs, and whether your dev team has any spare capacity left. Here's the framework that tells you which side of the line you're on.

12 MIN READ
UPDATED APRIL 2026
WRITTEN BY
Irakli B.

CRO Agency vs In-House Team: The Question That Costs Founders Six Figures

Most Shopify founders ask "should I hire in-house CRO or use an agency?" like it's a personality test. It isn't. The in-house CRO vs agency decision comes down to four numbers most people never run: your monthly revenue, your target test velocity, your dev capacity, and your fully-loaded cost per test. Miss the threshold and you either overpay for a single salary shipping two tests a month, or you outgrow a retainer that can't keep up with your velocity needs.
In this guide
  1. What does a CRO program actually need to run?
  2. How much does an in-house CRO team cost fully loaded?
  3. How much does a CRO agency cost and what do you get?
  4. When should you hire a CRO agency vs build in-house?
  5. The revenue threshold framework for Shopify operators
  6. Hybrid model: how to run both at the $500k-$2M stage
  7. Red flags to watch for in either model

What Does a CRO Program Actually Need to Run?

A real CRO program needs five functions running in sync: research, hypothesis generation, design, development, and analysis. Miss any one of them and your testing turns into expensive guesswork.
‍
Think of it like a restaurant kitchen. You don't just need a chef. You need a prep cook chopping vegetables, a dishwasher keeping plates clean, and someone actually taking orders from customers. Pull any of those people out and dinner service falls apart, no matter how good the chef is.
‍
The same applies here. Research digs through analytics, session recordings, and customer reviews to find what's broken. Hypothesis generation turns those findings into testable ideas with a predicted outcome. Design builds the variants. Development codes and QAs them inside your Shopify theme. Analysis reads the results and decides what to roll out, kill, or retest.
‍
When founders try to run CRO themselves, they usually handle research and hypothesis generation, then get stuck. Design goes to Fiverr. Development waits three weeks for the in-house dev team to have bandwidth. Analysis never happens because nobody wants to read the losing tests. The program dies not from bad ideas, but from broken workflow.
‍
Whether you go in-house or agency, these five functions must all be covered, on time, every cycle. That's the benchmark every cost comparison should start from.
Quick Note:
CRO is a pipeline, not a role. One person can't run a serious CRO programme alone. If you're shopping for a single "CRO manager" and expecting 3+ tests per month, you're shopping for a unicorn - and you'll pay unicorn prices.

How Much Does an In-House CRO Team Cost Fully Loaded?

A fully-loaded in-house CRO programme runs $180k-$260k per year and typically ships 2-3 tests per month. That number catches most founders off guard because they only budget for the CRO manager salary.
‍
Let's break the real maths down. A competent CRO manager in North America or Western Europe earns $90k-$140k base, plus 20-30% in benefits, payroll taxes, and equipment - call it $115k-$175k fully loaded. That person handles research and hypothesis generation full time.
‍
But they don't ship tests alone. They need roughly 40-60 hours of developer time per month to code variants inside your theme. If your developer bills $80-$120 per hour (internal loaded cost), that's $40k-$85k per year just for dev. Designer time adds another $15k-$30k annually for variant mockups and creative.
‍
Then there's tooling. A proper testing platform like Convert or VWO costs $500-$2,000 per month depending on traffic. Session recording, heatmaps, and survey tools add another $300-$800 monthly. Research databases and competitor intelligence add another line item. Call it $15k-$30k per year in stack costs.
The number that actually matters is cost per test shipped. At 30 tests per year, you're looking at $6k-$11k per test when you bake in every line item. That's the real benchmark to compare against any agency retainer.
Important update:
Most founders underbudget by 40-50%. The classic mistake is budgeting for the CRO manager and assuming "the dev team will handle it." They won't. Your dev team is building the roadmap and they'll deprioritise every CRO ticket that isn't urgent. Bake real dev time into your cost model from day one.

How Much Does a CRO Agency Cost and What Do You Get?

A full-service Shopify CRO agency retainer runs $4k-$15k per month and typically ships 3-6 tests monthly. That's $48k-$180k annually for a programme that delivers 40-70% more test volume than a single in-house hire.
‍
What's actually inside the retainer matters more than the headline price. A serious agency covers research (analytics audits, session recordings, customer surveys), hypothesis generation, design, development, QA, launch, monitoring, analysis, and monthly reporting. All five programme functions, handled by specialists who do this for 8-15 clients at once.
‍
The specialist depth is the part founders underestimate. An in-house CRO manager has seen their own store. An agency CRO team has seen what worked on 50+ stores across supplements, apparel, skincare, and electronics. That pattern library shortens the learning curve on what's likely to win before a single test runs.
‍
Here's how the cost compares when you line up test volume side by side:
Cost per test isn't the only variable, but it's the one that gets ignored most. At the retainer tier, you're typically paying 30-60% less per test shipped than the fully-loaded in-house equivalent.
‍
The trade-off is customisation. An agency splits its team across multiple clients, so you don't own their full week. For most Shopify stores under $2M per month in revenue, that's fine - you don't actually need 40 hours of CRO work per week, you need 40 hours of expert work per month.
Pro Tip:
Ask for the cost per shipped test. When evaluating agencies, don't just ask the retainer price. Ask: "How many tests did you ship last quarter across all clients, and at what retainer level?" That gives you cost per test - the only metric that makes agency vs in-house comparable.

When Should You Hire a CRO Agency vs Build In-House?

Four decision variables determine when to hire a CRO agency vs build in-house: monthly revenue, test velocity needs, internal developer capacity, and how much you value building institutional knowledge inside the business.
‍
Monthly revenue is the gravity variable. Below a certain revenue floor, the fixed cost of an in-house team swallows the gains from any test. You can't test your way to profitability if the team running the tests costs more than the tests return. Above a certain ceiling, test velocity and institutional depth outrun what a shared agency can deliver.
‍
Test velocity matters second. If you need 5+ tests per month running consistently, a single in-house manager physically can't produce that volume alone. Either you build a team (CRO manager + dedicated dev + designer), which pushes you into the $300k+ annual zone, or you use an agency that has that team already assembled.
‍
Developer capacity is the quiet killer. Most Shopify founders assume their dev team has 40 hours a month to spare for CRO variants. They don't. Your devs are shipping new features, fixing integrations, and managing the theme roadmap. CRO tickets sit in the backlog for weeks. An agency solves this by bringing their own Shopify developers.
‍
Institutional knowledge is the last one, and it only becomes decisive at scale. At $2M+ per month in revenue, the patterns specific to your customer, your catalogue, and your traffic sources become more valuable than generic CRO expertise. That's where an in-house team compounds, if you can afford to build it.

How to hire a CRO specialist if you go the in-house route

If you've done the maths and decided to build CRO team capacity internally, don't hire for "CRO manager" as a job title. Hire for the specific phase you're in.
‍
At the entry level, hire a CRO analyst who can run research, build hypotheses, and manage the testing tool. Pair them with existing dev and design resources. CRO manager salary at this level runs $75k-$95k.
‍
At the mid-tier, hire a full CRO manager with 4-6 years of Shopify experience who can own the full cycle, run stakeholder meetings, and mentor a junior. Salary runs $110k-$140k. This is the hire that fails most often, because founders expect one person to do three people's jobs.
‍
At the senior tier, you're hiring a head of CRO who builds and runs a team of 2-3. Salary starts at $150k and scales with equity. Only viable at $3M+ monthly revenue.
Reminder:
Hire for the stage you're in, not the stage you want to be in. A $400k/month store doesn't need a head of CRO. It needs an agency retainer that ships tests. Hiring ahead of your stage burns cash and creates a role nobody can succeed in.

The Revenue Threshold Framework for Shopify Operators

The revenue threshold framework gives Shopify operators three clear zones: under $500k/month, hire an agency; $500k-$2M/month, run a hybrid model; $2M+/month, build in-house with agency support for specialist projects.
‍
Think of it like transport. Up to a certain distance, a taxi makes more sense than owning a car - no parking, no insurance, no maintenance, you just pay per ride. Past a certain commute length, owning becomes cheaper. And past a certain scale, you hire a driver full time. Same economics. Different context.
‍
Zone 1: Under $500k/month revenue - agency almost always wins. At this stage, you're adding somewhere between $5k and $25k of incremental revenue per winning test (assuming 1-3% conversion lifts on your traffic volume). A mid-tier agency retainer at $4k-$7k per month pays for itself on 1-2 wins per quarter. A fully-loaded in-house programme at $200k+ annually would need to ship winning tests worth $17k+ every single month just to break even on headcount. The maths doesn't work.
‍
Zone 2: $500k-$2M/month revenue - hybrid model wins. Here the volume justifies dedicated internal attention, but not a full team. The winning structure is one in-house CRO lead (project manager / strategist) who owns the roadmap and customer knowledge, paired with an agency that runs research, design, dev, and analysis. Total cost lands around $180k-$250k annually. Test velocity runs 4-6 per month. You get institutional knowledge building internally while maintaining agency-level velocity.
‍
Zone 3: $2M+/month revenue - in-house team becomes viable. At this scale, a fully staffed in-house team (CRO manager + dedicated developer + designer + analyst) delivers 5-8 tests per month with deep institutional knowledge that compounds. Total cost runs $350k-$550k annually. Agencies still make sense for specialist projects (checkout optimisation, mobile rebuild, international expansion) where outside expertise beats internal learning curves.
The trap is the middle zone. Founders hitting $800k-$1.5M per month often hire a single CRO manager, expect agency-level velocity, and burn 9-12 months getting 1-2 tests out the door. That's where the six-figure mistake usually happens.

Hybrid Model: How to Run Both at the $500k-$2M Stage

The hybrid CRO model pairs one in-house strategist who owns the roadmap and brand knowledge with an agency that handles execution. It's the highest-leverage structure for Shopify stores doing $500k-$2M per month.
‍
The logic is division of labour. Your in-house lead knows your customer, your brand voice, your catalogue, and your internal priorities better than any external team ever will. They own the "what we should test and why" decision. The agency knows CRO best practices, owns the testing stack, brings dev and design capacity on demand, and executes the "how do we ship it" side.

What the in-house role actually looks like

Title: CRO programme manager, head of growth, or senior optimisation lead. Reports to the head of ecommerce or CMO.
‍
Responsibilities: owning the testing roadmap, approving hypotheses, running the monthly review with the agency, feeding customer research and brand context into the pipeline, coordinating with the dev team on edge cases, and reporting results to leadership.
‍
What they don't do: code tests, run the testing tool day-to-day, design variants, or write the analysis reports. Those are agency jobs.
‍
CRO manager salary for this role runs $95k-$130k. Fully loaded, you're at $125k-$170k. Add a $6k-$10k monthly agency retainer and you're at $200k-$290k total, shipping 4-6 tests per month. Cost per test lands around $3k-$6k, which is 40-50% better than a fully in-house programme.
Pro Tip:
Make the agency report into your in-house lead, not your CEO. When agencies report directly to the founder, monthly reviews turn into status updates and strategic drift sets in. When they report into a dedicated in-house owner, the agency gets sharper briefs, faster approvals, and better feedback loops. Ship velocity goes up 30-40%.

Red Flags to Watch For in Either Model

Whether you're hiring in-house or evaluating a CRO agency, the same small set of red flags predicts whether the programme will actually deliver ROI or just burn budget quietly for 9 months.
‍
Red flags when hiring in-house CRO. If a candidate can't walk you through a failed test and what they learned from it, they haven't run a real programme - they've consumed CRO blog posts. If their previous role was titled "CRO manager" but they shipped fewer than 15 tests in a year, the programme around them was broken (which means they don't know how to build one). If they start pitching ideas before asking about your customer research, they're selling templates.
‍
Red flags when hiring a CRO agency. If the agency pitches ideas in the sales meeting before seeing your data, run. If they can't show you cost per test shipped across their client base, run. If they promise double-digit conversion lifts in the first 30 days, run faster - real lifts compound slowly across many tests, not in one big win. If their case studies don't include tests that lost or tied (most CRO programmes have 40-60% losing tests), they're editing the truth.
‍
The common thread: real CRO practitioners are comfortable talking about what doesn't work. Amateurs only pitch wins.
FAQ

Do you have any questions left?

Here are the answers for you

At what revenue should I hire a CRO agency vs build in-house?

Under $500k per month in revenue, a CRO agency almost always beats in-house on ROI because fully-loaded internal costs ($180k-$260k/year) can't be justified by test volume at that stage. Between $500k-$2M/month, a hybrid model wins (in-house strategist + agency execution). Above $2M/month, a fully in-house team becomes viable and institutional knowledge starts compounding.

What's the real fully-loaded cost of in-house CRO beyond the manager salary?

Expect $180k-$260k per year fully loaded. The manager salary is only 50-60% of the real cost - you also need developer time ($40k-$85k), designer time ($15k-$30k), testing platform licenses ($6k-$24k), and research tooling ($9k-$18k) to ship 2-3 tests per month.

Can a single CRO manager run a full testing programme alone?

No. One person can't run research, design, development, analysis, and reporting at pace. At best a solo CRO manager ships 1-2 tests per month, often fewer, and burns out inside 9 months. Serious programmes need at least 3 functions covered (strategy, dev, design) plus tooling.

What if I already have a developer - do I still need an agency?

Only if your developer has 40-60 hours per month genuinely free for CRO variant work, and you already have someone running research, hypothesis generation, and analysis. In practice, most in-house Shopify devs are fully committed to feature work and the roadmap - CRO tickets slip by weeks.

Do I need to hire a senior CRO manager, or will a junior work?

At under $1M/month in revenue, a junior CRO analyst ($75k-$95k) paired with an agency is usually more effective than a senior solo hire. You get senior-level strategy from the agency and junior-level execution support internally. Hiring senior before the stage justifies it creates a role nobody can succeed in.

How do I know if my current CRO agency is actually delivering?

Look at cost per test shipped and win rate. A healthy programme ships 3-6 tests per month with a 20-30% win rate (meaning 1-2 winning tests per month) and clearly documents losing tests as learnings. If you can't see a pipeline of next quarter's hypotheses, or you're getting fewer than 2 tests shipped per month at a $5k+ retainer, something's broken. A free CRO audit will surface those gaps in under a week.

What's included in each plan?

Every plan includes complete care-driven CRO - what varies is testing capacity and analysis depth.

All Plans Include:

Onboarding (First 5 days):

  • Founder interviews & business deep-dive
  • Comprehensive technical website audit
  • Customer psychology analysis (ICP, 5 WHYs, SWOT)
  • AI-trained buyer personas creation
  • Ad creatives audit
  • Marketing ecosystem review

Ongoing (Continuous):

  • Psychology-first hypothesis generation
  • Conversion-focused UX/UI design
  • Strategic copywriting
  • Shopify development & implementation
  • A/B testing & QA
  • Transparent reporting & documentation
  • Strategy meetings (weekly or bi-weekly)

What Changes by Tier:

  • Tests per month: 2, 4, 6, or 8 A/B tests
  • Meeting frequency: Bi-weekly (Starter) or Weekly (Growth+)
  • Analysis depth: Post-purchase surveys, support analysis, inventory strategy, KPI planning, quarterly planning (varies by tier)

Bonus (Growth+): Comprehensive email marketing audit from specialist partners

What's the difference between Flexible and Scale plans?

Flexible plans give you complete control over costs. You pay for the essential CRO work - strategy, hypothesis generation, analysis, A/B test and project management - whilst design, development, and QA are billed separately at $70/hourly only when you need them.

This is perfect if you have an in-house design or development team, or if you want to manage exactly what gets built and when. You're not locked into paying for services you don't need.

Scale plans include everything - strategy, analysis, design, development, QA, and implementation - in one predictable monthly retainer. No surprises, no separate invoices, just complete care-driven CRO delivered autonomously.

Choose Flexible if: You have internal resources or want precise cost control
Choose Scale if: You want fully autonomous, hands-off CRO with everything included

How do your pricing tiers work?

Transparent pricing based on your monthly traffic.

We charge based on traffic volume because testing capacity and statistical significance directly correlate with session count. The more traffic you have, the faster we can run tests and deliver results.

Pricing:

  • Starter (50K-75K sessions): $1,650/mo - 2 tests
  • Growth (75K-150K sessions): $3,500/mo - 4 tests
  • Scale (150K-350K sessions): $6,600/mo - 6 tests
  • Enterprise (350K+ sessions): $10,700/mo - 8 tests

No long-term contracts. Cancel anytime.
Every plan includes our 30-day profitability guarantee.

Not sure which plan fits?
Book a discovery call - I'll help you find the perfect match for your business.

‍

What's your CRO process?

Our battle-tested frameworks and systems validate every hypothesis before we build.

Phase 1: Onboarding (First 5 days)

  • Deep-dive into your business, customers, and psychology
  • Comprehensive technical audit
  • 25+ care-driven optimisation hypotheses
  • Custom roadmap delivered

Phase 2: Operational (Continuous)

  • Validate hypotheses through AI-trained buyer personas
  • Ask: "Does this genuinely serve customer needs - not manipulate?"
  • Design, develop, and implement winning tests
  • Rigorous QA across all devices
  • Launch and monitor

Phase 3: Ongoing Analysis (Monthly)

  • Behavioural segmentation & data analysis
  • Post-purchase survey analysis (Growth+ plans)
  • Support ticket insights analysis (Growth+ plans)
  • Inventory strategy (Growth+ plans)
  • Monthly KPI planning (Growth+ plans)
  • Quarterly strategic planning (Scale+ plans)

‍

Do you use AI?

Yes - but as an addition to our battle-tested frameworks, not the foundation.

We've built a proprietary AI system that validates every hypothesis against your actual buyer personas before we build anything. This ensures we only create optimisations your customers will genuinely respond to.

How it works:

  1. Our frameworks identify conversion opportunities
  2. We generate psychology-first hypotheses
  3. AI-trained buyer personas validate each hypothesis
  4. We ask: "Does this genuinely serve customer needs—not manipulate?"
  5. Only validated hypotheses get built and tested

This approach achieves 84% test success rate vs 45% industry average - because we validate with your actual customers before building, not after.

AI enhances our care-driven methodology. It doesn't replace genuine customer understanding.

‍

What if I need more than my plan includes?

Simply upgrade to the next tier for more included tests and enhanced ongoing analysis.

We're completely flexible - scale up or down based on your business needs. No penalties, no long-term lock-ins.

Want to discuss expanding your plan? Your dedicated CRO manager can adjust your package anytime.

Can I cancel anytime?

Yes. No long-term contracts. Cancel anytime.

We earn your business every single month through results - not by trapping you in contracts.

If we don't make you profitable within 30 days, you pay nothing more until we deliver. That's our guarantee.

Most clients stay because care-driven CRO compounds month after month - each winning test keeps generating revenue whilst new tests add even more. But you're never locked in.

We're confident our results will speak for themselves.

‍

How involved do I need to be?

Zero micromanagement required. We operate completely autonomously.

We're an extension of your business - making decisions with your profit margins AND mission in mind, not billable hours.

Your involvement:

  • Initial onboarding: 2-3 hours (interviews, strategy alignment)
  • Weekly/bi-weekly meetings: 30-60 minutes (strategy updates, results review)
  • Ad-hoc questions: Slack chat for quick questions

We handle everything else:

  • Hypothesis generation
  • Design and copywriting
  • Development and implementation
  • QA across all devices
  • A/B test management
  • Data analysis and reporting

You focus on running your business. We focus on adding $50K+ monthly to your revenue.

That's the partnership.

What tools/platforms do you use?

We integrate with your existing tools—no forced changes.

Analytics: Shopify Analytics, Microsoft Clarity, GA4
Testing: Intelligems
Management: ClickUp, Figma, Slack

Your data stays in your systems. We integrate seamlessly.

How do you ensure my data is secure?

We sign NDAs before any work begins. Your data is protected - always.

Security measures:

  • Non-Disclosure Agreement (NDA) signed upfront
  • Limited access permissions (only what's necessary)
  • Data stored in your systems (we don't migrate your data)
  • Team access restricted to assigned personnel only
  • Regular security audits

We treat your business like our own - that includes protecting your data like it's our own.

You maintain full control over all access permissions and can revoke them anytime.

‍

What results can I expect?

Guaranteed profitability in 30 days. $50K+ monthly revenue boost within 60 days.

Tangible outcomes:

But more than numbers - you'll understand your customers deeply, remove friction authentically, and build genuine relationships that compound revenue month after month.

  • Increased conversion rates (50-100%+ improvements common)
  • Higher average order values
  • Improved ROAS (return on ad spend)
  • Enhanced customer lifetime value
  • Sustainable, compounding revenue growth

Our 84% hypothesis success rate means tests consistently work.

Real client results:

  • ForKeeps Merch: $2.3M added revenue (+70% conversion rate)
  • Organic Muscle: 128% conversion rate increase
  • CKitchen: $1.1M added revenue over 22 months
  • Mayven Studios: 50% conversion increase in 2 months
How long should I work with you?

For as long as care-driven CRO continues delivering massive ROI - which typically compounds over 6+ months.

Why long-term partnerships work:

  • Each winning test keeps generating revenue permanently
  • New tests stack on top of previous wins
  • Deeper customer understanding leads to better hypotheses
  • Compounding effects multiply over time

Typical timeline:

  • Months 1-3: Foundation + initial wins ($50K+ monthly added)
  • Months 4-6: Compounding effects visible (wins multiply)
  • Months 7-12: Sustainable growth system established
  • 12+ months: Category-leading conversion rates achieved

Most clients stay 12-24+ months because results compound. But there's no lock-in - cancel anytime.

We earn your business every month through genuine results, not contracts.

‍

How do I get started?

Three simple steps:

Step 1: Book a Discovery Call 30-minute conversation to discuss your traffic, goals, and biggest challenges. We'll explore if we're a good fit and map out your path to $50K+ monthly revenue growth.

Step 2: Get Your Free Audit We'll conduct a comprehensive CRO audit of your website, deliver 25+ psychology-first hypotheses, and show you exactly where your biggest revenue opportunities are.

Step 3: Choose Your Plan & Launch Select the plan that fits your traffic and business needs. We'll onboard you within 5 days and have your first A/B test live within 10 days.

Ready to grow with care-driven CRO?

Or have more questions? Email us: garyk@weblics.agency

Wait… Claim your free bonuses

Find out exactly where your store is leaking revenue — and 
where your ad spend is going to waste. No pitch, just data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
No spam. No agency pitch. 

Just honest findings delivered within 2 business days.