9 CRO Agency Red Flags You Can Spot Before You Sign
Nobody on their side decided to disappear. It happened a little at a time, and almost all of it was visible before you signed.
So let's do this backwards. Start at the silence and walk back to the first sales call.
- Why most agency pitches look the same
- Red flag 1: no cases in your niche
- Red flag 2: the quote came before they looked at your data
- Red flag 3: nobody can say what exists on day 14
- Red flag 4: the approach changes after you sign
- Red flag 5: the main deliverable is a document
- Red flag 6: every test is a big swing and nothing gets fixed
- Red flag 7: the roadmap is a list, not an order
- Red flag 8: the tests ignore your ICP, your offer and your voice
- Red flag 9: you can't see this week's work
- When the silence starts on your side
- FAQ
Why Most Agency Pitches Look the Same
A site review, a roadmap, three case studies and a monthly fee. That's most pitches, and it's fair to say you can't tell them apart from the deck alone.
Clients and agencies also remember breakups differently. In the 2025 Marketing Relationship Survey by Setup and the CMO Institute, clients named dissatisfaction with delivery and dissatisfaction with value, 61% each, as their top reasons for ending a relationship. Agencies pointed to client budget cuts, at 75%. And 44% of clients said the agency didn't understand their business, a view shared by 3% of agencies. The survey is small, about 100 responses, 57% of them from the client side, but the gap is hard to miss.
The silence usually shows up long before the breakup. It's in who gets named, what gets written into the contract and what's promised for the first two weeks. The buyers who get ghosted least read the contract before the case studies.
This list is about accountability. For method, our 7 questions to ask a CRO agency covers win rates and significance, and our breakdown of what one A/B test costs covers what a testing quote should include.
Red Flag 1: No Cases in Your Niche
Every deck has case studies. The question is whether any of them sell at your price, to your buyer, with your buying pattern. Supplements lose people at the subscription decision. Furniture loses them at delivery times. Apparel loses them at fit and returns. Electronics lose them at comparison. These are not the same store with a different logo, and an agency that has never worked one of them will spend your first quarter finding out where yours leaks.
Ask: "Which of your case studies is closest to my store, and why that one?"
A good answer: a named store in your category, or one with the same buying pattern, plus what they tested there, what failed, and which of those lessons they expect to carry over to you. "We work across all verticals" is not an answer, it's the absence of one. And if they honestly have no case in your niche, a good agency says so, shows the closest adjacent one and explains what transfers and what doesn't. That answer is worth more than a vague yes.
Watch the shape of the case too. Three screenshots and a percentage is a claim. A case is the starting number, the hypothesis, what was tested, how long it ran, and what happened to the losers.
Red Flag 2: The Quote Came Before They Looked at Your Data
This is where the story starts. One call, a proposal two days later, a monthly fee and a list of "quick wins" that could fit almost any store. It's also the quickest way to become one of the agencies that, in the Setup survey, 44% of clients said didn't understand their business.
Ask: "What did you look at before you wrote this?"
A good answer: your sessions and conversion rate by device, your top pages, your margins or at least your average order value, and any tests you've run before. If they haven't asked for any of it, they haven't seen it.
Red Flag 3: Nobody Can Say What Exists on Day 14
Before the silence, there was a kickoff call and a lot of access requests. After that, the word you heard most was "discovery".
Ask: "What will exist on day 14 that doesn't exist today?"
A good answer: specific things with dates. Access set up and checked, analytics compared against Shopify orders, a first list of hypotheses, and the first test being designed or built.
Every week of onboarding without a test is a week without an answer. Three quiet months cost you a quarter of testing, and then another few weeks to onboard whoever comes next.
Red Flag 4: The Approach Changes After You Sign
This one is easy to miss because nothing looks broken. You signed for a testing program. What arrived was a research sprint, then a redesign proposal, then a list of recommendations for your own developer to build. Each step had a reason. Nobody ever said "we've changed the plan", and by month three you're paying a testing retainer for consulting.
The other version is quieter. The strategist who ran the sales call, asked sharp questions and had opinions about your product page is replaced by a coordinator who forwards messages. The approach didn't change on paper. It changed in who is doing it.
Ask: "Who will be on our weekly call, can I speak with them before I sign, and what happens if the plan changes in month two?"
A good answer: names and roles, a short call with the strategist who'll run your account, and a plain rule for changes: the pitch is the plan, it's written into the agreement, and any change comes to you with a reason before it happens, not after. Twenty minutes with the person who will actually do the work tells you more than the case studies do.
Red Flag 5: The Main Deliverable Is a Document
Promised the world, delivered a PDF, went quiet. It's the most common version of this story, and it usually starts with a proposal where the document is the product.
Research has value. It just shouldn't be the finish line. It's only worth what it turns into, which is tests, fixes and changes to your store.
Ask: "When does the first test go live, and what will it change?"
A good answer: a date and a named first test, with the research feeding it. How research should turn into a testing roadmap is covered in our questions to ask a CRO agency.
Red Flag 6: Every Test Is a Big Swing and Nothing Gets Fixed
Start with the odds. In Analytics-Toolkit's analysis of 1,001 A/B tests, 33.5% produced a significant winner. A program made only of big swings therefore produces two quiet months out of three, and the biggest swings, the ones aimed at lifetime value through subscriptions, bundles or post-purchase offers, are also the slowest to read because the outcome arrives months later.
Meanwhile the obvious things sit there. In Baymard Institute's 2025 benchmark, 64% of desktop checkout flows rate mediocre or worse, none rate perfect, and 62% of sites don't make guest checkout the most prominent option. Those are not hypotheses. They're known defects with known answers, and you don't run a four-week test to find out whether a broken mobile filter should work.
The split is simple. Test the things where you genuinely don't know the answer. Fix the things where you do.
Ask: "What will you ship in the first month that isn't a test?"
A good answer: a short list of fixes found in the first look, shipped without a test because there's nothing to learn from one, running alongside the first real test. An agency that says "everything goes through a test" is either padding the hours or has never had to ship under a deadline. Our checkout optimization guide lists the defects worth fixing on sight.
Red Flag 7: The Roadmap Is a List, Not an Order
Thirty hypotheses in a spreadsheet look like a plan. They become a plan when each one has a score, the order is written down, and somebody can defend why number one is number one. Without that, the roadmap drifts toward low-effort tests on low-traffic pages, because those are the ones that can be shipped quickly when the month is running out.
Ask: "How did you decide what to test first, and what did you decide not to test?"
A good answer: a visible scoring method, impact, confidence and effort at minimum, traffic per page in the input, and a reason for the order that survives a follow-up question. The second half of that question matters more than the first. An agency that can't name what it chose to leave out hasn't prioritized anything, it has just written things down. We use ICE, PIE and RICE this way, and the scoring is visible to the client.
Red Flag 8: The Tests Ignore Your ICP, Your Offer and Your Voice
Move the reviews above the fold. Add a sticky add-to-cart. Try trust badges under the button. These are real tests and they're worth running, but they're page mechanics, and page mechanics have a ceiling. The lifts that change a quarter usually touch three things a layout test can't: the offer, which is what's bundled, what shipping costs and what the guarantee actually promises; the buyer, which is who the page speaks to in its first screen; and the voice, which is whether the page sounds like your brand or like a template.
An agency that has never asked who your customer is can only rearrange what's already on the page. That's the 44% from the survey, in practice.
Ask: "What did you read about our customers before writing this roadmap, and which test on it changes the offer rather than the layout?"
A good answer: they've been through your reviews, your support tickets and any post-purchase survey you have, they can name your buyer's main hesitation in one sentence, and at least one test on the list changes what's offered or how it's said, not only where it sits. If they ask for that material before quoting, better still.
Red Flag 9: You Can't See This Week's Work
A monthly report is written after the fact, by the people whose month it describes. By the time it lands, the month it covers is spent.
Ask: "Where will I see what you did this week, without asking?"
A good answer: a shared board or log that shows tasks and hours as the work happens, with larger pieces of work scoped and approved before they start. A monthly report on top of that is fine. A monthly report instead of it isn't.
Nine questions, nine answers. In three months, someone will ask why you picked this agency. If you've written these down, you'll have your answer.
Our answers, for the record. Cases: we'll name the closest store we've worked and say plainly what transfers to yours. Before the quote: we ask for analytics access first, and if we can't see your numbers, we don't price the work. Day 14: the first test is live by day 10. Changes of plan: they come to you before they happen, and you meet the strategist on your account before you sign. Deliverable: a test on your store, never a deck of recommendations for your team to build. Fixes: defects get shipped, not tested. Roadmap: scored, ordered and visible to you. Offer and voice: we read your reviews and support tickets before we write a hypothesis. Visibility: a shared backlog you can open any day, plus a Slack channel.
When the Silence Starts on Your Side
That's the other half of the survey. Agencies blame budget cuts and leadership changes, and sometimes they're right. More often it's smaller: a developer who can't get into the theme, a test waiting three weeks for sign-off, a founder who wants to see everything but has no time to look.
Before you sign, name one person on your side who can approve a test within two days, and have store and analytics access ready for day one. It's the cheapest way to make sure the silence, if it comes, isn't yours. If you're not sure your store is ready for an agency at all, do you need a CRO agency yet? has four checks you can run in ten minutes.
Run these nine questions on every agency you talk to, Weblics included. If we fail any of them, don't sign with us. If you want to put them to us directly, send us your store.
There's one more thing to bring to the call. An agency that has looked at your store should be able to name its first test on the spot. If you want to know what a good answer sounds like, our product page optimization checklist is the list we start from on a new store, and it says what has to be true before each pattern is worth testing.
And if you're not sure an agency is the right hire at all, CRO agency vs freelancer vs in-house compares the three at your revenue.
Ghosting rarely starts after the contract. It starts with what the contract leaves out.
Sources
• Analytics-Toolkit (Georgi Georgiev), "What Can Be Learned From 1,001 A/B Tests?", 2022 (33.5% of tests produced a statistically significant winner)
• Baymard Institute, "The Current State of Checkout UX", 2025 benchmark (64% of desktop checkout flows rate mediocre or worse, none rate perfect, 62% of sites don't make guest checkout the most prominent option)
• Weblics engagement terms as published on weblics.agency, October 2026
Worth a Read
Do you have any questions left?
Here are the answers for you
No case study in your niche, a quote written before anyone looked at your data, no specifics for the first two weeks, an approach that quietly changes after you sign, a proposal built around a document instead of tests, a roadmap of big swings with no fixes, a list of hypotheses with no scoring or order, tests that only move sections around, and no way to see this week's work.
Ask which case study is closest to your store and why, what they looked at before quoting, what will exist on day 14, who will run your account and whether you can speak to them first, when the first test goes live, what they'll ship that isn't a test, how they decided the order of the roadmap, what they read about your customers, and where you'll see this week's work.
It helps more than most buyers expect, because categories fail in different places: subscriptions, delivery times, fit and returns, or comparison. An agency without a case in your niche isn't disqualified, but it should be able to name the closest one it has worked, say what transfers to you and what doesn't, and show the full case rather than a screenshot and a percentage.
Access set up and checked, analytics compared against Shopify orders, a first list of hypotheses, obvious defects queued as fixes rather than tests, and the first test in design or build. If the answer is "discovery" with no dates, ask for specifics before you sign.
Sometimes the work was never scoped clearly, the person who sold the project isn't the one doing it, or nothing in the contract sets a cadence. Sometimes the plan quietly turns into something else that was never priced. And sometimes the agency is blocked by missing access or slow approvals on the client side. All of these are easier to prevent before signing than to fix after.
Read the case studies and the contract before the slides. Whether anyone has sold what you sell, what happens in the first two weeks, how the roadmap is ordered and where you can see the work separate agencies that look identical in a deck. For the method side, win rate and how they call significance, see our seven questions to ask a CRO agency.
Every plan includes complete care-driven CRO - what varies is testing capacity and analysis depth.
All Plans Include:
Onboarding (First 5 days):
- Founder interviews & business deep-dive
- Comprehensive technical website audit
- Customer psychology analysis (ICP, 5 WHYs, SWOT)
- AI-trained buyer personas creation
- Ad creatives audit
- Marketing ecosystem review
Ongoing (Continuous):
- Psychology-first hypothesis generation
- Conversion-focused UX/UI design
- Strategic copywriting
- Shopify development & implementation
- A/B testing & QA
- Transparent reporting & documentation
- Strategy meetings (weekly or bi-weekly)
What Changes by Tier:
- Tests per month: 2, 4, 6, or 8 A/B tests
- Meeting frequency: Bi-weekly (Starter) or Weekly (Growth+)
- Analysis depth: Post-purchase surveys, support analysis, inventory strategy, KPI planning, quarterly planning (varies by tier)
Bonus (Growth+): Comprehensive email marketing audit from specialist partners
Flexible plans give you complete control over costs. You pay for the essential CRO work - strategy, hypothesis generation, analysis, A/B test and project management - whilst design, development, and QA are billed separately at $70/hourly only when you need them.
This is perfect if you have an in-house design or development team, or if you want to manage exactly what gets built and when. You're not locked into paying for services you don't need.
Scale plans include everything - strategy, analysis, design, development, QA, and implementation - in one predictable monthly retainer. No surprises, no separate invoices, just complete care-driven CRO delivered autonomously.
Choose Flexible if: You have internal resources or want precise cost control
Choose Scale if: You want fully autonomous, hands-off CRO with everything included
Transparent pricing based on your monthly traffic.
We charge based on traffic volume because testing capacity and statistical significance directly correlate with session count. The more traffic you have, the faster we can run tests and deliver results.
Pricing:
- Starter (50K-75K sessions): $1,650/mo - 2 tests
- Growth (75K-150K sessions): $3,500/mo - 4 tests
- Scale (150K-350K sessions): $6,600/mo - 6 tests
- Enterprise (350K+ sessions): $10,700/mo - 8 tests
No long-term contracts. Cancel anytime.
Every plan includes our 30-day profitability guarantee.
Not sure which plan fits?
Book a discovery call - I'll help you find the perfect match for your business.
Our battle-tested frameworks and systems validate every hypothesis before we build.
Phase 1: Onboarding (First 5 days)
- Deep-dive into your business, customers, and psychology
- Comprehensive technical audit
- 25+ care-driven optimisation hypotheses
- Custom roadmap delivered
Phase 2: Operational (Continuous)
- Validate hypotheses through AI-trained buyer personas
- Ask: "Does this genuinely serve customer needs - not manipulate?"
- Design, develop, and implement winning tests
- Rigorous QA across all devices
- Launch and monitor
Phase 3: Ongoing Analysis (Monthly)
- Behavioural segmentation & data analysis
- Post-purchase survey analysis (Growth+ plans)
- Support ticket insights analysis (Growth+ plans)
- Inventory strategy (Growth+ plans)
- Monthly KPI planning (Growth+ plans)
- Quarterly strategic planning (Scale+ plans)
Yes - but as an addition to our battle-tested frameworks, not the foundation.
We've built a proprietary AI system that validates every hypothesis against your actual buyer personas before we build anything. This ensures we only create optimisations your customers will genuinely respond to.
How it works:
- Our frameworks identify conversion opportunities
- We generate psychology-first hypotheses
- AI-trained buyer personas validate each hypothesis
- We ask: "Does this genuinely serve customer needs—not manipulate?"
- Only validated hypotheses get built and tested
This approach achieves 84% test success rate vs 45% industry average - because we validate with your actual customers before building, not after.
AI enhances our care-driven methodology. It doesn't replace genuine customer understanding.
Simply upgrade to the next tier for more included tests and enhanced ongoing analysis.
We're completely flexible - scale up or down based on your business needs. No penalties, no long-term lock-ins.
Want to discuss expanding your plan? Your dedicated CRO manager can adjust your package anytime.
Yes. No long-term contracts. Cancel anytime.
We earn your business every single month through results - not by trapping you in contracts.
If we don't make you profitable within 30 days, you pay nothing more until we deliver. That's our guarantee.
Most clients stay because care-driven CRO compounds month after month - each winning test keeps generating revenue whilst new tests add even more. But you're never locked in.
We're confident our results will speak for themselves.
Zero micromanagement required. We operate completely autonomously.
We're an extension of your business - making decisions with your profit margins AND mission in mind, not billable hours.
Your involvement:
- Initial onboarding: 2-3 hours (interviews, strategy alignment)
- Weekly/bi-weekly meetings: 30-60 minutes (strategy updates, results review)
- Ad-hoc questions: Slack chat for quick questions
We handle everything else:
- Hypothesis generation
- Design and copywriting
- Development and implementation
- QA across all devices
- A/B test management
- Data analysis and reporting
You focus on running your business. We focus on adding $50K+ monthly to your revenue.
That's the partnership.
We integrate with your existing tools—no forced changes.
Analytics: Shopify Analytics, Microsoft Clarity, GA4
Testing: Intelligems
Management: ClickUp, Figma, Slack
Your data stays in your systems. We integrate seamlessly.
We sign NDAs before any work begins. Your data is protected - always.
Security measures:
- Non-Disclosure Agreement (NDA) signed upfront
- Limited access permissions (only what's necessary)
- Data stored in your systems (we don't migrate your data)
- Team access restricted to assigned personnel only
- Regular security audits
We treat your business like our own - that includes protecting your data like it's our own.
You maintain full control over all access permissions and can revoke them anytime.
Guaranteed profitability in 30 days. $50K+ monthly revenue boost within 60 days.
Tangible outcomes:
But more than numbers - you'll understand your customers deeply, remove friction authentically, and build genuine relationships that compound revenue month after month.
- Increased conversion rates (50-100%+ improvements common)
- Higher average order values
- Improved ROAS (return on ad spend)
- Enhanced customer lifetime value
- Sustainable, compounding revenue growth
Our 84% hypothesis success rate means tests consistently work.
Real client results:
- ForKeeps Merch: $2.3M added revenue (+70% conversion rate)
- Organic Muscle: 128% conversion rate increase
- CKitchen: $1.1M added revenue over 22 months
- Mayven Studios: 50% conversion increase in 2 months
For as long as care-driven CRO continues delivering massive ROI - which typically compounds over 6+ months.
Why long-term partnerships work:
- Each winning test keeps generating revenue permanently
- New tests stack on top of previous wins
- Deeper customer understanding leads to better hypotheses
- Compounding effects multiply over time
Typical timeline:
- Months 1-3: Foundation + initial wins ($50K+ monthly added)
- Months 4-6: Compounding effects visible (wins multiply)
- Months 7-12: Sustainable growth system established
- 12+ months: Category-leading conversion rates achieved
Most clients stay 12-24+ months because results compound. But there's no lock-in - cancel anytime.
We earn your business every month through genuine results, not contracts.
Three simple steps:
Step 1: Book a Discovery Call 30-minute conversation to discuss your traffic, goals, and biggest challenges. We'll explore if we're a good fit and map out your path to $50K+ monthly revenue growth.
Step 2: Get Your Free Audit We'll conduct a comprehensive CRO audit of your website, deliver 25+ psychology-first hypotheses, and show you exactly where your biggest revenue opportunities are.
Step 3: Choose Your Plan & Launch Select the plan that fits your traffic and business needs. We'll onboard you within 5 days and have your first A/B test live within 10 days.
Ready to grow with care-driven CRO?
Or have more questions? Email us: garyk@weblics.agency
Spending more on
Getting less back?
Your ads aren't broken. The store they land on is.
Send us your URL and we'll show you which leaks are eating your ad spend, and which one to fix first.
Just honest findings on your store, within 48h.