Competitive intelligence

Competitive intelligence ROI: skip the multiple and report three metrics you can defend

Skip the ROI multiple. Report win rate on comparable deals, card use in won versus lost deals, and response time, with a worked example and its error bars.

Gagan TomarCo-Founder, SteveUpdated

Do not report a return-on-investment multiple for competitive intelligence. Report three things you can defend in a budget review: your win rate against one named rival on comparable deals, whether that rival's battlecard was opened in the deals you won and the deals you lost, and how many hours pass between a competitor's public move and an updated card. Each has a definition, a data source and a known weakness, and the weaknesses are what leadership will ask about.

Why an ROI multiple fails the first question

CI vendors publish headline numbers without the method behind them. Klue's Measure page shows "35% increase in win rate" and "50% increase in competitive win rate" with no sample, period or baseline. Klue's own guide to CI metrics says that "proving direct causation of any two independent variables is intensely difficult", then recommends applying predicted win-rate increases to current revenue to make the case. Crayon's Measure page defines influenced revenue as revenue "won thanks to" reps' use of compete materials, which assumes the attribution nobody can prove. Kompyte's ROI post of May 31, 2024 offers "35% improvement in Win-Rates" as a sample slide headline and admits that measuring impact across teams "can be tricky".

An ROI multiple needs a counterfactual: what the same deals would have done without the programme. You do not have one. Reps who open cards differ from reps who do not; pricing, product and headcount change in the same quarter; and the competitor field in the CRM gets filled late or wrong. The CI Alliance's interview on measuring ROI (August 2, 2022) separates tangible wins you can measure, such as a shift in win rate, from intangible ones you cannot. Keep the two apart in the deck, and report the inputs leadership can check.

Metric 1: competitive win rate on comparable deals

Competitive win rate is won deals divided by won plus lost deals, counted only where a specific competitor was recorded, within one segment, one deal-size band and one region, by quarter. The comparability rule is what makes the number mean anything. A company-wide win rate mixes a rival that only appears in enterprise deals with one that only appears in self-serve, and the number moves for reasons that have nothing to do with the card.

Put an error bar on every cell before you read it. A 95% Wilson interval takes one spreadsheet formula or any online calculator, and it shows how little a small cell says: 6 wins from 20 deals is a 30% win rate with an interval from 15% to 52%. To confirm that a rate really moved from 30% to 40%, the standard two-proportion sample-size formula (5% significance, 80% power) needs about 356 closed deals in each period. Most teams never close that many against one rival in a quarter. So when two periods' intervals overlap, report the direction and the deal counts, pool quarters rather than splitting segments, and save the claim of a change for a control group.

The weakness is the CRM competitor field. If it is optional, filled at close by the rep who lost, or left blank on wins because nobody asked, the denominator is wrong before you divide. Metric 1 is only as good as that field, which is why the CRM section below comes before any chart.

Metric 2: card use in won versus lost deals

For every closed competitive deal, record whether the rival's battlecard was opened by the deal owner before the close date. Then compare the win rate of the opened group with the not-opened group. This ties adoption to an outcome, which the opens-and-views counts in the shelfware signals guide cannot do on their own.

Say in the same sentence that it shows correlation. Strong reps open more things, and a deal that is already going well gets more preparation. Two designs make the comparison more honest. A within-rep comparison looks at the same rep's opened and not-opened deals. The control group Kompyte describes arms "only a portion of the sales reps with Battle Cards during one sales cycle" and treats the reps without cards as the control. That design could support a causal claim, and it costs one sales cycle of partial rollout. Kompyte's own line is that success "is not distribution, or consumption, but usage".

Metric 3: hours from a competitor's move to your response

Measure the time between a rival's public change, such as a new pricing page or a launch post, and the moment the battlecard or Slack alert reflects it. This is the one metric the CI team controls end to end. Log the timestamp of the change as you found it, the timestamp of the published update, and who made the update.

It claims nothing about revenue, so nobody in a review can dispute it. Report the median and the worst case rather than the average, because one missed launch is the story leadership remembers.

What the three metrics give you

  • Each has a written definition, a named data source and a comparability rule, so a second person can recompute it.
  • Win rate per rival on comparable deals is the number sales leadership already understands.
  • Card use in won versus lost deals ties adoption to an outcome instead of reporting opens on their own.
  • Response time is fully under the CI team's control and cannot be argued with.

What they cannot do

  • None of them proves the programme caused a win; the control-group test is the only design that could.
  • Small deal counts per competitor and per quarter leave the win rate inside its own error bars.
  • A wrong or optional competitor field corrupts metrics 1 and 2 before you start.
  • Pricing, product and team changes in the same period cannot be separated out.

A worked example: Acme, two quarters, one new card

Take one rival, Acme, in mid-market deals. In Q1 you closed 40 competitive deals against Acme and won 12, a win rate of 30.0%. You shipped a new card at the start of Q2. In Q2 you closed 44 and won 18, a win rate of 40.9%.

Eleven points of lift looks ready for a slide. It is not. The 95% Wilson interval runs from 18% to 45% for Q1 and from 28% to 56% for Q2, so the two quarters overlap and the change sits inside quarter-to-quarter noise. The defensible line is that the rate moved in the right direction and that you need more closed deals, or a control group, before claiming the card did it.

Now metric 2 on the Q2 deals. The card was opened in 14 of the 18 wins (77.8%) and in 12 of the 26 losses (46.2%). Opened deals won more often. Say exactly that, and add that it does not show the card caused the wins, because the reps who open cards may be the reps who win anyway.

The slide that survives questioning has three rows. Win rate against Acme, mid-market, 30.0% to 40.9%, on 40 and 44 deals, intervals overlapping. Card opened in 78% of Q2 wins and 46% of Q2 losses, correlation only. Median response time to Acme's public moves, in hours, with the worst case named.

What you can and cannot claim

  • You can report the win rate against a named competitor, in one segment and period, with the deal count and interval beside it.
  • You can report that deals where the card was opened won more or less often, as a correlation.
  • You can report how fast the team responds to competitor moves, with the median and the worst case.
  • You cannot say the programme produced a share of revenue unless you ran a control group or a within-rep comparison, and you say which.
  • You cannot quote a vendor's win-rate lift as a benchmark for yours; none of the vendor pages above gives a method.
  • You cannot call a change real while the two periods' intervals overlap, however large the swing looks.

Where your CRM breaks this

Both outcome metrics depend on one field, the competitor in the deal. Check three things before you build a chart. Whether the field is required at any stage, or optional and filled only when a rep remembers. Whether it is filled at close, when the rep who lost names the rival, or earlier, while the deal was open. And whether wins carry a competitor at all: a programme that records competitors only on losses shows a competitive win rate near zero, which is an artefact of the field rather than a result. Comparing the CRM loss reason with what the buyer said is the check for the reason half of that problem, and the signals that a competitor is in a deal help fill the field before close.

If the field is unreliable, fix the field first and report metric 3 while you wait. A quarter of clean data beats a year of guessed data, and a reviewer who asks how the competitor was recorded deserves a one-sentence answer.

Where Steve fits

Steve fixes the field that metrics 1 and 2 stand on. Its Intelligent Deal Detection suggests the competitor on each deal in HubSpot or Salesforce, and a person approves the suggestion. When the deal closes, Steve Win/Loss emails the buyer an invitation to an AI-led interview that asks follow-up questions, so the buyer's reason sits beside the outcome you are counting.

The tradeoff is that cleaner inputs do not change the arithmetic. Steve makes the competitor field trustworthy; it does not give you more deals, so the error bars, the comparability rule and the control group are still yours to apply. If one or two rivals and a reliable competitor field describe your programme, the spreadsheet above is enough. If the field is the part you cannot keep filled, see the Deal Detection product page.

Frequently asked

Can I calculate the ROI of competitive intelligence?

Not defensibly. An ROI multiple needs a counterfactual you do not have, and Klue's own metrics guide calls proving causation between two variables intensely difficult. Report competitive win rate on comparable deals, card use in won versus lost deals, and response time instead, each with its deal count.

How many closed deals do I need per competitor?

About 356 per period to confirm a ten-point lift from 30% to 40%, by the standard two-proportion sample-size formula at 5% significance and 80% power. Few teams close that many against one rival in a quarter, so pool quarters, report direction only, or run a control group.

Does battlecard usage prove the programme works?

No. A higher win rate in deals where the card was opened is a correlation; the reps who open cards may win anyway. A within-rep comparison, or a control group where only some reps get cards for one sales cycle, is what could support a causal claim.

What is a control group for battlecards?

A set of reps who do not receive the battlecards during one sales cycle while the others do, as Kompyte's 2024 ROI post describes. Comparing the two groups' win rates on comparable deals is a design that could show the cards changed outcomes.

What should I show leadership?

Three rows per key competitor. Win rate on comparable deals, with the deal count and whether the change is outside its error bars. Win rate in deals where the card was opened versus not, labelled as correlation. Median and worst-case response time to competitor moves. No multiple.

Keep reading

All posts

Explore Steve Intelligent Deal Detection

Steve suggests the competitor on each HubSpot or Salesforce deal and you approve it, so the field every competitive win rate depends on holds a competitor a person has approved.