Is your technology intelligence tool worth renewing? Count the decisions, not the logins

A technology intelligence tool is worth renewing if it changed decisions during the contract year, not if people logged in. Score five lines against a pass mark you set before the review: the funding calls it changed or confirmed, the shifts it flagged before your experts or the market did, how its earlier calls held up, its use by the people who place the bets, and its full cost including your team's hours. Time saved is the smallest line. The GetFocus storage backtest shows why: the improvement rate flagged flash in 2001, and the incumbent paid $19 billion to catch up in 2015.

How do you measure whether a technology intelligence tool is worth renewing?

Review it as a business capability, not a feature list. Weight five lines by why you bought the tool:

Set the pass mark before the evidence comes in, and fix the options in advance: renew as is, renew a smaller scope, renegotiate, or stop. A threshold chosen after seeing the numbers only confirms what you already wanted to do.

This applies to tools bought to inform technology bets. A patent search tool bought for prior-art work should be judged on search recall and the IP team's hours instead.

Why are logins and hours saved the wrong measure?

Because they measure activity. A report downloaded is activity; a report that stops a programme is impact. Usage still matters at the margin: a seat nobody opens is shelfware, so cut seats to the people who use them. But a tool used every day that never moves a bet is a reading habit, not a decision system.

Hours saved are real, and worth a line. Christophe Perthuisot, former SVP of R&D at Moët Hennessy (LVMH), says of the experts who tested GetFocus: "They said it saved them months of work." (Read his story.) Still, time is the small number. The GetFocus proof library prices the big one: $20B+ sunk into hydrogen powertrains while batteries out-improved them from the first data point in 2001, and $19B paid in 2015 to buy back a storage future the rate had flagged 14 years earlier.

One wrong bet avoided, or one right bet made early, outweighs years of saved research hours. Keep the lines separate in the financial case: hard savings such as studies you no longer commission, hours released, and decisions influenced. Time saved is not cash saved unless the hours went somewhere useful, and no benefit gets counted twice.

What evidence should you collect during the contract year?

A decision log, started in month one. For every material call, four fields: the question, the signal the tool gave, the action taken and the outcome so far. At renewal the log is the evidence. Without it, the review falls back on memory and login counts.

A tool built for decisions writes most of that log itself. GetFocus reports monthly deltas against the baseline, fires a trigger alert the moment a threshold you set is crossed, and walks you through which verdicts should move in a quarterly executive review. Every one of those is dated, so the renewal question turns concrete: which of them did we act on, and what happened?

How do you check whether the tool's calls held up?

A technology forecast cannot be fully graded in twelve months, but it can be audited. Take last year's verdicts and ask three questions. Did the routes it rated as rising keep rising? Did the alerts it fired turn out to matter? When a verdict moved, could the tool show which filings moved it?

The last question separates a method from an opinion. GetFocus's forecast is an explainable regression on two citation metrics, cycle time and knowledge flow, and every claim traces back to the patent families behind it. A tool whose verdicts swing without a reason in the evidence is guessing.

Ask for the error as well as the hits. Testing forecasts on 53 technologies, Farmer and Lafond found that forecast errors follow a consistent distribution, so a forecast can come with realistic error bars (Farmer and Lafond, 2016). For the long run, check the method on history: the seven GetFocus backtests replay shifts that already played out using only the patents public at each date, and the method is validated on 150+ domains at R² ≈ 0.7.

How much of our team's time does a technology intelligence tool need before it pays off?

It pays off when the time it saves and the decisions it improves outweigh the time it takes to run, check and act on. Count net hours, not hours spent in the platform. A self-service tool needs someone to write the searches, clean the results and turn them into a recommendation, and that work is usually the largest hidden cost at renewal.

A service moves that line. With GetFocus, your input is one strategic question in plain language and a sign-off on the list of competing routes. The agents read the patents, the maths computes the rates and the dashboard stays current. Your team's recurring time goes to the monthly summary, the quarterly review and the decisions themselves. As Christophe Perthuisot puts it, the system lets you "boil things down from perhaps 100 solutions to the three, four, or five you really need to examine."

Should we renew the subscription or switch to one-off assessments?

Renew where decisions are live, and downgrade where they are not. A one-off assessment fits a single decision with a deadline, or a field you have not mapped yet. A subscription fits a domain where you will keep placing bets and the field keeps moving: new entrants, shifting rates, thresholds you need to hear about. In that second case, a one-off scan is stale on arrival.

Renew by domain, not by seat count. GetFocus prices its Annual Domain Subscription per domain, so the question per domain is short: did a verdict move, did a trigger fire, is a decision coming? If all three answers are no, drop that domain and keep the ones where the bets are. A single decision can still go through the free 48-hour Decision Brief, with no subscription attached.

FAQ

What KPIs show a technology intelligence tool is worth the money?

Decisions changed, early warnings that proved right, calls that held up, use by the people who sign off technology bets, and full cost including team time. Logins and downloads support the case; they are not the case.

How do you calculate the ROI of a technology intelligence tool?

Net benefit divided by full cost. Keep hard savings, hours released and decisions influenced on separate lines, never count a benefit twice, and value a changed decision at what the alternative would have cost.

When should the renewal review start?

Six to eight weeks before the renewal date: two weeks to pull the decision log and usage, two to interview the people who place the bets, and the rest to compare alternatives and negotiate.

Is a one-off technology assessment or a monitoring subscription better?

A one-off assessment for a single decision or an unmapped field. A subscription for a domain where you keep placing bets and the field keeps moving.