What to Ask an Agency About How It Measures

Ask an agency to show you a reading it did not like. An agency that measures an outcome has one; an agency that reports activity does not, because activity always looks like progress. That single question does more sorting than any credential check, and the rest of this page is the follow-up.

The questions that sort quickly

The fastest question is what gets measured and against what fixed list, because a vendor without a fixed list is choosing the questions after seeing the result. Ask for the list. Ask when it was frozen. Ask what happens when a question stops being relevant, and listen for whether the answer is that it gets edited or that a new one is added alongside. Why a frozen list is what makes a percentage mean anything is the reasoning underneath that question.

Then ask who chose the questions. If the vendor chose all of them, the number describes a set assembled by the party being judged. That is not disqualifying and it is worth knowing, because the correct response is to add questions of your own to the same list. The other question worth asking early is when the number should start moving, and why nobody can honestly name a date is worth knowing before you hear one.

What a reporting cadence tells you

A monthly report that arrives on a predictable date and repeats the same measurement is worth more than a richer report that arrives when the news is good. Irregular reporting is rarely a scheduling problem. It usually means the report is assembled when there is something to say, which selects for good news without anyone deciding to deceive.

Ask for the last three reports rather than a sample. A sample is built to be shown. Three consecutive real ones reveal whether the format changed, whether the metric changed, and whether anything was quietly dropped.

The attribution question, which most cannot answer

Ask how the agency knows its work caused the result, and expect a weak answer, because a strong one would be dishonest. Nobody can isolate the effect of a content programme from seasonality, competitor behaviour and engine model changes without a controlled test that almost nobody runs.

The good answer describes what is controlled and admits what is not. The bad answer is a confident causal story. We would rather say that our readings are before and after a recorded change, and that this is weaker than proof, than offer a narrative that sounds better and holds less.

Answers that should end the conversation

Some answers are disqualifying rather than merely weak. A guarantee of a specific position in an AI answer is one, because the systems producing those answers are not under anyone’s control. What can honestly be promised instead is a shorter list than most proposals carry, and it is the list worth holding a vendor to. A refusal to name the engines tested is another, since engines differ enough that a number without them is uninterpretable.

The third is a report you cannot get a copy of. If the reporting lives only in a dashboard you lose access to when the contract ends, the record of what happened is the agency’s, not yours. These questions belong to the reporting check, which sits third among the five things a buyer can settle before signing, because what an agency will tell you only matters once you believe it can do the work.

SEO Is My Love Language was founded by Jose Villalobos, who has spent his career on a single discipline: getting businesses found, cited, and recommended by AI search. He has been a member of Koray Tuğberk Gübür’s Holistic SEO Community since 2022, is a graduate of the Topical Authority Course, holds the Google AI Professional Certificate, and is a member of Kyle Roof’s IMG. That combination, topical authority strategy paired with rigorous on-page execution, is what our team brings to every business we work with.

Two of our vertical pages close on what a business should be counting, which is the same list from the other side of the table: the stage numbers a dental practice should be shown and the roofing numbers worth asking a vendor for.

Frequently asked questions

What is the single best question to ask an agency about measurement?

Ask to see a reading it did not like. An agency that measures an outcome will have one, because real measurement produces unflattering results. An agency that reports activity will not, because activity always looks like progress. That question sorts faster than any credential check.

How do I know whether the questions being measured were chosen fairly?

Ask who chose them and when the list was frozen. If the vendor chose all of them, the number describes a set assembled by the party being judged. That is worth knowing rather than disqualifying, and the correct response is to add questions of your own to the same fixed list.

What does an irregular reporting schedule usually mean?

That the report is assembled when there is something to say, which selects for good news without anyone deciding to deceive. Ask for the last three consecutive reports rather than a sample, because a sample is built to be shown and three real ones reveal whether the format or the metric quietly changed.

Can any agency prove its work caused the result?

No, and a confident causal story is a warning sign. Seasonality, competitor behaviour and engine model changes cannot be separated from a content programme without a controlled test that almost nobody runs. A good answer describes what is controlled and admits what is not.

Which answers should end the conversation entirely?

A guarantee of a specific position in an AI answer, because those systems are not under anyone’s control. A refusal to name the engines tested, because engines differ enough that a number without them cannot be interpreted. And reporting you cannot keep a copy of, because then the record of what happened belongs to the agency rather than to you.

See where your business stands in AI answers

See where your business stands in AI answers today and what it would take to own the answer. Book a strategy call and we will show you.