Framing a question that data can answer

Analysis often fails at the start, when a vague question produces a vague answer. “Why are sales down?” cannot be answered; “Did the decline in September orders come from fewer new customers, fewer repeat customers or lower order value, by region?” can. Rewrite every incoming question using four parts: the metric, with its exact definition; the comparison, such as this month against the previous three or against the same month last year; the breakdown dimensions, such as region, channel or customer type; and the decision that depends on the answer.

The decision part is what keeps analysis useful. If no decision depends on the answer, the analysis is curiosity, which is fine but should be prioritised accordingly. If the decision is “whether to shift marketing budget from paid search to partnerships,” the analysis needs to produce the numbers that would change that decision: cost per new customer by channel and repeat rate by channel. Ask the requester directly: “What would you do differently if the answer were X rather than Y?” If the answer is nothing, narrow or decline the request.

Write the framed question at the top of your analysis file before touching data, along with the definitions. “Order” means a paid order excluding refunds and tests. “New customer” means first paid order in the period. “Region” uses the shipping country mapped to four regions. These definitions take ten minutes to write and prevent the most common disagreement in any data review, which is two people using the same word for different things. Every chart and table in the final memo will refer back to them.