How to Measure Whether a Promotion Worked

The usual way a store judges a promotion is to look at the sales total on the last day and feel good or bad. That tells you almost nothing, because sales go up and down for reasons that have nothing to do with the offer: a weekend, a payday, a newsletter, a competitor's stock-out. To measure promotion effectiveness you do not need a data team. You need to decide what success is before the offer goes live, compare against weeks that are honestly comparable, read two reports Shopify already gives you, and look at the weeks after the offer ended, which is where most promotions quietly lose. This page gives the method in that order, with one worked example whose numbers are invented for the illustration and labeled as such.

Decide the metric before launch

Write one sentence before the offer goes live: "This promotion worked if X rose by at least Y without Z falling." Choose X from a short list:

• Units of the target product, for an offer meant to move a specific item.
• Orders, for an offer meant to bring in more buyers.
• Profit per order, for an offer that changes the basket. This is the safest metric when the discount costs money, because revenue can rise while profit falls. Break-Even Discount: Extra Volume You Need shows the volume a given discount has to add before profit is level.

Z is your guardrail, typically gross profit or the orders of your full-price products. Writing it down matters for a plain reason: after the fact, every promotion can be described as a success by choosing the metric that went up. Stop Training Shoppers to Wait for Sales is about what happens when that habit repeats.

Pick a comparison window you can defend

You are comparing the promotion period with a baseline. A baseline is only fair if it matches in three ways.

• Same weekdays. Compare whole weeks, or the same count of each weekday. A promotion running Thursday to Sunday cannot be compared with Monday to Wednesday.
• Same season. Compare to the weeks around the offer, not to a different month. Last year's same weeks are useful for gift categories, with a note about what else changed.
• No other promotion. If a free shipping banner was live in the baseline weeks and not in the offer weeks, you measured two changes at once.

The simplest baseline is the average of the three weeks before the offer, provided none of them had another promotion. The simplest comparison is promotion week against that average, then the weeks after against the same average.

Read the reports Shopify already has

Shopify's Help Center lists the sales reports in Analytics. Three of them cover most of what you need: "Total sales over time", "Total sales by product" and "Sales by discount codes".

Total sales over time and Total sales by product give you the baseline and the promotion window for the store and for the target item. Export or read the weekly totals and put them in a table.

Sales by discount codes is the report for the offer itself. According to the page it groups sales by the name of the discount, covering automatic discounts as well as codes, and its columns include orders, discount amount and total sales. Two details change how you read it:

• The page says the sales value in this report includes only the total sales for the specific line items affected by the discount. Treat the figures as a way to compare discounts with each other, not as a count of how much the offer added.
• If you have combinable discounts, the page says the same order might be included multiple times, once in each row for every applied discount. Do not add rows together.

The report also covers only sales after May 1, 2017 (UTC), which is not a limit you will hit. The takeaway is that this report tells you how much the offer was used and what it cost. It does not tell you whether the shopper would have bought anyway. For that you need the baseline.

Pull-forward: look at the weeks after

A promotion often moves purchases forward in time. Shoppers who were going to buy next week buy this week at a discount, and the weeks after dip. The only way to see it is to keep measuring after the offer ends.

A worked example with invented numbers, for illustration only. A store promotes one product. In the three weeks before the offer it sold 120, 118 and 122 units, an average of 120 a week. During the promotion week it sold 190. In the two weeks after, it sold 85 and 118.

• Apparent lift: 190 minus 120 is 70 extra units.
• Dip in the first week after: 120 minus 85 is 35 units below baseline.
• Dip in the second week: 120 minus 118 is 2 units below baseline.
• Net lift: 70 minus 35 minus 2 is 33 units.

The promotion week looked twice as good as the result. Whether 33 extra units pay for the discount is the profit question, and the profit-per-order metric answers it. Keep measuring for as long as it takes the weekly number to return to the baseline, and no longer than the typical time your customers take between purchases. If your product is bought once or twice a year, a four-week window shows nothing and you will need last year's numbers.

A one-page promotion log

Keep one log, in a spreadsheet, with one row per promotion. The fields, described in words:

The log has two jobs. It forces the success sentence to exist before the offer, and in a year it lets you compare your own promotions with each other. Entries like "other activity: newsletter on day two" explain half of the surprises.

Show one offer at a time so the result is readable

Overlapping promotions are the biggest threat to a clean read. When a free shipping bar, a code and a gift offer are live together, you cannot tell which one moved the number. Running one offer at a time during a test is a discipline of the calendar, and it is also a matter of what the storefront shows. The Today's offers widget template in Freebies Engine by Log Studio Lab is described as "A launcher on your storefront listing live offers." Use it so the shopper sees the one offer under test, and keep the rest of the offers off while the test runs. When the test is done, check the result against the log and the success sentence. The holiday offer calendar is a helpful way to give each test its own window and keep the weeks around it clear. Plan requirements are on the pricing page.

Common ways the read goes wrong

• A baseline with another promotion in it. The offer looks weak because the baseline was strong.
• Stopping the measurement on the last day. Pull-forward is invisible if you stop.
• One large order. A wholesale buyer in the offer week can account for the whole lift. Remove it from both windows and compare again.
• Revenue as the metric. A discount can raise revenue and lower profit. Choose profit per order for any offer that costs money.
• Comparing a short offer with a long baseline average. Compare weeks to weeks.

None of these needs software. They need a written sentence, a fair baseline and the patience to look at the weeks after.