Upsell Revenue Attribution, Counted Right

Upsell revenue attribution is where an upsell widget makes it easy to produce a flattering number. Add up the revenue on every line that shoppers added through the widget and you get a figure that grows with traffic and looks like the widget's contribution. It is not. Some of those shoppers would have put the same item in the cart on their own, and revenue they would have given you anyway is not revenue the widget created. This page separates the two numbers, explains why the first overstates the second, says what Shopify's reports can and cannot show for this, and walks through a holdout that measures the second one. The worked example uses invented numbers chosen for the arithmetic, not benchmarks.

Two numbers that get confused

Revenue on added lines is what shoppers spent on items they added through the widget. You can count it from your orders once you can tell which lines came from the widget. It answers the question "how much money passed through this feature?"

Incremental revenue is the revenue you would not have had without the widget. It answers "what did this feature add to the store?" You cannot read it off the orders, because the orders only show what happened, not what would have happened without the widget. The only way to see it is to compare shoppers who saw the widget with comparable shoppers who did not.

Attach rate, which attach rate defines, is a count of how often the widget's item is on the order. It is useful for tuning the widget, and it shares the weakness of the first number: it measures activity, not effect.

Why "added via widget" overstates

Three kinds of shopper end up in the added-line revenue.

1. Would have added it anyway. They came for the main product and the accessory. The widget put the accessory one click closer, but they would have found it.
2. Swapped, not added. They bought the widget's item instead of something else they would have bought. The order total barely moves.
3. Truly persuaded. They had not planned to buy the item, and the suggestion changed the order. This is the only group that is incremental.

The first number counts all three. The second counts only the third. How big the third group is depends on your products and cannot be inferred from a rate someone else measured, which is why this page quotes none.

If the widget also carries a discount, the first group makes it worse: those shoppers now receive a discount on something they planned to buy at full price. Frequently bought together on Shopify covers building the widget; this page is about not fooling yourself afterward.

What Shopify's reports can and cannot show

Shopify's sales reports page describes a report called Sales by discount codes, which "displays your sales grouped by the name of the discount, such as the automatic discount name or discount code." If your upsell offer uses a named automatic discount, that report is the quickest way to see sales that carried the discount, and how often it was applied. The page does not say whether the sales figure covers whole orders or only the discounted lines, so check with a test order before you quote it as widget revenue.

The same page lists a Total sales by product report, which shows product sales and net quantity without shipping. That tells you how many units of the add-on item sold in total, but not how many came through the widget.

Exporting orders gives you the line-level view. Shopify's export article says the file includes line item details such as quantity, name, price and SKU, with multi-item orders listed on separate lines, a Discount Code column that "does not include automatic discounts," and a Discount Amount column showing the discount applied to the order. So an order that used a discount code for the widget's item shows that code in the export, at order level rather than line level, and an automatic discount cannot be seen in that column.

What none of these pages describe is a built-in view of "revenue that this widget created." That comparison needs a holdout.

A widget holdout

The idea is simple. Run the widget for part of your traffic and not for the rest, over the same period, and compare revenue per session. Incrementality for small stores covers how to split traffic fairly and how long to run it; two cautions are specific to upsells.

• Compare all revenue, not just the add-on's. If the widget makes shoppers swap one item for another, the add-on sales rise while the total stays flat. Only total revenue per session shows that.
• Compare contribution, not just revenue, because widget lines usually carry a discount and a different margin.

Here is a worked example. Assume a two week test with 10,000 sessions in each group.

The added-line figure is 7,500 dollars, or 0.75 dollars per session. The incremental figure is the difference in revenue per session: 5.20 minus 5.00, which is 0.20 dollars per session, or 2,000 dollars across the 10,000 sessions. The widget created about 27 percent of the revenue that passed through it (2,000 divided by 7,500, which is 26.7 percent), and the rest is the first and second groups above.

Now add margin. Assume widget lines earn a 30 percent contribution after their discount and all other revenue earns 40 percent. The widget group's contribution is 7,500 times 0.30, which is 2,250, plus 44,500 times 0.40, which is 17,800, for 20,050 dollars. The group without the widget earns 50,000 times 0.40, which is 20,000 dollars. The widget added 50 dollars of contribution per 10,000 sessions, under 1 percent of the 7,500 dollars of revenue it handled (50 divided by 7,500 is 0.67 percent). At a 25 percent margin on widget lines the same test would show a loss: 7,500 times 0.25 is 1,875, plus 17,800, is 19,675 dollars, which is 325 below the group without the widget.

One more caution. A 4 percent revenue difference on 20,000 sessions can be noise. Run the test long enough that the result holds on both weeks, or run it twice, before you act on a small difference.

A one-page readout

Write the result on one page so that you can compare one test with the next.

1. Dates and traffic: the period, sessions per group, how the split was made.
2. Revenue per session in each group, and the difference.
3. Contribution per session in each group, and the difference.
4. Added-line revenue and the share of it that was incremental.
5. Decision: keep, change the discount or the product, or turn it off, with the date to test again.

Promotion attribution explains how Shopify's own signals disagree; the readout is where you choose which of them you trust and why.

Building frequently bought together

Open the Frequently bought together template, which the app describes as "Suggest matching products on the product page." Decide the test before you turn it on: the dates, the share of sessions that see the widget, and the three figures for the readout. Then compare total revenue and contribution per session between the groups, not the revenue on the lines the widget added.