Free Sample With Purchase: A Test Plan

A free sample with purchase is not a discount and not quite a gift. It is a small bet that a shopper who tries one product will come back and buy it at full size. That bet is easy to make and hard to read, because the order that carried the sample looks fine whether or not the sample did anything. This is a test plan for the free sample with purchase offer on a Shopify store: what question it answers, how to choose the sample, what to write down before launch, which Shopify reports to read afterward, and how to turn the result into a cost per repeat buyer.

The question the test answers

The sample is cheap, so the real question is not "did the order go through." It is this: do shoppers who received the sample go on to buy the sampled product later, at a higher rate than shoppers who did not?

Everything else is a side effect. Average order value barely moves, because the sample is not something shoppers add for the price. Conversion at checkout may move a little, but that is hard to separate from everything else running that week. The repurchase of the sampled product is the one outcome that only the sample can explain, so it is the one to measure.

If you only want to lift the order total today, a spend-based gift is the better tool, and Free Gift Threshold: Pick a Spend Level covers it. A sample earns its place when the product is a repeat purchase: something consumed, replaced or refilled.

Choose the sample: a full-size product must exist for it

Three filters decide whether a product makes a good sample.

1. A full-size version is for sale. A sample of something you do not sell in a normal size is just a gift. The test needs a product the shopper can buy next.
2. The sample can be judged quickly. Something a shopper can try in a week works. A product that needs a month of use will give you a test that outlasts your patience.
3. It is cheap to land. Add the unit cost, the packaging and any extra shipping weight. The formula for that is in Free Gift Margin Math, With Worked Examples, so it is not repeated here. Keep the sample's landed cost below the margin on one full-size order, or the first repeat purchase cannot pay it back.

If your catalog is beauty or personal care, Beauty Promotions: Samples, Minis, Bundles covers sampling as one tactic among several; this post is only the measurement plan.

Then pick one sample, not three. Offering a choice of samples splits a small test into smaller groups that are too thin to read.

Define success before launch

Write the definition down before the first sampled order ships. Changing it afterward is how a flat result becomes a "win."

A workable definition has four parts:

• Who counts: customers whose order included the sample, matched by email.
• What counts: a later order that contains the full-size version of the sampled product. A later order for something else is not a repeat for this test.
• By when: a fixed window, for example 90 days after the sampled order. Pick it from how fast the product is used up.
• Against what: a comparison group of customers who bought the same product without a sample in a similar period. The next section explains how to build it.

Also write down the decision rule: "If the sample group's repurchase rate is not at least X points higher than the comparison group's, we stop." Setting X from the cost math below is better than setting it from hope.

Reading Shopify's customer reports

Shopify's Help Center describes a set of customer reports, and two of them fit this test.

Customer cohort analysis. According to the Help Center, the report tracks customer acquisition and retention by grouping customers into cohorts based on the date of their first order. Its Metric menu offers number of customers, customer retention rate, gross sales, net sales, average order value and amount spent per customer. The page does not give a formula for retention rate, so read it as Shopify's own measure and compare cohorts against each other rather than against an outside number. The Help Center pages we read do not state which plans include the report, so check your own admin.

Returning customers. The Help Center says this report shows data about all customers whose order history includes two or more orders, and lists each customer's name, email, first and most recent order dates, order count, average spend and total spend.

The Returning customers description lists no filter by product, and that matters, because "returned and bought again" is not the same as "returned and bought the sampled product." The cohort report does offer a first-order filter named Product name, among others (sales channel, marketing channel, marketing type, subscription), so you can build cohorts of customers whose first order included a given product (for example the sample itself, if it is a line item). Use the reports for the shape of the result, and use an orders export for the exact one. Shopify's export lists additional line items of a multi-item order on separate lines and leaves many fields blank on those lines, so fill the order-level columns (such as email) down before you filter. The export has line item name and line item SKU columns. Filter to the full-size product's SKU, match its emails against the sample group, and count the distinct customers who ordered it within your window.

One caution on the comparison group: Shopify's discount tools do not randomize who receives an offer (the pages we read describe no such feature). So build the comparison yourself. The simplest honest version is time-based. Run the sample for a fixed stretch, and compare against the stretch of the same length before it, with no other promotion on the product in either period. Say in your notes that it is a before-and-after comparison, because seasonality can leak into it.

Sample size in plain terms

You do not need a statistics course, only a rule that stops you reading the result too early. For the general method of judging a promotion against a baseline, see How to Measure Whether a Promotion Worked. Three things make a small test misleading:

• Few repeat buyers. If 40 shoppers in the comparison group re-bought, adding or removing 10 buyers moves that rate by a quarter. Counts that small swing by luck.
• A window that has not closed. Read the result only after every sampled order is at least as old as your window.
• A second promotion on the same product. Anything that discounts the full-size product during the window clouds the comparison.

A planning rule, not a law of statistics: do not read the result until each group has at least 1,000 orders and every order is older than your window. If your store ships fewer than that in a season, run the test for longer instead of reading it sooner.

Cost per converted repeat buyer

Here is a worked example. Every number is an assumption for illustration, not a benchmark. Replace each with your own.

• Full-size product: 24.00 dollars, with a 60 percent gross margin, which is 14.40 dollars of margin per full-size order.
• Sample landed cost: 1.40 dollars including packaging, with no added shipping cost.
• Sample group: 2,000 orders. Comparison group: 2,000 orders.
• In the 90-day window, 100 sample-group customers (5.0 percent) and 40 comparison-group customers (2.0 percent) bought the full-size product.

Read the last row carefully. In this made-up case the first repeat order does not pay the sample back. The offer only pays if those 60 extra buyers keep buying, about three orders each over their time with you, so the decision depends on how often the product is re-bought, not on the sample. That is a customer lifetime value question; Customer Lifetime Value on Shopify, Plainly shows how to estimate it from your own reports.

There is a shortcut for the stop rule. For the first repeat order alone to pay back the sample, the sample group's repurchase rate must exceed the comparison group's by the sample's cost divided by the margin on a full-size order: 1.40 / 14.40, or about 9.7 percentage points. If you think a gain that large is unrealistic for your product, you are really betting on several repeat orders, and your test window should be long enough to see them.

Mind the "free" rules while you test

A sample offered with a purchase is a "free" offer, so the Federal Trade Commission's guide on the word applies. Three parts matter for a test:

• The terms of a free offer should be set out clearly and conspicuously at the outset, in close conjunction with the offer, and an asterisk footnote does not count (16 CFR 251.1(c)).
• The guide says a buyer is entitled to believe the merchant will not recover the free item's cost by marking up the regular price of the item that must be purchased (251.1(b)).
• On frequency, the guide says a single size of a product should not be advertised with a "free" offer in a trade area for more than 6 months in any 12-month period, and at least 30 days should pass before another such offer (251.1(h)).

The last point shapes the test: a sample that runs all year does not fit that guidance. Plan a fixed run of up to six months, then a gap, and do not raise the product's price during the run to cover the sample. Swapping the word "free" for "gift" or "bonus" does not fix a problem, since the guide addresses similar terms too (251.1(i)). This is not legal advice; confirm your own offer with a lawyer.

Setting up the sample

In Freebies Engine by Log Studio Lab, the Free sample with purchase template is described as "Every order gets a small free product." Pick the sample product, set the start and end dates from your test plan, and put the terms next to the offer wherever it is shown. Keep a one-line log of the start date, the end date, the sample SKU, and the full-size SKU the test will count. When the window closes, run the export, fill in the table above with your numbers, and apply the stop rule you wrote at the start. For how a single free item compares with a discount on cost, see Why a 15% Discount Costs More Than You Think, and for the basics of the offer type, What Is a Free Gift With Purchase?.