Test a mini app before you keep it.
Mini app tests are A/B tests that Experiment Engine runs on the Freebies Engine mini apps: a version with the mini app against one without it, or two versions of its settings or of its discount. They are for Shopify stores that want to know what a mini app earns on their own traffic before they keep it.
Experiment Engine by Log Studio Lab is not on the Shopify App Store yet. The mini apps work on their own in the meantime; ask us to tell you when Testing opens.
OriginalBundle Builder3 items: 10% offSummary beside the products
Version BBundle Builder3 items: 15% offSummary beside the products
How it works
- Start from the mini app. In the mini app's editor, choose Test this with Experiments. Or open the publish dialog and choose Publish as a test: your draft becomes version B and what is live stays the original.
- Pick what version B changes. Switch the mini app off, change some of its settings, or, for Bundle Builder and Quantity Breaks, change the discount. Visitors are split between the versions.
- Read the results, then apply the winner. Results compare the versions per visitor, with the mini app's own numbers next to them. Apply the winner from the results page, or keep things as they are.
What each mini app can test
| Mini app | On or off | Settings | Discount |
|---|---|---|---|
| Bundle Builder | Bundle Builder shown against a version without it. | The summary beside the products against the summary below them, on desktop. | One discount percent against another, or different percents per tier. Checkout charges the version each shopper saw. |
| AI Chat | The chat button shown against a version without it. | Not available | Not available |
| Urgency Bar | The bar shown against a version without it. | One message against another, or a different set of rotating messages. | Not available |
| Sticky Add to Cart | The bar shown against a version without it, on every device or on phones only. | Not available | Not available |
| Sales Pop | The card shown against a version without it. | Not available | Not available |
| Quantity Breaks | The tiers shown against a version without them, and without their discount. | Cards against a compact row, or a different spacing or corner style. | Different tiers, or a different kind of discount: percent off, an amount off each item or a price each. Checkout charges the version each shopper saw. |
Results in the mini app's own terms
Bundle impact
The share of orders with a bundle from Bundle Builder, and the revenue those bundles brought.
Quantity break uptake
The share of orders that used a Quantity Breaks tier.
Chat-assisted results
AI Chat conversations, the orders they led to and their revenue.
- Discount versions run inside the mini apps discount Function, so checkout charges what the version a shopper saw promised.
- While a test holds a mini app, its list row shows who leads, its editor shows an In test banner, and editing, pausing or archiving it asks you to confirm.
- If the mini app's subscription ends, the test pauses by itself instead of measuring a mini app that has stopped showing.
- Results show conversion, revenue per visitor and order value for every version, and profit per visitor on plans with profit tracking.
Mini app tests on each Experiment Engine plan
Experiment Engine Free1 running
Experiment Engine Growth3 running
Experiment Engine ProNo set limit running
Mini app tests run on the Freebies Engine mini apps only. See Experiment Engine pricing
Questions about mini app tests
- Which mini apps can I test?
- All six: Bundle Builder, AI Chat, Urgency Bar, Sticky Add to Cart, Sales Pop and Quantity Breaks. Each can be tested on or off. Urgency Bar, Bundle Builder and Quantity Breaks can also test settings, and Bundle Builder and Quantity Breaks can test their discount.
- Do I need both Freebies Engine and Experiment Engine?
- Yes. The mini apps live in Freebies Engine, and Experiment Engine runs the test. If Experiment Engine is not installed, Test this with Experiments becomes a link to install it.
- Does a discount test charge the right price at checkout?
- Yes. The discount of each version runs in the mini apps discount Function, which reads the version the shopper was assigned, so checkout matches what that shopper was shown.
- How many mini app tests can run at once?
- It depends on your Experiment Engine plan. The table above lists the limit of each plan.