Shopify Flow | Culsin

Simple Split Testing adds one Shopify Flow action: Record split test event.

It records a custom event for the visitor who placed an order. Use it when the thing you want to measure is a fact about the order rather than a click on the storefront. “The order included a strap tool” is a good example: you can make that the metric a test is judged on, instead of waiting for revenue per visitor to reach a verdict.

Custom events are available on Growth and above.

Build the workflow

In Shopify admin, go to Flow and click Create workflow
Set the trigger to Order created
Add a condition on the order’s line items, for example any line item’s product is in a collection, or any line item’s product has a tag
Add the action Record split test event
Fill in the fields below
Turn the workflow on

The event starts counting from the next order that matches. It does not fill in past orders.

The action’s fields

Field	Required	What it does
Order	Yes	The order the event belongs to. Pick the order from the trigger
Event name	Yes	Lowercase letters, numbers and underscores, up to 64 characters, for example `tool_attached`. Created in Simple Split Testing on first use if it does not exist yet
Value	No	A number that is totalled per variant, for example the line item price
Label	No	A short piece of text the event is broken down by, for example the product title

Value gives you a total per variant on top of the count. Label gives you a breakdown, so you can see which products or reasons drove the fires rather than just how many there were. Both are optional and you can start sending them later without changing anything else.

Use the event as a test’s metric

The event appears on the Custom events page in the app, the same as one you define by hand.

To judge a test on it, open the test and set its primary metric to that event. Per variant you then get the number of visitors who fired it, the rate, and a significance verdict against your confidence level, exactly as for any other conversion goal.

The event counts once per visitor when the test is judged on the event’s rate, so a visitor who places two qualifying orders is one conversion, not two. Pick the event’s per order option as the metric instead to judge the test on the share of orders that carry it. That is the right unit when the question is about the basket (“how many orders included a tool”) rather than the visitor, and it is the unit to size such a test on.

Timing

Flow’s “Order created” trigger and the app’s own record of the order arrive at nearly the same moment, so the first run often reaches the app a second or two early. When that happens the action asks Flow to try again shortly, and Flow does. Nothing for you to do, and the run finishes normally.

Each run records the event once. If Flow re-sends a run it did not get an answer for, it does not count twice.

An order that is still not on record a day later is one the app was never told about, usually because it was placed before the app was installed. Those runs end with a message saying so.

When a run shows an error in Flow

Open the workflow’s run log in Flow to see the reason on the run.

The event name is not valid. Use 1 to 64 lowercase letters, numbers and underscores, for example newsletter_signup. Names that clash with a built-in event such as checkout_completed are refused too.
Your plan’s custom event limit is reached. The message includes a link to the plans page. Delete an event you no longer use, or upgrade, then run the workflow again.
The order is not on record. See Timing above.

A refused run does not consume one of your plan’s custom events. The event is only created once the order has been found.

Refunds

A recorded event is a count, and a later refund does not remove it. An order that is fully refunded still counts toward the variant it was placed on.

If you want refunds reflected in the verdict, judge the test on revenue per visitor or AOV instead, which do account for them. See statistics.