AI agents & MCP

Simple Split Testing runs a Model Context Protocol (MCP) server. Point an AI coding agent at it with a token and the agent can create tests, author variants, launch them, and read results without you leaving your editor. Most apps you install. Culsin apps you build with.

One honest note up front. Creating a test defines the experiment, but the variants swap real elements in your theme. The agent hands you the exact Liquid to add (via get_storefront_markup), and can paste it in itself if it has access to your theme files. Connecting is no code. Authoring a variant is one paste into your theme. See Storefront markup.

What you can ask an agent to do

“Create an A/B test on my hero headline with two variants.”
“Launch the hero test.” / “Pause it.”
“How is the checkout-button test doing? Is it significant yet?”
“Only run this test on mobile visitors in the US.”
“Give me preview links for each variant.”
“Add a custom conversion event called newsletter_signup.”
“The test is live but showing nothing. What’s wrong?”
“Which plan am I on and how many sessions have I used?”

The agent has 16 tools covering the full test lifecycle.

1. Generate a token

In the app, open Integrations and click Generate token. Give it a name so you can tell tokens apart later, and pick an expiry of 30 days, 90 days, or 1 year. The token is shown once, so copy it then.

A token is scoped to your store. It can only read and write that store’s tests. Revoke it any time from the same page. A revoked or expired token stops working immediately, and so does every token for the store if the app is uninstalled.

2. Connect your client

The MCP server URL is:

https://simple-split-testing.culsin.com/mcp

The Integrations page shows the same URL alongside a config block you can copy and drop your token into.

Claude Code

claude mcp add --transport http simple-split-testing \
  https://simple-split-testing.culsin.com/mcp \
  --header "Authorization: Bearer sst_live_your_token_here"

Then, in a session, ask Claude to list or create split tests.

Cursor

Add this to .cursor/mcp.json in your project, or ~/.cursor/mcp.json for all projects:

{
  "mcpServers": {
    "simple-split-testing": {
      "url": "https://simple-split-testing.culsin.com/mcp",
      "headers": { "Authorization": "Bearer sst_live_your_token_here" }
    }
  }
}

Any MCP client

The server speaks MCP over HTTP (Streamable HTTP transport). Point any compatible client at the URL with an Authorization: Bearer <token> header:

{
  "mcpServers": {
    "simple-split-testing": {
      "url": "https://simple-split-testing.culsin.com/mcp",
      "headers": { "Authorization": "Bearer sst_live_your_token_here" }
    }
  }
}

3. A worked example

Ask your agent: “Create an A/B test on my hero CTA, control versus a green button, and give me the markup to add.”

The agent calls create_split_test with two variants. The response includes the test and a block of Liquid.
Add that Liquid where the CTA renders in your theme. If the agent has access to your theme files, it can make the edit for you. The markup uses data-split-id and data-split-variant. See Storefront markup for the full attribute reference.
Ask: “Show me both versions.” The agent calls get_preview_urls and hands back one signed link per variant. Each link is good for an hour and renders your live storefront with that variant forced. See Preview links.
Ask: “Launch it.” The agent calls set_split_test_status and moves the test to active.
Once traffic flows, ask: “How is the hero CTA test doing?” The agent calls get_results and reports per-variant conversion and the significance verdict. See Statistics for how those numbers are computed.

Tools

Eight tools read, eight write.

Tool	Reads or writes	What it does
`list_split_tests`	Reads	List all tests with status and variants
`get_split_test`	Reads	Get one test with variants and audience rules
`get_storefront_markup`	Reads	Get the Liquid to add to your theme for a test
`get_preview_urls`	Reads	A signed link per variant that renders the storefront with that variant forced
`get_results`	Reads	Per-variant sessions, conversions, conversion rate, and significance
`get_setup_status`	Reads	Web pixel state and whether the theme app block is enabled on the published theme
`get_plan_usage`	Reads	Plan name, limits, and current usage for sessions, concurrent tests, and custom events
`list_custom_events`	Reads	List the store’s custom conversion events
`create_split_test`	Writes	Create a test with 2 or more variants. Returns the storefront markup
`update_split_test`	Writes	Edit name, description, or primary KPI
`replace_variants`	Writes	Replace the variant set. Pass a variant’s handle to keep it across a rename
`set_audience`	Writes	Set URL, country, device, and visitor-type targeting, or clear it
`set_split_test_status`	Writes	Launch (`active`), pause, revert to draft, or complete
`delete_split_test`	Writes	Delete a test and its recorded data
`create_custom_event`	Writes	Define a custom conversion event by name
`delete_custom_event`	Writes	Remove a custom event definition. Firings already recorded are kept

Every tool is scoped to the store the token belongs to. There is no way to reach another store’s data through the MCP.

Guardrails

An agent hits the same rules you would in the admin.

Launching needs the Split Embed Script app embed enabled on your published theme. Activating a test, creating one directly as active, and resuming a paused one are all refused while it is off, since the test would collect nothing. get_setup_status reports the current state, and it is the first thing to check when a live test shows no data.
delete_split_test and replace_variants require confirm: true when the test is active or paused and already has assigned visitors. The refusal names the visitor count, so the agent has a real number to show you before it retries. Editing a draft needs no confirmation.
set_audience needs the Growth plan or above. Clearing targeting works on any plan.
Custom event names are 1 to 64 characters, lowercase letters, digits, and underscores only, and cannot reuse a built-in event name such as checkout_completed.
primaryKpi accepts conversion_rate, aov, sessions, or add_to_cart_rate.

Plan limits

The MCP enforces the same plan limits as the app. On the Free plan an agent can run 1 concurrent test and define 3 custom events. Paid plans raise or remove those caps. If a call would exceed your plan, the tool returns a clear message telling you to finish a test, delete an event, or upgrade. Ask the agent for get_plan_usage to see where you stand before it starts creating things. Manage your plan on the Plans page in the app.

Good to know

The token goes in the Authorization header, never in a URL. Treat it like a password.
Tests an agent creates are identical to tests created in the admin. Edit or launch them from either side.
Changing variants on a running test re-buckets visitors, so prefer editing while a test is still a draft.
Uninstalling the app stops every token for the store. Reinstalling does not bring them back, so generate a new one from the Integrations page.
The agent can read results but cannot see a visitor’s personal data. Variant payloads carry handles, not names or customer information.