A/B Testing

The A/B Testing feature allows organizations to compare different process alternatives enabling process owners to make informed decisions based on data, leading to improved efficiency and effectiveness of business processes.

By using A/B Testing, organizations can achieve the following benefits:

• Improved Process Performance: Identify process alternatives that deliver better results, such as reduced cycle time or increased productivity.

• Optimal Resource Allocation: Determine the most efficient allocation of resources to maximize process outcomes.

• Continuous Process Improvement: Utilize insights gained from A/B testing to continually optimize and refine business processes.

• Enhanced Decision-making: Make data-driven decisions based on objective evidence rather than subjective opinions.

Features

• Alternative Creation: Users can create and maintain two independent variants of a process model, enabling testing and comparison against different scenarios.

• A/B Testing: Designers can activate both alternatives for live execution and define rules for distribution, ensuring reliable and unbiased results.

• Data Collection & Analysis: Process metrics and user feedback are collected and compared against live alternatives, offering quantitative insights for informed decision-making.

A/B Testing addresses the challenges organizations face in identifying and improving their business processes efficiently. This feature provides concrete evidence to support decision-making by collecting and analyzing data from different process alternatives. It tracks and measures key performance indicators (KPIs) for each alternative, providing objective proof of which performs best.

Watch the following product tour to get a quick overview of how A/B testing works.

Enable A/B Testing

The first step to enable A/B Testing is to add an alternate path that a process can take.

Follow the steps below to create an alternative for a process model:

1. Edit an existing process or create a new one in the Modeler.

2. From the top left of the Modeler, click the + Create the Alternative B icon to add an alternate flow for the process
   .

3. When creating an alternative, ProcessMaker will clone the existing model (Alternative A) into a separate tab (Alternative B). Subsequent edits to either alternative will be saved independently.

4. Edit the two alternative as needed and click Publish to publish the alternatives.

5. The Start Event for both alternatives must be identical to enable A/B Testing. If the events differ, A/B Testing cannot be enabled, and the following message will be displayed.

   

• When publishing the process, Alternative A will be the default version, however, Designers can choose to publish either Alternative A, or B, or both.

• Simultaneously publishing both alternatives enables the A/B Testing configuration, while publishing just one will disable it.

• The active alternative will be highlighted in green, while the inactive alternative will be displayed in gray, indicating its disabled status.
  

Simple Testing Configuration

Follow these steps for configuring basic A/B Testing settings:

1. From the Publish New Version screen, select the A+B Alternatives tab.

   

2. Click the A/B Settings button to configure test settings.

3. For simple testing configuration, select the Simple option and adjust the slider to set the percentage distribution of incoming requests between the two alternatives. By default, requests are evenly distributed between the two alternatives.

   

4. Click Save and Publish to publish the two alternatives with basic A/B Testing configuration.

5. Click the Version Info button to go back to the alternative selection screen.

6. Click See Process A/B Testing Configuration to view tests results and summary.

7. For advanced configuration, see Advanced Testing Configuration.

• If no changes are made to the A/B Testing configuration, requests are distributed equally by default, with 50% allocated to Alternative A and 50% to Alternative B.

• Both alternatives must have the same start event for A/B testing to work correctly.

Advanced Testing Configuration

For advanced testing configuration, choose the Advanced option in A/B settings when publishing the alternatives and follow these instructions:

1. From A/B Settings, choose the Advanced option and input an expression to set the rules for activating either of the two alternatives.

   

   For more information about expressions, see FEEL Expression Syntax.

   Here are a few examples of how to enter expressions in the advanced settings:

   • When routing based on the value of a Request Variable, an expression like this can be used: loanApproved == "Yes"

   • If loanApproved variable stores a Boolean value, the expression can be written simply as:  loanApproved

   • It is also possible to combine expressions such as:  (loanApproved) or (loanRating > 500)

2. When the entered expression evaluates to TRUE, requests will follow the process model in Alternative B.

3. When the entered expression evaluates to FALSE, requests will be allocated between Alternative A and B based on the percentages selected in the slider.   Following table provides some examples of how advanced A/B Testing configuration works:

1. Click Save and Publish to publish the process.

2. Click the Version Info button to go back to the alternative selection screen.

3. Click See Process A/B Testing Configuration to view tests results and summary.

Watch the following product tour to learn how to setup advanced configuration for A/B Testing.

Republish Alternatives

After the initial configuration, you can edit and republish one or both alternatives.

Follow these steps to edit and republish the alternatives:

1. Edit an existing process that has A/B Testing enabled. The two alternatives will display in their respective tabs.

2. Edit one or both alternatives as needed and click the Publish button to publish them.

3. To individually publish the alternatives, select the tab associated with that alternative.

   

4. Optionally, enter a Version Name and Description and click Save and Publish.

   • Publishing only one alternative will disable the other, hence disabling the A/B Testing configuration. In this scenario, all requests will be routed using the published alternative only.

   • The active alternative will be highlighted in green, while the inactive alternative will be displayed in gray, indicating its disabled status.
     

5. To publish both alternatives, select the A+B Alternatives tab. This action enables the A/B Testing configuration.

6. Optionally, click the A/B Settings button to adjust the configuration for A/B Testing.

7. Click Save and Publish when done.