What is Multi-Armed Bandit (MAB)?

The Explore vs. Exploit Dilemma

• Exploration helps you discover potential winners: what if something unexpected performs best?
• Exploitation gets you the most gains quickly by prioritizing what already seems to be working.

How MAB Works in PageSense

• Dynamic Allocation: All variations start with equal traffic, but successful versions rapidly receive a higher share of new visitors.
• Intelligent Learning (Thompson Sampling): We use a Bayesian approach called Thompson Sampling. This means the algorithm models each variation’s conversion rate as a probability distribution. Every time a user arrives, PageSense "samples" from these distributions and sends the visitor to the variation most likely to be a winner in that moment. Over time, the best-performing variation naturally gets the most traffic, while the system continues to test alternatives—never fully shutting off exploration.
• Real-Time Optimization: No waiting for weeks to reach a confidence level. MAB instantly and continuously optimizes, making it perfect for time-sensitive campaigns or when rapid gains are critical.

How Does Thompson Sampling Work?

• Every variation is assigned a probability model based on its past results (e.g., number of successes and failures).
• When a new visitor lands, Thompson Sampling draws a random value from each variation’s probability curve and picks the variation with the highest sampled value.
• Over thousands of visits, the best-performing variations emerge with higher probability and thus collect more traffic, but there’s always a small chance underdogs are revisited, keeping the system adaptive as performance shifts.
• This approach balances learning enough about all possibilities (exploration) with maximizing wins (exploitation) in real time.

Key Benefits of Using Multi-Armed Bandit

• Maximized Conversions During Testing: Rather than waiting until the end, MAB ramps up gains by directing visitors to top performers throughout the experiment ("earn while you learn").
• Perfect for Fast, High-Stakes Campaigns: Quickly react and capture value in short-term promotions or news cycles—where instant action matters most.
• Fully Automated Optimization: Once started, MAB takes over the traffic shifts, eliminating the need for manual tuning or constant attention.
• Adapts to Trends and Shifts: If user behavior or market dynamics change mid-campaign (seasonality, new trends), MAB spots the shift and finds the new winner, no need to restart tests from scratch.

Limitations to Keep in Mind

• Statistical Trade-Off: MAB is focused on maximizing immediate outcome, not providing scientific, final answers to “which variant is best?” Use traditional A/B tests for definitive, reportable, statistical comparison.
• Not for Research or Causal Insights: Need to precisely attribute cause and lift? A/B/n is still the gold standard for analytical rigor.
• Poor Fit for Delayed Feedback: Avoid for scenarios like email campaigns or long sales cycles where conversions don’t happen quickly, as that breaks MAB’s adaptive feedback loop.