100k A/B test per year at Microsoft
CRO Weekly #241
Hi,
Welcome to a fresh CRO weekly newsletter. In this edition 5 articles and 5 awesome new jobs.
Have a nice week!
Tom
CRO weekly sponsors
Thanks to Convert, Sitespect and Statsig for their support
Interesting reads
100k A/B test per year at Microsoft
At Microsoft, they run ~100k A/B tests annually with their internal Experimentation Platform ExP. To achieve this scale, they evolved ExP over the years, and one investment that truly enabled them to democratize decision making and make A/B testing effective across diverse organizations was to make ExP extensible.
Link to article
What to do when your experiment is postponed again?
Although it might feel very natural to push harder in such cases, take a step back instead. The product owner might already be stressed through having to tell you this. So, when you hear they don’t have time for an experiment to run, try to align with their goals and show the added value of experimentation for decision making.
Link to article
The fundamentals of javascript (JS) for AB testing
Check this video from Ricardo Martinez if you would like to learn Javascript (JS) to advance your AB Testing program.
Link to article
How organizations can embrace experimentation
What can organizations do to successfully integrate experimentation into their measurement frameworks? It’s not a question of choosing between observational models and experiments but of combining the two and embracing experimentation as a cornerstone of your strategy.
Link to article
The hamburger menu on the left or right side on mobile?
In this experiment they moved the burger menu on mobile devices from right-aligned (control) to left-aligned (variant).
Why?
Because we assumed users would find left-side menus more intuitive, reducing cognitive load and improving navigation (Cognitive Ease).
But here’s what we saw instead: ARPU decreased by -3.45% / Conversion Rate dipped by -0.74% / Average Order Value shrank by -2.56%.
Job opportunities
Looking for a new challenge in experimentation? Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.
This week’s featured roles:
CRO Lead at Fella Health (Europe, Latam and Asia)
Product Owner (CRO) at Leadtech (Remote)
Web Optimization Manager (CRO) at Pipedrive (Berlin, Germany)
Senior Conversion Rate Optimization (CRO) at Amenitiz (Barcelona, Spain)
Landing Page & CRO Strategist at Lunar Solar Group (USA)
Are you hiring?
Are you not subscribed yet to this newsletter? Click on the button below.