Prerequisite: None

Ronny Kohavi

Distinguished Engineer, General Manager, Analysis and Experimentation

Microsoft

The Internet provides developers of connected software, including web sites and applications, an unprecedented opportunity to accelerate innovation by evaluating ideas quickly and accurately using trustworthy controlled experiments (e.g., A/B tests, randomized controlled trials, field tests, bucket tests, flights).

While the theory of a controlled experiment is simple, and dates to Sir Ronald A. Fisher’s experiments at the Rothamsted Agricultural Experimental Station in England in the 1920s, the deployment and mining of online controlled experiments at scale (e.g., hundreds of experiments run every day at Microsoft) and deployment of online controlled experiments across dozens of web sites and applications has taught us many practical lessons. We provide an introduction, share real examples, pitfalls, and discuss scale challenges.

Want to hear more on A/B testing from Ronny? Check out his latest article in Harvard Business Review: http://bit.ly/HBR_AB