Demetri Pananos Ph.D

While at Zapier, I remember PMs asking me about using Bayesian statistics in our A/B tests because Bayes makes the experiment “run faster”.

I want to start this post at the end: statistically significant results from low powered experiments are, almost certainly, an exaggeration to the truth.

We’re gearing up for a really cool decision-theory feature at Datadog.

Earlier this year, I worked on our cumulative impact feature; an implementation of the hierarchical normal normal model to enable customers to estimate the cumulative impact of past experiments while adjusting for the Winner’s Curse.

Before we get started, I want to say that here may be good reasons to use a normal prior over a Student’s t prior.

The “Text” section of my “How I Use AI” reads All text on this blog, and in publications in which I am listed as an author, is written by me […] I will never copy-and-paste from an LLM, nor will I ask an LLM to write a blog post, section of a paper, or em…

Motivation When estimating a treatment effect (typically the lift), , using the Bayesian methodology, Datadog uses a default prior of Do I really believe that true effects are normally distributed?

Let be a sample of dimensional random variables with mean .

I work 10 hours per week at my University’s medical school consulting on statistics for residents and physicians.

Important In accordance with my AI policy, I have written this blog post myself.

This post is a spiritual successor to an early post on this blog and a fast follow here from this post about A/B test run times for Bayesians.

Important This blog post is inspired, almost entirely, by Tyler Buffington’s excellent statistics reading group presentation today at Datadog on the value of information in A/B testing.

I asked claude code to read through all the posts I’ve made in this blog in order to pull out some of my principles.

I still clean data from time to time, and I find myself writing patterns that looks like raw_data %>% mutate( smoking = case_match(str_to_lower(smoking), "y" ~ "Y", "n" ~ "N", .default = "N" ), hypertension = case_match(str…

On The Validity of a Normal Approximation to The Ratio of Means A/B tests often target the “lift” in the metric as an estimand.

To the Economists, I am sorry.

I’ve been climbing this hill for what feels like a long time.

Take a look at this interaction I had with user jginestet on Cross Validated.

I keep coming back to A New Look at P Values for Randomized Clinical Trials – an instant classic paper on RCTs, which could be extended to online experimentation as well.

I recently wrote a blog post for my employer in which I made some claims regarding Bayesian statistics.