Biased and Inefficient

I have seen the school ‘climate strike’ in NZ being described as a publicity stunt that won’t provide any real solutions.

From XKCD today How can we check this calculation?

A couple of years ago, I stored a lot of Auckland bus location data for what was going to be a news story.

Someone on Twitter (who I’m reluctant to out, considering) said they’d got reviews back from an epidemiology journal, and one reviewer had cautioned against the use of R, because “the opening sentence in an R output points out that basically you use it at…

The Shapiro-Wilk test is a test of the null hypothesis that data come from a Normal distribution, with power against a wide range of alternatives.

I’m not on that list, but I do have reckons anyway.

Saying that you don’t know something can be hard for people; it’s also hard for prediction algorithms.

Can we invent the case-control design?

Suppose you have a (potentially unequal-probability) sample of schools, and within each school a (potentially unequal-probability) sample of students, and you want to fit a linear mixed model.

They’re back!

I wrote a while back about a toy case of the Bayesian surprise problem: what does Bayes Theorem tell you to believe when you get really surprising data.

It’s an interesting question, but the implication of wasted depends not just on the actual statistics about package survival (which we’ll get to), but on why people write packages.

I got asked for more detail about the svycontrast() function, so I thought I’d post it here too.

Via Scott Aaronson’s blog I found an arXiv abstract and then an early paper (PDF) about doing singular value decomposition of an \(m\times n\) matrix in less than \(O(mn)\) time.

We have four academic positions open in the Department of Statistics.

The svy2lme package for linear mixed models under complex sampling may still contain nuts, but at least the user interface has settled down and it gives plausible answers for some toy examples.

There’s a new version of the survey package on CRAN, version 3.34.

As I’ve written before, New Zealand has a National Pseudo-Random Number Generator.

Last year, Robyn Speer wrote a really great post How to make a racist AI without really trying.

Newsroom, an online-only New Zealand news site, has written a series of stories critical of Sir Ray Avery and his R&D efforts in medical devices.

(for Deborah Nolan and Louise Ryan, ISCB/ASC 2018, after Henry Reed) Today we have naming of stats.

Notes and references for my keynote at the joint International Society for Clinical Biostatistics and Australian Statistical Conference: Abstracts from the JSM on imputation and weighting for audit subsets: Shepherd & Giganti; Shaw & Oh Preprint on nearly…

Where are the buses?

In the ‘regression tests’ that are part of any change to the base-R source code, there’s a file called p-r-random-tests.R.

Miles McBain has a nice post about quoting in R and the tidyeval procedure.

Q: What do you mean, “reboot”?

I don’t normally explain jokes, but it’s clear from the useR conference that the name of the new R package reticulate divides people into two groups: amused or bemused.

Admittedly, it’s conceivable for parents to make a simple error in assigning a child’s name.

Charco Hui, as his Honours project in Statistics, has been writing a package for complex-survey analysis using dplyr and dbplyr.

This is a picture of all the genetic associations found in genome-wide association studies, sorted by chromosome.

One way to prove you can really keep up with the cool kids is to move your blog to GitHub the day after they get bought by Microsoft.

I’m a statistical researcher in Auckland.

So, I gave one of the graduation addresses during Silly Hat Week last week.

I’m working on an R package for mixed models under complex sampling.

The proposal to change p-value thresholds from 0.05 to 0.005 won’t die.

Chebyshev’s inequality (or any of the other transliterations of Чебышёв) is a simple bound on the proportion of a distribution that can be far from the mean.

Xudong Huang and I are working on fitting mixed models using pairwise composite likelihood.

There basically isn’t an algorithm for generalised linear models that computes the maximum likelihood estimator in a single pass over the \(N\) observatons in the data.

If you have a thing with lots of indices, such as a fourth-order sampling probability \(\pi_{ijk\ell}\) (the probability that individuals \(i\), \(j\), \(k\) and \(\ell\) are all sampled), there will likely be scenarios where it has lots and lots of symme…

They’re back!

If you know about design-based analysis of survey data, you probably know about the Rao-Scott tests, at least in contingency tables.

I’ve been reading two somewhat depressing documents today.

Attention Conservation Notice: I’m putting this in a blog post in the hope it makes it easier for other people to find when they encounter the problem.

If you own an e-bike, you get used to certain questions: how fast does it go, how often does it need to be charged, how much did it cost?

I’m interested in estimation for complex samples from structured data — clustered, longitudinal, family, network — and so I’m interested in intuition for estimating statistics of pairs, triples, etc.

A quadratic form in Gaussian variables has the same distribution as a linear combination of independent \(\chi^2_1\) variables – that’s obvious if the Gaussian variables are independent and the quadratic form is diagonal, and you can make that true by cha…

Attention Conservation Notice: while this probability question actually came up in in real life, that’s just because I’m a nerd.

There was (as usual) controversy about some of the NCEA maths questions this year.

From a math point of view, it’s an interesting example of how the mean of the maximum of a set of random variables is higher than the max of the individual means – Andrew Gelman Controlling the maximum of a set of random variables is an important problem…

Later this year, Auckland is hosting the Asian regional meeting of the International Association for Statistical Computing.