Biased and Inefficient

There’s a new version of the survey package on CRAN (yay!).

Lemma: Suppose \(X_1,\dots,X_K\) are bounded non-negative random variables, \(f_1,\dots, f_K\) are bounded non-negative functions of the corresponding \(X_k\), and \(g(X_1,\dots,X_K)=\sum_{k=1}^K f_k(X_k)\).

A couple of days ago, Emma Vitz posted some depressing maps of NZ housing affordability on Twitter.

One of the big steps forward in statistics over the past few decades is the widespread appreciation that regression modelling for causal inference and predictive inference are different1.

Suppose you have a data frame containing a matrix.

The moons of Mars are called Phobos and Deimos, named for twin sons of the war god Ares in Ancient Greek myth.

Attention conservation notice: an attempt to get a small number of other people, probably not including you, to adopt our notation.

Co-linearity of regression predictors (often ‘multicollinearity’) is one of those topics where a lot of regression textbooks are Unhelpful, in part because they don’t think about the reasons for fitting a model.

The Ihaka Lectures for 2021!

Attention Conservation Notice: these are the most popular posts, so if you cared you would already have seen them According to Google Analytics, these were the most popular pages on this site in 2020: What have I got against the Shapiro-Wilk test?

Attention conservation notice: genre fiction from decades ago Emma Lathen wrote a long series of light mystery novels starring John Putnam Thatcher, the senior vice president of a large investment bank.

We have an existing course at third-year undergraduate level that reprises the second-year regression course using SAS and also does a bit more SAS for data management.

If you have a fixed number of observations to allocate over a set of sampling strata and want to estimate the population total or mean of a variable \(Y\), Neyman proved in 1934 that the optimal allocation is proportional to the population size in the str…

Michael Droste tweeted Should I use Stata, R, Matlab, Julia, etc etc for my research?

I’m hearing that, in contrast to basically every other political question ever, there might be some benefit in boring middle-aged white men saying what they think about the cannabis referendum.

The VGAM package’s vglm() function, like the survey package’s svymle() function, allows for maximum likelihood fitting where linear predictors are added to one or more parameters of a distribution — but vglm() is a lot faster and has many distributions al…

“How do you know the treatment has the same effect on everyone when you don’t even know whether it makes the outcome go up or down?”

There are roughly three and half distinct uses of the term weights in statistical methodology, and it’s a problem for software documentation and software development.

Ok, so everyone and their companion animal is baking sourdough at the moment.

Version 4.0 of the survey package is on its way to CRAN.

The problem We want to estimate the population (or cohort) total of a variable \(X\) (actually, we don’t, we want to fit a regression model, but this part of the maths is the same).

The NZ Ministry of Health has been releasing information about COVID-19 cases, and Chris Knox (of the NZ Herald) has been collating it on github Here, I’m going to map it by District Health Board, using theDHBins package.

These are not the standard supermarket hot cross bun, which tends to a cinnamon and sugar flavour profile and a soft, moist crumb.

Josh De La Rosa wanted to know how to do quadratic trend tests such as these in the R survey package If you can’t read the image, the code extracts a sets of means for years from 2005 to 2014 and then tests a particular linear contrast of them, with coeff…

The Ihaka Lectures in computationally-oriented statistics are back!

Version 3.37 of the survey package is on CRAN now.

The Multiphase Optimisation Strategy for designing multifactor behavioural interventions should be used more.

This was my reply to the call for comments on the proposed Algorithms Charter Introduction I am a statistician with an interest in the technical details of predictive models and in their social impact.

Two years ago I started a 3rd-year (final-year) undergraduate course called ‘Data Science Practice’1.

Since it’s, if not Christmas, at least Advent, here’s a book review.

I’m involved a research project that, among other things, will be comparing various health variables across NZ District Health Boards (DHBs).

The Oxford-Munich Code of Conduct for Professional Data Scientists (http://www.code-of-ethics.org/code-of-conduct/) is worth reading.

The old-fashioned1 way to review a book, such as, say, Randall Munroe’s how to, involves reading it.

The rimu package, which I published last week, does not use the tidyverse.

In various places, from the R help pages to books to course materials, you see R code like (x = runif(10, 0, 10)) ## [1] 1.610466 2.662462 1.517036 1.372483 4.272460 6.402148 8.347196 ## [8] 8.775537 1.863104 8.912241 which displays the value of x.

Multiple-response data is like factor data, except that you can be in more than one category.

I had an email request about making ivreg work with the survey package.

Attention Conservation Notice: This is a working note; when I understand it better, there will be changes in the survey package.

One issue in teaching generalised linear models (or likelihood theory) is the relationship between the Wald, score, and likelihood ratio tests.

The issue A paper reporting the induction of autism-type behaviour in mice by fecal microbiome transplants from humans was recently published in Cell.

It’s important for non-Bayesians (or non-exclusively-Bayesians) to remember that being a 95% confidence interval procedure is a fairly weak property.

An important concept in multistage survey analysis is the design degrees of freedom, which describes (or estimates) how many independent observations go into calculating variances, in a similar way to error degrees of freedom in experimental designs.

There is discussion from time to time on Kiwi Twitter about which public figures get treated worse on Twitter.

Elections and opinion polls look a bit similar from a distance, but they’re very different beasts.

I’ve written a bunch of times about nearly-true models.

Version 3.36 of the survey package and version 2.4 of mitools are up on CRAN.

Rosemarinus officinalis, by Samules on Pixabay April 25th is not the anniversary of a victory, or an armistice, or a successfull retreat like Dunkirk, or even a tragic last stand like Masada.

Surveys (especially educational surveys) have a thing called ‘plausible values’, which are a form of multiple imputation, only by design rather than because of non-response.

In our last episode, we worried about the penalised least squares criterion for linear mixed models.

‘Network meta-analysis’ is the only term I’ve coined that’s actually entered the general biostatistical vocabulary.