Peter Austin of the University of Toronto has written a useful paper that’s just out in Stats In Med. He looks at 12 different ways of matching exposed and unexposed cases on their propensity scores and covariates, using a simulation study. The algorithms range from computationally complex OR-type approaches where every potential pair is evaluated and an overall optimum matching found, to simpler ‘greedy’ ones where each exposed observation grabs the nearest unexposed observation without consideration for the overall optimality. Then, you have the choice of putting the paired unexposed observations back into the pot to be paired again to another exposed observation. Finally, you could define the closest match in some different ways, which were investigated back in 1985 in a very readable paper by Rosenbaum and Rubin.

The solid black circle is the exposed case. You might choose the closest unexposed observation on the basis of the covariates alone (Mahalanobis distance – the triangle), the propensity score alone (the square), both (Maha again – the star), or both having restricted to a subset within a certain distance in the propensity score, which Rosenbaum and Rubin memorably call “calipers” (the hollow circle).

The trouble with not using the calipers is two-fold: greater computing challenge and the propensity score being swamped by the many dimensions of covariates in the Mahalanobis distance. Rosenbaum & Rubin showed that the effect was to achieve balance in covariates, but quite bad imbalances in the propensity score. It also reduces the number of matrix manipulations by a few degrees of magnitude.

Austin’s final conclusion is that in most situations greedy calipers without replacement are best. There are some more subtle points made too, so if you are interested in propensity score matching, get a hold of this paper!