New Study: BoneXpert bone age has a precision of 0.08 years

A new longitudinal study from Philadelphia finds that BoneXpert’s automated bone age has a precision (repeatability) of 0.08 years – far better than manual rating. The authors examine what this level of precision could mean when a child’s bone age is followed over time.

The first question you ask of any automated bone age method is: how accurate is it? For BoneXpert, the answer is that the deviation from the average rating of six radiologists is 0.45 years (the root mean square deviation). Accuracy is what matters the first time a child is assessed – to establish whether bone age is delayed or advanced, and to predict adult height.

But there is a second property of bone age method, at least as important and often overlooked: how precise is it? That is, if a hand is X-rayed several times, what is the standard deviation (SD) of the automated ratings? Precision is what matters once a child is followed longitudinally, because then the quantity of interest is the change in bone age from one visit to the next.

As an example of where precision matters, consider a girl with precocious puberty, who is on a course of bone age advancing at, say, 1.5 years per year. Hormone therapy is often initiated, to slow down this progression, and hence one wants to know whether that rate has changed. Detecting such a change depends entirely on how precisely each reading can be made – so here it is precision, not accuracy, that sets the limit.

Manual rating has notoriously poor precision. When 12 experts rated the same images, the SD was 0.63 years, so the SD of a bone age increment (the difference of two readings) is √2 × 0.63 = 0.89 years. At that level, manual reading cannot reliably resolve year-to-year changes.

The new study – a placebo-controlled trial following 90 children with repeated hand X-rays – reports that automated bone age is markedly more precise than manual reading: BoneXpert’s bone age precision is 0.08 years. The SD of an increment is therefore √2 × 0.08 = 0.113 years, and the smallest change that can be detected with confidence is 0.23 years. At this level, an increment of 0.5 years is clearly distinguishable from one of 1.5 years. The authors discuss what this could mean for following children and adolescents over the course of growth-modulating treatment and stress the importance of using the same X-ray equipment for the baseline and the follow-up images.

Where does the high precision come from? Two things:

A) BoneXpert assesses each of the 21 tubular bones (radius, ulna, metacarpals, and phalanges) independently and averages the 21 results, thereby reducing the noise.

B) The averaging is standardised, with equal 1/21 weight on each bone. Methods, that interpret the hand as a whole, are prone to changing the relative weights of the bones between visits, which harms precision, because bone age varies slightly across the bones.

Thus, BoneXpert is designed to yield high precision, while modern automated bone age methods based on deep learning focusses solely on accuracy and never report precision, but there is one study that allows an estimate: It found an SD between and left- and right-hand bone age of 0.29 years, indicating a precision of 0.20 years – the analogous left-right analysis using BoneXpert yields 0.11 years, indicating that BoneXpert is about twice as precise as the deep learning method.