Validation of BoneXpert in India

Dr Anuradha V. Khadilkar’s research group in Pune, India, has published a study of BoneXpert bone age, reporting its accuracy and presenting reference curves for Indian children.

Study overview

The study recruited 920 normal children prospectively for a hand X-ray, covering the age range 2-17 years evenly.

BoneXpert determined the main bone age – computed from the 21 tubular bones – for all images, while the carpal bone age was determined in the lower half of the age range.

Four paediatric endocrinologists rated the images according to Greulich and Pyle, and the average was used as reference.

BoneXpert accuracy

A priori, it was unknown how much weight the raters assigned to the carpals, so the study defined a composite bone age as a weighted average of BoneXpert’s tubular and carpal bone ages ­– if carpal bone was not reported by BoneXpert, the composite bone age was set to the tubular bone age.

The study found that a weight of 50% on the carpals gave the best match to the bone age reference. This suggests that the paediatric endocrinologist places 50% weight on the carpals in the lower age range. This is a remarkable result, because such a large weight has not been observed in studies in Europe and USA.

The difference between the BoneXpert composite bone age and the reference is shown against their average in this plot:

 

BoneXpert deviated in only 6.7% of the cases by more than 1 y (represented by the solid lines), and the Root Mean Square (RMS) error was 0.54 y.

To interpret this result, it is useful to introduce the concept of “true bone age” of an X-ray, defined as the average of vey many manual ratings of the image.

Assuming a SD of human ratings of 0.63 y, and assuming that the rater errors were independent, the reference had an RMS error relative to the true rating of 0.315 y.

This allows estimating BoneXpert’s error relative to the true rating as:

RMS error of BoneXpert = sqrt(0.54² – 0.315²) y = 0.44 y.

This is about the same as the error of the average of two manual ratings

RMS error of two manual raters = 0.63 y/sqrt(2) = 0.45 y.

In other words, BoneXpert is equivalent to double manual rating.

Carpal bone age

The modelling of the human rating as a mixture of tubular and carpal bone age is important for this study not only because the large weight of the carpals, but also because carpals were remarkably delayed relative to the tubular bones in this Indian population.

This plot shows the average carpal bone age minus tubular bone age versus age.

In the age range 2-6 years, female carpals were 1 y delayed, while male carpals were 1.5 y delayed, relative to tubular bone age. Such a large delay would perhaps not have been surprising for children with pathologies, where carpals are known to be sometimes delayed, but these data are for normal children, so the delay is quite remarkable. It has not been seen in normal Caucasian children.

This means that using a large weight on carpals will lower the average bone age considerably.

Resolving rater variability

The fact that users of the GP method are free to choose what weight they place on the carpals is thus a significant source of human rater variability.

BoneXpert clears up this ambiguity by reporting tubular and carpal bone age as two separate measurements.

For adult height prediction, the tubular bone age (which is simply called “bone age” in BoneXpert) is considered the optimal.

Bone age reference curves

The final contribution of the paper exploits that these data are perfectly representative of the local Indian population, so a bone age reference curve can be derived.

The figure below shows the average bone age minus age versus age for the two genders.

Up to age 12 y, the bone age agrees well with age, on average, while for ages 12-15 y, girls are slightly advanced and boys slightly delayed.

The drop in bone age – age after age 15 y for girls and after 17 y for boys should not be taken at face value – it is an artefact of reaching the ceiling bone age in the GP method, so one should not even draw the curves up here.

Summary

The study can be summarised as follows:

• The study is strong, because it is prospective, it is large, and it has a wide age coverage
• The human raters seem to place 50% weight on carpals in the lower half of the age range
• The accuracy of BoneXpert is equivalent to the average of two manual ratings
• Carpal bone age is delayed by 1 y in girls and 1.5 y in boys for age 2-6 y
• The bone age reference curves show that age and bone age are on average equal – except at ages 12-15, where girls are advanced and boys delayed by up to 6 months

 

The abstract of the paper is available here.

Contact for free trial of BoneXpert

BoneXpert is available for clinical use in India. For a free trial, please get in touch with Sales director Matthew Coakley, who recently introduced BoneXpert at Ruby Hall Clinic in Pune.