External validation of commercial AI products

Kicky van Leeuwen and the AIR Working Group of Radboud University (Nijmegen, the Netherlands) have performed an external validation of commercial AI systems for bone age and lung nodules.

They invited three bone age AI vendors, and two of them accepted to join, including Visiana (BoneXpert). The paper concludes:

“For bone age prediction, no difference was observed in the performance of the two algorithms tested and human readers. It is conceivable that in the future, radiology departments will require vendors to participate in transparent and comparative
evaluations as a prerequisite for purchasing AI products. Similarly, health care insurers might base reimbursement decisions on performance comparisons.”

It is important to have independent validation of AI on data from new sites, and it is good to see that the results are in line with previous studies, thus reproducing the prior validation work of BoneXpert and demonstrating generalisability of the algorithm.

Read the open access paper here.