Evaluation of an artificial intelligence algorithm (HeartAssistᵀᴹ) in the assessment of fetal cardiothoracic ratio: a prospective study

Introduction

Measurement of fetal cardiac size is of paramount importance in the study of the fetal thorax, and it has been applied in the identification of different cardiac and non-cardiac diseases [1, 2]. The fetal cardiothoracic (C/T) circumference ratio is a parameter more currently used in the assessment of fetal cardiac size – it is the ratio of the cardiac circumference (CC) to the thoracic circumference (TC) and is measured by fetal ultrasonography [3, 4].
However, its evaluation requires the manual measurement of both CC and TC. Manual fetal biometric measurement is an error-prone and time-consuming procedure. Also, it suffers from inter- and intra-sonographer variability [5, 6]. Therefore, there is an essential need for a robust and accurate method that measures the fetal cardiac parameters automatically. This methodology may improve the workflow and reduce user variability in measuring fetal cardiac variables.
We therefore tested a commercially available software package, HeartAssistTM, based on an artificial intelligence (AI) algorithm in the assessment of TC and CC circumferences and their ratio, to assess its applicability and reproducibility when compared to manual assessment.

Material and methods

Population

This was a prospective observational study considering singleton pregnancies attending, from 1 April to 30 June 2022, the antenatal clinic of the Department of Obstetrics and Gynaecology of the Università Roma Tor Vergata for second-trimester ultrasonographic examination at 19–24 weeks of gestation. Inclusion criteria were as follows: 1) certain gestational age assessed by crown-rump length measurement and 2) absence of associated fetal chromosomal, genetic, or structural anomalies. Exclusion criteria were as follows: 1) presence of maternal complications (hypertension, diabetes, autoimmune disease) and 2) loss at follow-up. The study was approved by our Institutional Ethics Board (RS 45.22, 29 March 2002), and all the included women signed an informed consent form.

Ultrasound examination

All ultrasonographic examinations were performed by 2 trained observers (ME P and P M), using a Hera W10 Ultrasonographic system (Samsung Ltd., Seoul, South Korea) equipped with a transabdominal volumetric probe.
All women underwent a detailed evaluation of fetal anatomy and growth according to our national guidelines. For the objective of the study an axial view of the fetal thorax and the level of the 4-chamber view was acquired during diastole using the cine loop function. The thoracic view was considered suitable for measurement when the whole thorax was seen on the screen, there was a complete rib on both sides of the thorax, no abdominal contents were in the frame, and a good 4-chamber view was visualized (Figure 1). All measurements were performed in diastole. First a copy of the thorax image was stored for later analysis and then the HeartAssistTM (Samsung, Seoul, South Korea) function was activated for the automatic calculation of TC and CC.
At the end of the examination 2 observers, blinded to each other and to the automatic evaluation, performed the measurements of TC and CC using AutoCAD (Autodesk, San Rafael, CA) software calibrated on the size markers of the ultrasonographic images. The time necessary to obtain each measurement was calculated with a digital chronometer and stored.
To evaluate the intra-observer variability one of the observers repeated the measurements twice with the AutoCAD, with an interval of at least one month from the first assessment, on 34 images randomly selected from the study group.

Statistical analysis

Continuous variables are expressed as mean and standard deviation or median and interquartile range according to their distribution. Categorical variables were expressed as number (n) and percentage (%). Differences were compared using analysis of variance for repeated measurements or Kruskal-Wallis test plus post-hoc test for multiple comparisons as appropriate. A comparison of the time necessary to perform the measurement between the 2 methods was performed by Mann-Whitney U test.
The agreement between (inter-observer) and among (intra-observer) the 2 examiners was quantified, calculating the intraclass correlation coefficient (ICC) and its 95% confidence interval (CI). Similarly, the inter-method agreement (manual vs. heart assist) was evaluated by ICC.
To assess systematic bias between measurements, differences between values were plotted against means of the measurements, and limits of agreement with 95% CIs of the lower and upper limits were calculated as described by Bland and Altman [7]. Statistical analysis was performed using SPSS 28 (SPSS Inc., Chicago, IL, USA) and MedCalc (MedCalc Software Ltd., Ostend, Belgium) software. A 2-tailed p < 0.05 was considered statistically significant.

Results

Fifty consecutive women were considered for this study, and successful measurements for TC, CC, and C/T were obtained in all the cases. Their general characteristics are shown in Table 1. The median values for the first operator, second operator, and heart assistances are presented in Table 2. There were no systematic differences between first, second, or third automatic measurements for TC (p = 0.23), CC (p =0.66) or CC/TC measurements (p = 0.93).
The ICC values between the 2 observes were 0.931 (0.879-0.961) for TC, 0.944 (95% CI: 0.9221-0.975) for CC, and 0.939 (0.924-0.974) for CC/TC. As shown in Bland and Altman plots, the difference between the 2 operators was 0.1 (95% CI: 1.1 - –1.1) for TC, –0.2 (95% CI: 1.8 - –1.6) for CC, and 0.03 (95% CI: 0.7 - –0.8) for the C/T ratio (Figure 2).
The intra-observer ICC resulted in 0.986 (0.975-992) for TC, 0.972 (95% CI: 0.952-0.98) for CC, and 0.975 (0.976-0.993) for CC/TC. The mean intra-observer differences were –0.03 (95% CI: 0.8 - –0.9) for TC, –0.02 (95% CI 1.1 - –1.3) for CC, and –0.08 (95% CI: 0.6 - –0.6) for the C/T ratio, as shown in Figure 3. The analysis of inter-method agreement of the mean between the 2 operators and heart assistance measurements resulted in ICC values of 0.929 (0.901-0.942) for CC, 0.933 (95% CI: 0.919-0.944) for TC, and 0.930 (0.919-0.947) for CC/TC. The differences were 0.11 for CC, 0.14 for TC, and –0.06 for the C/T ratio (Figure 4).
The time necessary was significantly lower (p < 0.0001) for the measurements obtained by heart assistant (22.5 IQR: 17.7-26.7) than with the manual method (89.2 IQR: 76.5-82) (Figure 5).

Discussion

Main findings

In this study we found that there is good intra- and inter-observer reproducibility in the measurement of thoracic and cardiac circumference measurements and C/T ratio. We also demonstrated that the same results may be obtained by using heart assist and artificial intelligence-based software. This indicates that manual and automatic methods might be used interchangeably. Furthermore, we demonstrated that the latter technique allows the measurement in a significant shorter time, which is more suitable for clinical practice

Strengths and limitations

The strength of this study is its prospective design based on consecutive pregnant women undergoing a second-trimester examination. Furthermore, the examiners performing the measurement were blinded to each other’s results.
The major limitation is that the analysis was applied only to normal fetal heart, and there is no account of whether the same performance may be obtained in the presence of cardiac anomalies. A second limitation is that the frame of the fetal thorax was obtained by expert sonographers, and we cannot exclude that the result may be different in the presence of less trained operators. Finally, the measurements were obtained only in the second trimester, and there is no account of the performance of heart assist earlier or later in pregnancy.

Comparison with existing literature

Although there is evidence that manual measurement of the C/T ratio is fairly reproducible, this is to the best of our knowledge the first test of the role of artificial intelligence-based software in measuring cardiac biometry. Automatic biometry has already been tested in the past to obtain standard fetal biometric variables [8-11] and for the study of the fetal central nervous system [12, 13], proving to be compatible with the routine ultrasonographic workflow. The study of fetal heart biometry is challenging because its size is affected by the cardiac cycle. The use of artificial intelligence-based methods may reduce these difficulties [14-16].

Conclusions

The application of heart assist software allows us to obtain an automatic measurement of the C/T ratio. We have demonstrated that this technique is reproducible and has the same accuracy as manual measurements. Given that this method is faster, it has the potential to become the preferred technique to obtain cardiac biometry.

Acknowledgments

The authors are grateful to the research midwifes Ms Rosy Barbosa and Paola Dei Giudici, and Oriana Marucci MD and Umberto Maria Romeo MD for their support in the study.

Conflict of interest

The authors declare no conflict of interest.

REFERENCES