Purpose
Conformal radiotherapy techniques require accurate delineation of target volume for precise treatment delivery [1,2,3]. At a busy radiation therapy center, many oncologists and post-graduate trainees are collectively responsible for the care and timely treatment of patients. Despite uniform training and using contouring guidelines, inter-observer variability is known to exist [3].
This inter-observer variability in practicing radiation oncologists has been extensively reviewed in the recent past for many treatment sites [2,3]. In literature, various methods have been published for evaluation and reporting the inter-observer variability among different observers and for a variety of disease sites [4,5,6,7,8]. The introduction of various indices for quantitative dosimetric evaluation facilitate to accurately select the best plan [9,10,11]. Several studies have also investigated methods to evaluate the dosimetric impact of interobserver target delineation variability [12,13,14,15]. Few of these brachytherapy studies reported dosimetric impact by generating an optimized reference plan by reference contour and evaluation of various contours by observers [12,13]. A study on thoracic tumors treated with external beam radiation therapy evaluated the impact of different plans generated by observer’s target volumes on a reference target volume [14]. Another similar study on brain metastasis reported the impact of inter-observer contouring variability on dosimetry by creating plans for all target contours, and reported the differences in dosimetric indices when all plans were compared to all contours without generating a reference target contour [15]. Despite these studies, there is no known index to evaluate and quantify the dosimetric impact of inter-observer variability. Therefore, in our study, we proposed an index called “coverage with dosimetric concordance index” (CDCI), based on the results of our extensive work on inter-observer variability [16,17].
Material and methods
The most popular method to analyze the variability in target delineation is by computing Jaccard’s coefficient (spatial concordance) in delineations, which is the ratio of common volume to encompassing volume among observers [4,5]. As expected, this spatial concordance worsens with an increasing number of observers due to a decrease in common volume [6]. However, when a large number of observers participate in delineation, more realistic method of comparison may be warranted [6,7]. In the present work, we adopted these methods to conceptualize and propose two new indices – CDCIcommon and CDCIpair, which could be used for evaluating dosimetric impact of inter-observer variability in the target delineation.
CDCIcommon
The dosimetric impact of variability in target delineation (shown in Figure 1A for 3 observers) in observers can be determined by generating optimized conformal plans for the planning target volume (PTV) of each observer (Figure 1B, solid lines), ensuring consistency in the optimization process for all PTV’s. The common volume irradiated by prescription dose (Vref_common) in plans of all observers can be calculated as follows (Figure 1B, dotted white line):
where n is the number of delineations and Vref is the volume irradiated by prescription dose.
Fig. 1
A) Target delineation variability among 3 observers; B) The prescription isodoses for optimized plans made for the PTV of all observers and the common volume irradiated by prescription dose in white dotted line; C) The comparison of individual PTV with the common volume irradiated by prescription dose (white dotted line)
Dosimetric coverage of individual PTV with Vref_common was computed and represented as “coverage with dosimetric concordance” (CDCcommon) (Figure 1C):
where PTVVref_common is the volume of PTV covered with common volume irradiated by prescription dose, and VPTV is the volume of PTV.
CDCIcommon is the mean coverage of all PTV with Vref_common:
CDCIpair
As previously described, the common volume continues to shrink with an increasing number of observers and CDCI calculation for that common volume irradiated by prescription isodose represents the worst-case scenario. In order to distinguish the index as more meaningful and realistic, the dosimetric concordance between 2 observers (pair) can be computed. The same can be used for greater number of observers by computing dosimetric concordance for all the possible pairs (as explained in detail for 5 observers in the ‘Analysis’ section of the present study). This method is similar to the indices proposed for the measurement of similarity of target delineation, which is independent of the number of observers [6]. The total number of unique paired computations possible for n observers can be derived by the equation:
where nC2 is the combination of pairs of observers from a total of n observers, and
n! = n × (n – 1)× (n – 2) ×... ... ... × 3 × 2 × 1.
Dosimetric coverage of individual PTV with the common volume irradiated by prescription dose among respective pair can be calculated by the following equation:
where PTVVref_pair is the common volume irradiated by prescription isodose between the 2 plans made on the individual target contours for a pair of observers. CDC of this pair can be computed by evaluating the coverage of individual PTVs with the common volume of prescription isodose.
The CDCIpair is the mean coverage of all (CDC pairs) PTVs with common prescription dose volume with respective pairs, among all pairs (nC2), which can be represented as:
Theoretically, the CDCI is ‘0’ if there is complete discordance between the volumes of prescription isodoses. However, the CDCI value of ‘1’ represents the common volume irradiated by prescription dose involving delineated target volumes of all observers. Therefore, a CDCI value closer to unity is desirable.
Target delineation and treatment planning
For this study, we analyzed the data of our previously published work [16]. Twenty breast cancer patients, suitable for APBI, underwent flexible nylon tube interstitial implant using the open cavity technique. Axial computed tomography (CT) scans with 3 mm thickness were acquired on Somatom Emotion scanner (Siemens Healthcare, Erlangen, Germany) for planning. Inter-observer target delineation variability among five observers on the CT scan of twenty patients was investigated and reported previously [17]. One hundred optimized conformal plans were subsequently generated by a single planner on the clinical target volumes (CTV’s) delineated by five radiation oncologists (observers) on twenty patients treated with multi-catheter interstitial partial breast brachytherapy (MIB) in Oncentra brachytherapy planning (Oncentra v 4.3, Elekta AB, Stockholm, Sweden) [16,17]. For brachytherapy, CTV was considered as PTV. Each plan (A1) was graphically optimized to individual target volume contoured by an observer (A) to get maximum coverage of target volume with prescription dose. The goal of the treatment planning was to achieve target volume coverage of ≥ 80% and dose homogeneity index (DHI) ≥ 0.75 [16]. As the MIB was delivered within 3 days of lumpectomy at our institution, an underdose was observed and clinically accepted to the target volume in the region close to the skin with an air pocket. This was specifically performed to prevent high-dose radiation to the skin sutures and avoid producing wound complications. This clinical scenario has been reported in our previous publications [18,19].
Analysis
The CDCIcommon was computed using the equation (2). For the CDCIpair calculation in our study with 5 observers (A, B, C, D, and E), evaluations were made between 10 pairs: A & B, A & C, A & D, A & E, B & C, B & D, B & E, C & D, C & E, and D & E. CDC was calculated for each pair, where each common volume irradiated by prescription isodose was compared with respective PTV. For example, when evaluating A & B, plans were made for each of these PTV contours A1 and B1, and an intersection volume of the prescription isodose of the 2 plans was individually evaluated against the PTV from the pair of observers. Thereafter, the CDCIpair was computed using the equation (6). Both the indices (CDCIcommon and CDCIpair) were compared and analyzed for all the 20 study patients. Mean percentage differences between coverage index (CI, defined as the fraction of target volume receiving prescription dose) for each observer’s PTV with CDCIcommon (ΔCDCIcommon) and CDCIpair (ΔCDCIpair) were estimated for all 20 patients. Correlation between both the indices (CDCIcommon and CDCIpair) and between ΔCDCIcommon and ΔCDCIpair was analyzed.
Percentage differences (Δ) between the CI for an individual observer’s PTV (A) by their custom plan (A1) with the coverage using the common volume irradiated by prescription isodose [coverage of PTV (A) using common volume of A1∩B1∩C1∩D1∩E1 (Vref_common)] was calculated and expressed by ΔCDCcommon. Similar computation was performed for mean PTV dosimetric coverage with paired common prescription dose volume (Vref_pair) for individual observers PTV’s (ΔCDCpair). The impact of PTV volume on coverage with dosimetric concordance was analyzed by evaluating a correlation plot between relative PTV volume (RPV) with the ΔCDCcommon and ΔCDCpair, where RPV was determined by computing mean PTV of all patients for a specific observer normalized with the smallest PTV among observers. Statistical Package for Social Sciences (SPSS version 20.0, IBM, Chicago, USA) was used for statistical analysis. Pearson correlation was tested among CDCI, ΔCDCI, and ΔCDC.
Results
Figure 1A presents target delineation variability among 3 observers, and Figure 1B represents the prescription isodoses for optimized plans made for the PTV of all observers and the common volume irradiated by prescription dose in white dotted line. Figure 1C shows the comparison of individual PTV with the common volume irradiated by prescription dose (white dotted line); the Figure shows findings of only 3 of the 5 observers. The target delineation variability among the 5 observers showed a mean decline of 5.6 ±3.2% and 11.3 ±5.7% in CDCIpair and CDCIcommon, respectively, when compared to CI of PTV. Figure 2A describes a plot between both the indices for twenty cases. Both these indices showed linear relation to each other with a statistically significant correlation (r = 0.964, p < 0.000). The CDCIcommon ranged from 0.544 to 0.873, whereas that of CDCIpair ranged from 0.631 to 0.922. The ΔCDCIpair also increased linearly with an increase in the ΔCDCIcommon (Figure 2B) and statistically significant correlation (r = 0.962, p < 0.000).When comparing the CI of PTV, the maximum decline observed due to inter-observer variability in CDCIcommon and CDCIpair was 27.2% and 15.5%, respectively. Statistically significant correlation (r = 0.938, p < 0.018) was also observed between ΔCDCcommon and ΔCDCpair, respectively, when mean differences in CI for PTV’s of all observers for their individual treatment plan with the mean Vref_common and Vref_pair was analyzed (Figure 2C). ΔCDCcommon ranged between 8.1% and 13.1%; however, the ΔCDCpair showed variations between 4% and 8.4%. The impact of PTV volume on CDC is presented in Figure 2D. An increase in RPV showed an increase in both the indices. However, the steeper increase was observed for ΔCDCcommon, as compared to ΔCDCpair.
Fig. 2
Plot between A) Both CDCI indices for all twenty patients; B) % of ΔCDCIcommon and % of ΔCDCIpair for all twenty patients; C) % of ΔCDCcommon and % of ΔCDCpair for all five observers; D) RPV and % of ΔCDCcommon and % of ΔCDCpair for all five observers
Δ CDCIcommon– mean % of differences between CI for each observer’s PTV with CDCIcommon
Δ CDCIpair– mean % of differences between CI for each observer’s PTV with CDCIpair
% of Δ CDCcommon–% of difference between CI of PTV in their individual plan and dosimetric coverage of the PTV with common prescription volume
% of Δ CDCpair– % of difference between CI of PTV in their individual plan and dosimetric coverage of the PTV with paired prescription volume with respective pairs
Discussion
Target delineation is considered as the weakest link in the process of radiotherapy [1], and delineation errors are a potential source of uncertainties in delivering high precision radiotherapy [2]. There are few studies that have reported dosimetric impact due to the target delineation variability [12,13,14,15]. Among these, ours is the first study to propose an index, which can be used to report and compare the dosimetric impact of this inter-observer variability.
The review of literature revealed no standardized or uniform method of dosimetric comparison of target delineation variability [2]. Hellbust et al. and Kosztyla et al. used a method by generating a reference treatment plan optimized on reference contour [12,13]. Hellebust et al. [12] generated target contours with an expert consensus as well as by using an expectation-maximization algorithm for simultaneous truth and performance level estimation (STAPLE); however, Kosztyla et al. [13] utilized in-house MATLAB software to create a reference target contour. Reference plans were made on reference target contour and dosimetric comparisons of reference plan were made with the target contours of all observers individually to evaluate the dosimetric impact of delineation variability. Dewas et al. [14] created the optimized plans on target volumes of various observers and evaluated the dosimetric outcome on a reference target volume. Our study is unique in this aspect; we did not generate a reference plan but computed the dosimetric concordance with the common volume irradiated by prescription dose, and performed direct comparisons in pairs, which may be practically more relevant.
In a similar study, Stanley et al. [15] performed detailed analysis by creating plans for all target contours and comparison of all plans analyzed to all contours without having a reference target contour. In the present work, we proposed an index, and used the index to estimate the dosimetric impact of delineation variability.
Our study showed that there is a linear relationship between both the indices CDCIcommon and CDCIpair for multi-catheter interstitial breast brachytherapy. The difference (Δ) in CI of PTV with CDCI, which represents the decline of target coverage due to inter-observer delineation variability, was also found to have a linear relationship with both the indices. The ΔCDC tends to increase when the relative PTV volume (RPV) among observers increases. This is due to a decrease in the CI of PTV when the RPV increases, as shown in our previous publication [16]. Larger PTV volumes had poorer dosimetric coverage and therefore greater percentage reduction in both the indices with CI of PTV. As expected, CDCcommon also showed a larger percentage decrease when compared to CDCpair.
The proposed CDCI indices were conceptualized from the methods published for quantification of contouring variability among observers [4,5,6,7]. With Jaccard’s coefficient, the common volume becomes smaller for each addition of delineated volume [6]. Similarly, the CDCIcommon considers the common volume of prescription dose among all observers and reduces with the addition of more observers. This reduction in the common intersection volume is reflected in the reduction of numerical value of the indices. Therefore, CDCIcommon represents the coverage assuming the worst-case scenario. On the other hand, the CDCIpair is independent of the number of observers and therefore provides a more realistic evaluation of the dosimetric impact. The variation between the dosimetric coverage of the target volume (CI of PTV) in the treatment plan with CDCI (ΔCDCI) is the indicator of the dosimetric impact due to target delineation variability. Theoretically, the minimum value of CDCI is ‘0’, which indicates that there is no overlap of prescription isodoses. The maximum value of the CDCI is ‘1’, when common volume irradiated by prescription dose encompasses delineated target volumes of all observers. The limitation of the CDCI is that it may have a higher value, even though the target dosimetric conformality for each observer’s individual treatment plan may be inadequate for sufficient target coverage. Therefore, it is vital to evaluate target conformality along with CDCI. Another limitation of the CDCI index is that it does not consider the dosimetric impact of delineation variability in organ at risk (OAR). OAR nearer to the target volume may have the dosimetric impact of inter-observer variation of target and OAR delineation. Nelms et al. [20] proposed a scoring method to quantify the contouring variability of OAR’s, which incorporates the absolute degree of overlap along with the distance to agreement. However, they only quantified the dosimetric impact as differences of mean and maximum doses received by OAR’s.
In our study, the concept of CDCI was developed and evaluated only for 3D conformal brachytherapy of breast implants. Nevertheless, the concept of CDCI can also be utilized to conformal radiotherapy for any site.
Conclusions
The proposed indices can be used for evaluating the dosimetric impact of target delineation variability. Both the indices showed linear correlation with each other and the difference between the dosimetric coverage of target volume (CI of PTV) with CDCI (ΔCDCI) and can be used as an indicator of the dosimetric impact due to the target delineation variability for conformal brachytherapy.
