# A multi-institution evaluation of MLC log files and performance in IMRT delivery

- James R Kerns
^{1, 2, 4}Email author, - Nathan Childress
^{3}and - Stephen F Kry
^{1, 2, 4}

**9**:176

https://doi.org/10.1186/1748-717X-9-176

© Kerns et al.; licensee BioMed Central Ltd. 2014

**Received: **10 February 2014

**Accepted: **21 July 2014

**Published: **11 August 2014

## Abstract

### Background

The multileaf collimator (MLC) is a critical component to accurate intensity-modulated radiotherapy (IMRT) delivery. This study examined MLC positional accuracy via MLC logs from Varian machines from six institutions and three delivery techniques to evaluate typical positional accuracy and treatment and mechanical parameters that affect accuracy. Typical accuracy achieved was compared against TG-142 recommendations for MLC performance; more appropriate recommendations are suggested.

### Methods

Over 85,000 Varian MLC treatment logs were collected from six institutions and analyzed with FractionCHECK. Data were binned according to institution and treatment type to determine overall root mean square (RMS) and 95^{th} percentile error values, and then to look for correlations between those errors and with mechanical and treatment parameters including mean and maximum leaf speed, gantry angle, beam-on time, mean leaf error, and number of segments.

### Results

Results of treatment logs found that leaf RMS error and 95^{th} percentile leaf error were consistent between institutions, but varied by treatment type. The step and shoot technique had very small errors: the mean RMS leaf error was 0.008 mm. For dynamic treatments the mean RMS leaf error was 0.32 mm, while volumetric-modulated arc treatment (VMAT) showed an RMS leaf error of 0.46 mm. Most MLC leaf errors were found to be well below TG-142 recommended tolerances. For the dynamic and VMAT techniques, the mean and maximum leaf speeds were significantly linked to the leaf RMS error. Additionally, for dynamic delivery, the mean leaf error was correlated with RMS error, whereas for VMAT the average gantry speed was correlated. For all treatments, the RMS error and the 95^{th} percentile leaf error were correlated.

### Conclusions

Restricting the maximum leaf speed can help improve MLC performance for dynamic and VMAT deliveries. Furthermore, the tolerances of leaf RMS and error counts for all treatment types should be tightened from the TG-142 values to make them more appropriate for clinical performance. Values of 1 mm for the 95^{th} percentile of leaf RMS error and 1.5 mm for the 95^{th} percentile leaf error are suggested as action levels for all treatment types.

## Keywords

## Background

Actual multileaf collimator (MLC) position data can be tracked throughout a treatment delivery in the form of MLC log files. This data is important because it gives a permanent record of the actual MLC positions throughout the course of any intensity-modulated treatment which can be analyzed later. These log files can reveal the extent of MLC positioning errors in specific treatments or used to provide pretreatment quality assurance (QA) [1–4]. If the data of errors is aggregated, the overall MLC performance of a machine or clinic can be determined for comparison or to develop a baseline.

AAPM TG-142 [5] recommendations state that a step and shoot and dynamic MLC test should be run annually to determine the maximum root mean square (RMS) error during delivery, as well as the 95^{th} percentile error of all leaf errors over the course of treatment; the tolerances given for both metrics is 3.5 mm. Given that TG-142 generally uses physical machine performance as the basis for recommendations, this study also studies machine performance to determine appropriate tolerances. Although there have been many studies of MLC log files, a study broadly comparing independent institutions for overall performance has not yet been done. A better understanding of how certain mechanical and treatment planning parameters impact the MLC performance could help physicists develop more robust treatment plans while not unnecessarily limiting planning techniques as well as allow comparison to field-wide standards.

In this work, aggregate MLC data were collected over numerous Varian (Varian Medical Systems, Palo Alto, CA) machines at several institutions for different treatment-delivery modalities to determine the actual MLC performance experienced in clinics. We reviewed tens of thousands of treatment logs to determine typical RMS error values for Varian linacs, compared actual RMS error with TG-142 MLC recommendations, determined whether values varied by institutions and between treatment types, and assessed what parameters contributed to RMS error. With this information, quality assurance tests could be better tailored to catch errors or trends as well as identify clues to understand why certain plans or delivery types have atypical RMS error. This aggregate data could also serve as a benchmark of MLC performance that institutions could use to decide whether treatments should be altered to reduce MLC error values or whether established criteria are too restrictive. In addition, these data could be used to set action levels, indicating when a treatment could be approaching a tolerance limit.

## Materials and methods

### MLC logs

Varian MLC logs used in this study are of the pre-TrueBeam format, known as “dynalog” files, and can be captured on most any Varian linac through the MLC-to-clinac communication terminal. Two ASCII files are generated at the completion of a treatment field delivery, one for the A and B MLC banks. These files contain mechanical information of the linac throughout the treatment delivery, captured in 50 ms intervals. At every instance of information capture, data on MLC actual and planned positions, gantry, jaw, and collimator positions and beam on/hold states are acquired.

**Log breakdown summary**

Institution | Total | Step & shoot | Dynamic | VMAT |
---|---|---|---|---|

A | 30747 | – | 30173 | 574 |

B | 25866 | 22865 | 3001 | – |

C | 5624 | – | 5624 | – |

D | 3850 | – | 3850 | – |

E | 3211 | – | 2590 | 621 |

F | 15912 | 15912 | – | – |

There are numerous pieces of data not recorded by the dynalogs, which limits the number of potential correlations, but this weakness is also an appreciable strength. The treatment planning system and version, linac type, age of the linac, MLC model, treatment site, record and verify system, dose rate, maintenance schedule, and many other parameters are not only not recorded but expected to differ at difference institutions. The drawbacks of this mean the data could be muddled; however, any conclusions and correlations drawn from such data would also be independent of these parameters.

### Data analysis

Examination of dynalog files for various other purposes has been done which also explain some of the terms, but because of the subtle differences in terminology it is helpful to define some of the terms used here. Leaf error is defined as the difference between the MLC (or collimator, gantry, etc) planned position and the actual position. These actual/planned differences are synonymous with error. Leaf RMS error is a single value that is the root mean square error of an individual leaf, taking into account the leaf error at every point over the course of the delivery. The direction of the error does not matter. This value is given in Varian’s Dynalog File Viewer application (DFV) for each leaf. Mean, or bank RMS error is the mean of the leaf RMS errors of a given bank (A or B) and treatment delivery; this is also given in DFV. The 95^{th} percentile error specifically references a single value drawn from the total list of errors captured in the file. The direction of the error does not matter. These data are binned in the DFV, but the exact 95^{th} percentile error value is not immediately available. Mean leaf error is the mean of the error of a given leaf over the course of the delivery, but takes into account the direction, or sign, of the error and is not in the DFV.

MLC RMS error and 95^{th} percentile error data were compared between institutions. For all subsequent analyses, the data were assimilated by delivery type to give the broadest and most representative RMS error values and treatment parameters.

Data were compiled and analyzed using FractionCHECK (Mobius Medical Systems LP, Houston, TX). All mathematical calculations (RMS error, 95^{th} percentile errors, etc.) were done using this software unless otherwise mentioned in the fashion mentioned above. Other parameters like leaf speed were calculated using moving averages by dividing the distance covered by the leaf in 20 capture intervals (20 intervals × 50 ms/interval = 1 s) to give distance per second. This was calculated at every capture interval and the average value was reported as the mean leaf speed. The largest distance per second value was the maximum leaf speed. Only leaves that moved during treatment were included in the analyses. Because treatment type is not recorded in the log file, logs were categorized into treatment type depending on leaf movement (static or dynamic) and gantry movement. As appropriate to the modality, the leaf or bank RMS error was compared to many mechanical parameters including the gantry angle, gantry speed, number of plan segments, number of beam holds during delivery, total beam-on time, and mean and maximum leaf speed during delivery, mean leaf error, and internal tolerance, where applicable. Because of the large amount of data extracted, MATLAB (v7.14, 2012a; The MathWorks, Natick, MA) scripts were used to assimilate the results from Fraction CHECK and to display the data.

For each treatment parameter analyzed a correlation coefficient between the parameter values and the RMS error was determined. Correlation coefficients between ±0.5 and ±1.0 were considered strong. A p-value of <0.05 was taken to denote statistical significance. To determine whether institutions differed from one another in RMS error or 95^{th} percentile error, statistical analysis was done using analysis of variance between institutions within each treatment delivery type.

## Results

### Institutional comparison

^{th}percentile error values were compared by institution and treatment type (Figure 1). The numbers of leaf RMS error data points by treatment modality were approximately 321,000 for step-and-shoot, 1,152,000 for dynamic, and 28,500 for VMAT. Table 2 shows the mean leaf RMS error and the mean 95

^{th}percentile leaf error. The maximum RMS leaf errors are discussed in relation to TG-142 in the Discussion section. The step and shoot method had far lower leaf RMS error values than the other treatment types. The RMS error means of the dynamic and VMAT methods were similar, although there were many exceptions in the dynamic treatments but none were greater than 0.5 mm.

**Summary of log file error results sorted by treatment type**

Modality | Institution | Mean leaf RMS error (mm) | Mean 95th percentile error (mm) |
---|---|---|---|

| B | 0.004 | 0.020 |

F | 0.011 | 0.020 | |

Overall | 0.008 | 0.020 | |

| A | 0.37 | 0.69 |

B | 0.25 | 0.61 | |

C | 0.22 | 0.45 | |

D | 0.22 | 0.47 | |

E | 0.30 | 0.65 | |

Overall | 0.32 | 0.64 | |

| A | 0.46 | 0.96 |

E | 0.46 | 0.94 | |

Overall | 0.46 | 0.95 |

With the exception of the VMAT data, the differences in average leaf RMS error between institutions were statistically significant. These differences may reflect differences in machine maintenance or plan difficulty. However, these differences were small, and are likely only significant because of the large volume of data analyzed (>1M points). For the step-and-shoot technique, the two institutions’ average leaf RMS error differed by 0.007 mm. For the dynamic method, the largest difference in mean leaf RMS error between any two institutions was 0.15 mm, and the largest difference in 95^{th} percentile error was 0.24 mm. These differences are unlikely to affect clinical dose delivery.

### Step-and-shoot

Figure 2(D) plots the bank RMS error versus the 95^{th} percentile error count of the leaf deviations, which compares the two TG-142 parameters with each other. A strong correlation between these parameters would indicate that they are linked and therefore that only one parameter would be necessary to assess MLC performance. The parameters are in fact strongly and statistically correlated. This suggests that only one of these two parameters may be necessary to sufficiently analyze step-and-shoot MLC performance.Figure 2(E) plots the individual leaf RMS error versus the mean leaf error for each bank. Since mean leaf error is signed it indicates the direction of the error. A mean leaf error of 0 indicates an even division between overshooting and undershooting the planned leaf position, while the opposite case, a linear correlation between RMS and mean leaf error would indicate a leaf steadily over or undershooting throughout the delivery. The mean leaf error data did not follow any specific pattern, being randomly distributed and having data evenly split on either side of 0 mm.

### Dynamic

The bank RMS error correlated strongly and significantly with the 95^{th} percentile error (Figure 3[G]). These data suggest that when the mean RMS error of a treatment is high, the 95^{th} percentile error is very likely also high. This can be interpreted to mean that individual leaf performance does not vary substantially within the bank, otherwise the 95^{th} percentile error would show more spread amongst the data owing to underperforming leaves.The individual leaf RMS error is plotted against the mean leaf error in Figure 3(H), with each bank plotted separately. The mean leaf error was split almost evenly in both directions, and correlated significantly with the leaf RMS error. The leaves of bank A consistently overshot the planned position across the treatment and thus had a positive mean leaf error, while bank B consistently undershot the planned position and thus had a negative mean leaf error. This type of systematic error may play a larger role dosimetrically than random errors, and large RMS errors may therefore be of particular concern for dynamic treatments.

#### VMAT

Bank RMS error clearly and significantly correlated with the 95^{th} percentile error (Figure 4E). Just as for the dynamic deliveries, all leaves of a bank perform approximately the same otherwise the 95^{th} percentile error would be higher or have more variation.Unlike that of the dynamic technique which showed a consistent correlation of leaves leading or lagging from their intended position, the distribution of mean leaf error for VMAT was clustered around 0 mm, and did not significantly correlate with the leaf RMS error (Figure 4F). These results indicate that the MLCs oscillated between overshooting and undershooting the planned position and that despite the fact that the leaf RMS error increased with the mean leaf speed, the net effect may have largely canceled out.

No holdoffs were issued by the clinac in any VMAT treatment. Therefore, this parameter was not studied.

## Discussion

Results of treatment logs from six institutions showed that RMS error and 95^{th} percentile leaf error were consistent between institutions, but varied by treatment type. On average, the step and shoot technique had very small errors, while VMAT had the largest errors, but only slightly more than the dynamic technique. MLC RMS error correlated with mechanical, treatment, and plan parameters. For the dynamic and VMAT techniques, the mean and maximum leaf speed was significantly linked to the leaf RMS error. For dynamic delivery, the mean leaf error correlated with leaf RMS error, whereas for VMAT the average gantry speed was also correlated. For all treatments, the RMS error and the 95^{th} percentile leaf error were correlated.

Parameters commonly thought to affect MLC performance were found to have no such effect including gantry angle (i.e. gravity), number of beam holdoffs, and number of segments. Similarly, treatments with numerous beam holdoffs would be expected to frequently approach the maximum error allowable as the leaves have reached tolerance, resulting in higher level of error overall, but this was not observed in the data. Finally, increases in leaf error due to an increasing number of field segments could be expected because a larger number of segments would be clinically associated with a larger number of small MU MLC shapes. However, the data for both step and shoot and dynamic delivery indicated that this parameter also did not affect the RMS error. Reasons for differences with previous studies may be an improvement in MLC controller performance and improved treatment planning algorithms.

Parameters that did affect MLC performance included the mean and maximum leaf speed, relevant to dynamic and VMAT treatments. As well, the dynamic data showed that the mean leaf error was systematically related to the RMS leaf error – leaves either systematically overshot or undershot their intended position. This relationship can be largely explained by the fact that dynamic treatments using Varian linacs and planned with the Eclipse treatment planning system move MLCs in only one direction. Another possible cause for this is lag between the MLC controller and the clinac [8]. If the lag observed is real, such a systematic leaf offset would result in a shift of the treatment fluence. This has the potential to have notable dosimetric effects as studies examining MLC shifts have shown that random shifts do not have a significant effect on dosimetry, whereas systematic shifts can [9–12]. Actual assessment of the dosimetric impact of the RMS errors seen here were not possible, and further study of this issue is warranted. Although specific recommendations are not given based on dosimetric outcomes, recommendations for better mechanical MLC performance can be made based on the present data.

As mentioned earlier, certain other parameters were not able to be analyzed, being both a strength and weakness. Dose rate in the clinical sense (600 MU/min, etc) is not recorded, but will affect the MLC speed and thus RMS error [13].

The recommendations of TG-142 are based largely on what the machine is able to do mechanically, not necessarily on its dosimetric impact and such are our recommendations here. Improved MLC performance can be most readily achieved by limiting the mean and/or maximum leaf speed. The machine dose rate and the internal tolerance were not able to be studied because it is either not recorded or did not vary in the log files, but they can have a significant effect on the performance. Restricting the internal tolerance of dynamic treatments to 1 mm limit the maximum leaf error, improving performance.

Evaluation of the RMS error and 95^{th} percentile error, for which TG-142 recommends tolerances of 3.5 mm, is critical to understanding the current performance level of MLCs. Noticeably, almost all data in Figure 1 are well below TG-142 criteria. In fact, only a handful (out of the ~85,000) in the dynamic modality are above the TG-142 tolerance value. Therefore, based on current performance, it would be prudent to implement more appropriate criteria for Varian machines.

^{th}percentile error greater than 2 mm. These data suggest that the TG-142 criteria may be unnecessarily loose for Varian machines.

**Log failure rates with various tolerances applied**

Leaf RMS error | 95th percentile error | |||||||||
---|---|---|---|---|---|---|---|---|---|---|

Tolerance (mm) | 0.5 | 1.0 | 1.5 | 2.0 | 3.5 | 0.5 | 1.0 | 1.5 | 2.0 | 3.5 |

| ||||||||||

Step & shoot | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

Dynamic | 14.02 | 0.12 | 0.06 | 0.03 | 0 | 67.98 | 8.16 | 0.13 | 0.06 | 0.05 |

VMAT | 49.30 | 0 | 0 | 0 | 0 | 85.75 | 61.44 | 0 | 0 | 0 |

In clinical practice, the leaf error results of MLC tests can differ greatly from test to test and from modality to modality, depending on the nature of the test, treatment site, plan complexity, and the average and/or maximum leaf speeds and should be accounted for when performing, for example, demanding MLC QA tests or complex treatment plans or when introducing a new treatment technique into the clinic. Because of this, the MLC RMS error of a strenuous test should not be assumed as what a typical treatment RMS error is. However, on the basis of the results of Table 3, a tolerance value of 1 mm is likely an appropriate criterion for the maximum RMS leaf error tolerance or action level, of all treatment techniques. Correspondingly, the 95^{th} percentile error threshold should likely be 1.5 mm. MLC leaf position errors above such a threshold do not necessarily indicate a dosimetric or calibration problem, but could indicate a difficult plan to deliver, pushing the MLCs to their limits, and can therefore serve as possible action levels.

Another point of interest is the correlation between bank RMS error and the 95^{th} percentile error. All treatment techniques were significantly correlated and generally had little spread in the data. It can be concluded from the results that a high leaf RMS error will not be unnoticed via the 95^{th} percentile error and vic versa, suggesting that only one metric may be needed for this part of MLC QA. What is also noteworthy about the correlation is the very comparable performance of MLCs, even among different institutions, presumably with different QA standards. Dirt and grease can affect performance, but the data presented here shows consistent performance [15]. It would therefore be unusual for one leaf to be noticeably slower than the rest of the MLC bank. A discrepancy between the expected values of the 95^{th} percentile error and the bank RMS would quickly reveal such a leaf rather than having to visually inspect the leaves as they move during tests (e.g. the picket fence test).

The number of metrics a physicist can use depends on the analytic power of the software they use. The most basic way of assessing the logs is through the Dynalog File Viewer, which explicitly gives the mean and maximum RMS error of the leaves and shows a graph of the leaf errors in bins, but does not explicitly give the 95^{th} percentile error. Homemade scripts can be made to assess the log files automatically and can show additional metrics; a graphical user interface is also available from the authors. Furthermore, commercial software as used in this study also shows more metrics and can be used for trend analysis.

## Conclusion

This study examined MLC log data from Varian linacs, specifically dynalogs, to determine typical MLC performance during treatments, which mechanical treatment factors influence MLC leaf error, and whether the tolerance values of MLC performance metrics given in TG-142 are appropriate. Between institutions, the typical leaf RMS error values and 95^{th} percentile error values were not substantially different from each other. Comparing treatment modalities, step and shoot had the lowest typical leaf RMS error at 0.008 mm, dynamic treatments had a value of 0.32 mm, and VMAT had 0.46 mm.

Most other mechanical treatment parameters, like gantry angle, number of beam holds, and number of segments did not correlate to an increase in RMS error. For both dynamic and VMAT treatments, the parameters found to affect leaf RMS error were mean and maximum leaf speed. Furthermore, for dynamic treatments the mean leaf error was also correlated. For VMAT treatments, the average gantry speed was also found to correlate with RMS error. Considering that physicists can usually adjust the maximum leaf speed parameter within their treatment planning system, reducing leaf RMS error is reasonably achievable if need be and would positively affect both dynamic and VMAT deliveries.

In examining the MLC performance and distribution of error in a large number of treatments across multiple clinics, we recommend for Varian machines to use tighter tolerances than TG-142 gives, as those metric values appear to be inappropriately loose compared to physical machine performance. Various tolerance values or action levels were derived from the large dataset (Table 3) to reflect real-world performance. A tolerance value of 1 mm is recommended either as the maximum leaf RMS error or an action level for investigation for all treatment techniques and 1.5 mm is recommended for the 95^{th} percentile error.

MLC logs continue to be a valuable piece of information to study to better understand individual treatments as well as assess overall performance. Additional treatment data will shed more light on the accuracy of these recommendations.

## Declarations

### Acknowledgements

This work was supported by Public Health Service Grant CA10953 awarded by the National Cancer Institute, United States Department of Health and Human Services.

## Authors’ Affiliations

## References

- Agnew CE, King RB, Hounsell AR, McGarry CK:
**Implementation of phantom-less IMRT delivery verification using Varian DynaLog files and R/V output.***Phys Med Biol*2012,**57**(21):6761-6777.View ArticlePubMedGoogle Scholar - Sun B, Rangaraj D, Boddu S, Goddu M, Yang D, Palaniswaamy G, Yaddanapudi S, Wooten O, Mutic S:
**“Evaluation of the efficiency and effectiveness of independent dose calculation followed by machine log file analysis against conventional measurement based IMRT QA,”.***J Appl Clin Med Phys Am Coll Med Phys*2012,**13**(5):3837.Google Scholar - Litzenberg DW, Moran JM, Fraass BA:
**Verification of dynamic and segmental IMRT delivery by dynamic log file analysis.***J Appl Clin Med Phys Am Coll Med Phys*2002,**3**(2):63-72.View ArticleGoogle Scholar - Luo W, Li J, Price RA Jr, Chen L, Yang J, Fan J, Chen Z, McNeeley S, Xu X, Ma C-M:
**Monte Carlo based IMRT dose verification using MLC log files and R/V outputs.***Med Phys*2006,**33**(7):2557-2564.View ArticlePubMedGoogle Scholar - Klein EE, Hanley J, Bayouth J, Yin F-F, Simon W, Dresser S, Serago C, Aguirre F, Ma L, Arjomandy B, Liu C, Sandin C, Holmes T, Task Group 142, American Association of Physicists in Medicine:
**Task group 142 report: quality assurance of medical accelerators.***Med Phys*2009,**36**(9):4197-4212.View ArticlePubMedGoogle Scholar - Stell AM, Li JG, Zeidan OA, Dempsey JF:
**An extensive log-file analysis of step-and-shoot intensity modulated radiation therapy segment delivery errors.***Med Phys*2004,**31**(6):1593-1602.View ArticlePubMedGoogle Scholar - Ling CC, Zhang P, Archambault Y, Bocanek J, Tang G, Losasso T:
**Commissioning and quality assurance of RapidArc radiotherapy delivery system.***Int J Radiat Oncol Biol Phys*2008,**72**(2):575-581.View ArticlePubMedGoogle Scholar - Litzenberg DW, Moran JM, Fraass BA:
**Incorporation of realistic delivery limitations into dynamic MLC treatment delivery.***Med Phys*2002,**29**(5):810-820.View ArticlePubMedGoogle Scholar - Rangel A, Dunscombe P:
**Tolerances on MLC leaf position accuracy for IMRT delivery with a dynamic MLC.***Med Phys*2009,**36**(7):3304-3309.View ArticlePubMedGoogle Scholar - Betzel GT, Yi BY, Niu Y, Yu CX:
**Is RapidArc more susceptible to delivery uncertainties than dynamic IMRT?***Med Phys*2012,**39**(10):5882-5890.PubMed CentralView ArticlePubMedGoogle Scholar - Mu G, Ludlum E, Xia P:
**Impact of MLC leaf position errors on simple and complex IMRT plans for head and neck cancer.***Phys Med Biol*2008,**53**(1):77-88.View ArticlePubMedGoogle Scholar - Bai S, Li G, Wang M, Jiang Q, Zhang Y, Wei Y:
**Effect of MLC leaf position, collimator rotation angle, and gantry rotation angle errors on intensity-modulated radiotherapy plans for nasopharyngeal carcinoma.***Med Dosim Off J Am Assoc Med Dosim*2013,**38**(2):143-147.Google Scholar - Zhang Y, Li Y, Xia H, Wang J:
**Impact of dose rates on the position accuracy of multi-leaf collimator.***Radiat Phys Chem*2012,**81**(12):1813-1816.View ArticleGoogle Scholar - Otto K:
**Volumetric modulated arc therapy: IMRT in a single gantry arc.***Med Phys*2008,**35**(1):310-317.View ArticlePubMedGoogle Scholar - Losasso T:
**IMRT delivery performance with a varian multileaf collimator.***Int J Radiat Oncol Biol Phys*2008,**71**(1):S85-S88.View ArticlePubMedGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.