Journal of Medical Physics
ORIGINAL ARTICLE
Year
: 2011  |  Volume : 36  |  Issue : 3  |  Page : 138--146

Performance evaluation of three computed radiography systems using methods recommended in American Association of Physicists in Medicine Report 93


Wilbroad Muhogora1, Renato Padovani2, Faustino Bonutti2, Peter Msaki3, R Kazema4,  
1 Tanzania Atomic Energy Commission, Arusha, Tanzania
2 Fisica Sanitaria, Ospedale Universitario, P.le Santa Maria della Misericordia 15, 33100, Udine, Italy
3 Department of Physics, University of Dar es Salaam, P.O. Box 35063 Dar es Salaam, Tanzania
4 Muhimbili University of Health and Allied Sciences, Dar es Salaam, Tanzania

Correspondence Address:
Wilbroad Muhogora
Tanzania Atomic Energy Commission, P. O. Box 743 Arusha
Tanzania

Abstract

The performances of three clinical computed radiography (CR) systems, (Agfa CR 75 (with CRMD 4.0 image plates), Kodak CR 850 (with Kodak GP plates) and Kodak CR 850A (with Kodak GP plates)) were evaluated using six tests recommended in American Association of Physicists in Medicine Report 93. The results indicated variable performances with majority being within acceptable limits. The variations were mainly attributed to differences in detector formulations, plate readers«SQ» characteristics, and aging effects. The differences of the mean low contrast scores between the imaging systems for three observers were statistically significant for Agfa and Kodak CR 850A (P=0.009) and for Kodak CR systems (P=0.006) probably because of the differences in ages. However, the differences were not statistically significant between Agfa and Kodak CR 850 (P=0.284) suggesting similar perceived image quality. The study demonstrates the need to implement quality control program regularly.



How to cite this article:
Muhogora W, Padovani R, Bonutti F, Msaki P, Kazema R. Performance evaluation of three computed radiography systems using methods recommended in American Association of Physicists in Medicine Report 93.J Med Phys 2011;36:138-146


How to cite this URL:
Muhogora W, Padovani R, Bonutti F, Msaki P, Kazema R. Performance evaluation of three computed radiography systems using methods recommended in American Association of Physicists in Medicine Report 93. J Med Phys [serial online] 2011 [cited 2019 Oct 16 ];36:138-146
Available from: http://www.jmp.org.in/text.asp?2011/36/3/138/83478


Full Text

 Introduction



Computed radiography (CR) is now a firmly established digital imaging modality along with digital radiography (DR) systems. [1],[2] The advantages of CR and DR systems over screen-film-based technology are well acknowledged. [1],[2],[3],[4] In DR systems, the image is obtained immediately after the x-ray examination of the patient in real-time manner, which expedite the patient workflow. The DR systems are also known for their high image quality images at low patient doses. [1] For CR systems, the image plate (IP) must first be read using the laser scanner before the image is obtained. In developing countries or small radiology departments, the CR technology can be of first choice than the DR technology because of low running costs. In order to realize the digital benefits fully, it is necessary to perform the acceptance tests for quality assurance purposes. Such accepting testing methods for CR systems have been developed [1],[5] The objective of this study was to evaluate the performance status of three clinical CR systems using six tests recommended in American Association of Physicists in Medicine (AAPM) Report 93. [1]

 Materials and Methods



Computed radiography systems

The experimental setup during IP exposure for all CR systems is shown in [Figure 1]. The Agfa CR 75 system was operated with Philips Bucky Diagnost radiographic x-ray equipment (Philips Medical Systems, Hamburg, Germany) and was used with Agfa CRMD 4.0 IPs (Gaevart AG, Belgium). The Kodak Direct View CR 850 system (hereinafter referred to as Kodak 1) was operated with Philips Super 120 radiographic x-ray equipment (Philips Medical System: Hamburg, Germany) and was used with Kodak general plate (Kodak GP) (Eastman Kodak Company: Rochester, New York). The Kodak Direct View CR 850A system (hereinafter referred to as Kodak 2) was operated with Optimus, Philips Bucky Duo Diagnost radiographic x-ray equipment (Philips Medical System; Hamburg, Germany) and was used with Kodak general plates (Kodak GP)(Carestream Health TH, Inc., Rochester, New York, USA). All three x-ray equipment used in the study were being maintained according to the standard requirements. [6] {Figure 1}

Methods

Determination of incident air kerma

The incident air kerma values to plate were measured using a calibrated ionization chamber (RadCal, model 20X6-60 with electrometer model 2026C) (RadcalCorporation: Monrovia, CA, USA). The air kerma was measured at 50 cm focus-to-chamber distance and the corresponding incident air kerma to IP calculated by an inverse square law. Each incident air kerma on IP was corrected for "off axis variation." The determination of incident air kerma was repeated before the irradiation of each IP.

Determination of pixel value and pixel value standard deviation

For purposes of mean pixel value (PV) and the standard deviation of mean pixel value (PVSD) measurements, images were transferred to the table top computer and analyzed using Image J, a freeware image processing and analysis software available in the public domain ( http://rsb.info.nih.gov/ij/ ). [7],[8] The system-indicated parameters, such as speed index (lgM), scanned average level (SAL), window, level and exposure index (EI), were read from the respective equipment consoles. The lgM, SAL, and EI were also calculated for comparisons with the systems' indicated values.

Performance test methods

Twelve tests are recommended for acceptance testing and quality control of CR systems in AAPM Report 93. In this work, five quantitative methods and one subjective method were applied. Quantitative tests included IP dark noise and uniformity, and dose (exposure) indicator calibration accuracy. Others were system linearity and auto-ranging response, evaluation of noise level erasure thoroughness. The advantage of the results from quantitative tests is the fact that objective comparisons can be done with other similar studies. In addition, the contrast detail analysis, which is one of the recommended subjective tests, was also included in the study because it is part of noise level evaluations. Moreover, the clinical image quality evaluation is eventually performed in a similar subjective way, which provides a link to clinical practice. Unless specified otherwise in the text, the evaluation was done using the recommended testing parameters and conditions. [1] The tests evaluated are briefly described in the next sections.

Imaging plate dark noise and uniformity

The purpose of dark noise test was to assess the level of noise inherent in the system since excessive noise in the plates can compromise image quality. All IPs in the inventory for each CR system were erased to remove any residual PSL signals and exposed as recommended. Also dark noise uniformity was evaluated since a non-uniform response can affect the clinical image quality. The dark noise characteristics were evaluated according to the recommended test criteria. [1]

Dose (exposure) indicator calibration accuracy

The purpose of this test was to assess the accuracy of the plate exposure values calculated using dose (exposure) indicators. The dose indicator gives a surrogate measure of the PSP detector equivalent radiographic speed for a given exposure. This test was performed using three different sizes of IP by applying the tests recommended by manufacturers except the delay read out time, which varies between manufacturers. In this study a fixed delay time of 10 min was applied in order to reduce variation in phosphorescence lag. [1] In addition to this test, the comparison of the measured air kerma to plate and the calculated values was performed. However, such comparison is not recommended in AAPM report 93 under the test of exposure (dose) indicator calibration accuracy.

System linearity and auto-ranging response

The purpose of this test was to determine the response of the detector and read-out systems to verify the linearity at least to three decades of exposure variation (1000 times difference). For Agfa plates, slopes of linear fits to log (SAL) versus log air kerma, log (E), PV versus log (E) and lgM versus log (E) should result in straight lines, with correlation coefficients (CCs) > 0.95. Also the slope lgM -1< ±0.1, slope SAL /0.5-1 <±0.1 and slope PV /1250-1 <±0.1 requirements should be fulfilled. For Kodak GP plates, slopes should be such that slope EI /1000-1< ±0.1 and slope PV /1000-1 <±0.1; the plots of EI and PV versus air kerma on a linear-log scale should result in straight lines with CCs > 0.95. In addition to this evaluation, the values of the exposure indicator were calculated for each IP type and compared to the measured values as recommended. [1]

Evaluation of noise level and contrast detail analysis

The purpose of this test was to estimate the noise properties of CR systems and to monitor the image quality by assessing the visibility of low contrast details. The contrast resolution should be limited by quantum statistics (random variations in the number of x-rays absorbed in the IP) in a well-designed system. [1] For this evaluation, the TORRCDR test object (University of Leeds, Leeds) was used during image acquisitions as recommended. [1] The images were transferred to the workstation and the contrast detail threshold scored by three experienced persons on display monitor, type BARCO MED E-3621 (BARCO VIEW, Belgium), which is used for presentation. In addition to contrast detail scores, the pixel values (PVs) and PVSDs were measured within a fixed small region of the images using the image J software. The PVSDs were plotted against log (air kerma) and analyzed as recommended. [1]

Erasure thoroughness

The purpose of this test was to evaluate the ability of the read-out-erase cycle in removing ghosting artifacts under severe exposure conditions. This is important in order to assure that the IP can be re-used without residual signals from previous exposures. A 10-cm diameter and 3-cm thick lead block of about 1 kg was placed on each IP type with light field margins as in [Figure 1]b. Each IP was exposed to 438 μGy by using 60 kVp x-rays with no added filtration at appropriate SID. The results were analyzed as recommended. [1]

Data analysis

The differences of the mean low contrast scores.i.e. for 11 mm detail sizes between the imaging systems for three observers were tested for significance using the Mann-Whitney test software ( http://faculty.vassar.edu/lowry/utest.html ). The difference was considered significant if the probability P was less than 0.05 at 95% confidence level.

 Results



Dark noise and uniformity response

The evaluations of dark noise and uniformity response for studied CR systems are presented in [Table 1], [Table 2] and [Table 3]. It can be seen that the results were well below the test criteria for Agfa CR system [Table 1]. For Kodak CR system, the EI results were also adequate [Table 2] and [Table 3]. In contrast, the PVs for Kodak CR systems appeared to be higher than the recommended limiting values. However, if the definition of EI [1] is considered, an offset PV value of 2000 is ought to be subtracted from the determined mean PV value and hence compliant with test criteria. The PVSD values for dark noise test were also observed to be higher than the recommended test criteria. This is probably attributed to the influence of PV offset effect and intrinsic detector characteristics such as noise, sensitivity, etc, which vary from one plate to another. [9],[10],[11],[12] On the uniformity response, it can be seen that the EISD of Kodak 1 CR system met the test criterion [Table 1] while the compliance of EISD of the Kodak 2 CR system and the PVSD values for both systems were less satisfactory. The same reasons given for Kodak 1 CR system with regards to high PVSDs also applies for Kodak 2 CR system. {Table 1}{Table 2}{Table 3}

Dose indicator calibration accuracy

The results of exposure indicator accuracy test are presented in [Table 4], [Table 5] and [Table 6]. For the Agfa CR system, both the normalized lgM and normalized SAL values were lower than expected and therefore non-compliant with the test criteria, whether single screen or for all screens averaged [Table 5]. Also shown in [Table 4] is the comparison of calculated against measured incident air kerma to each plate, which shows systematic lower calculated than measured values. This is an indication that the original tuning of the CR system had changed.{Table 4}{Table 5}{Table 6}

The service vendor was informed of this discrepancy for corrective actions. The exposure (dose) indicator accuracy is known to be mainly influenced by many factors. [1] However, the most relevant factors that could have influenced the results are likely to be the beam quality and SID, which were different from the recommended parameters because of facility limitation. In addition, less realization of the recommended air kerma of 8.76 μGy to IP due to the selection limitation of mAs setting is likely to be another possible explanation. For Kodak 1 CR system, the normalized EI values were lower than expected except one plate for single screen test criteria [Table 5]. For all screens averaged, the mean value of normalized EI for this system was within the ±20% limit. However, varied results between the measured and calculated values of incident kerma plate air kerma values were observed for this system. For Kodak 2 CR system, all normalized EI values met the test criteria for all screens averaged but the value of one plate was higher than the test criteria for single screen [Table 6]. The calculated values of incident air kerma to plate and the measured values were also complied with the test criteria. The performance difference between the two Kodak CR systems can be due to varying IP characteristics [11],[12],[13],[14],[15] and degradations related to aging effects. This view appears to be valid taking into account that the Kodak 2 was relatively newer than the Kodak 1 CR system. System linearity and auto-ranging response

[Figure 2] shows the linearity and auto-ranging properties for the Agfa CR system. The comparison of the calculated to the measured air kerma values during this test is presented in [Table 7]. {Figure 2}{Table 7}

It can be seen that the CCs were adequate as per test criteria, i.e. R 2 > 0.95 [Figure 2]. The expected slopes of 1, 0.5, and 1250 for plots of lgM versus log K, log SAL versus log K and PV versus log K respectively were also adequately met. This implies that no other noise sources were interfering with the quantum-limited operation of CR system. [1],[9],[12],[15] The results presented in [Table 7] shows systematic lower calculated incident air kerma to the plate than to the measured values as previously observed in [Table 4]. The results of linearity and auto-ranging characteristics for Kodak CR systems are presented in [Figure 3] and [Figure 4]. The test criteria (slopes and CCs squared (R 2 ) of 1000 and R 2 > 0.95) in relevant plots were adequately achieved for Kodak CR systems. The expected slope was 1000 for EI versus log K graph and PV versus log K while the R 2 > 0.95 for the graphs as required. [Table 8] and [Table 9] show varied results between the calculated and measured values of incident air kerma to plate for both CR systems, which is mainly attributed to individual system's characteristics.{Figure 3}{Figure 4}{Table 8}{Table 9}

Noise and low contrast resolution

The noise characteristics for Agfa and Kodak CR systems are presented in [Figure 5]. The CR systems exhibited the expected performance as the quantum noise should ideally decrease with increasing air kerma to plate. [1],[12] With regards to the CCs, all values were above 0.95 as required by test criteria except in one situation. {Figure 5}

[Figure 6] presents the contrast detail results of the studied CR systems. The inter-observer variations are indicated as the standard deviations of the means of individual scores. The differences of the mean low contrast scores between the imaging systems were statistically significant for Agfa and Kodak CR 850A at 95% confidence level (P=0.009) and for Kodak CR systems (P=0.006). This is probably due to the fact that CR systems had different ages in operation. However, the differences were not statistically significant between Agfa and Kodak CR 850 (P=0.284) suggesting similar perceived image quality because of similar ages. The number of visible details (NVD) varied depending on the type of CR system, IP type, IP size, or detail size. Generally, NVD increased with plate exposure above 10 μGy for most CR systems, plate sizes, and detail sizes, with some exceptions. This was expected since contrast sensitivity should improve with increased exposure in a quantum limited operation of CR system. [1] Exceptions are likely to be attributed to intrinsic variations in IP readers, IP sensitivities as also experienced elsewhere. [11],[16] The inherent subjectivity in the contrast detail scoring methodology can be another possible explanation. [16] The results suggest that the plate exposures beyond 10 μGy do not improve the detail visibility and hence the potential for patient dose reduction if such exposures are avoided.{Figure 6}

Erasure thoroughness

The results of the erasure thoroughness test for three CR systems are presented in [Table 10]. Except PVs and PVSDs of Kodak CR systems (whose explanation has been already given), the results demonstrated adequate erasure performance. {Table 10}

 Discussion



The evaluation of the performances of CR systems is one of the important aspects of quality assurance in diagnostic radiology. This may take different forms such as acceptance testing, constancy testing, clinical testing, or optimization. [15] In this study, the performances of three CR systems have been evaluated in order to establish the actual status for constancy testing purposes. Despite this evaluation, the choice of the suitability of particular CR system over another is beyond the scope of this work as extensive tests including physical image quality metrics are necessary before such conclusion is made. Varied performances between the clinical CR systems have been attributed to differences in detector formulations, plate readers' characteristics, or ageing effects.

There are limited studies in the literature for direct comparison with the present results because of differences in methods used or type of CR systems tested. The Medicine and Healthcare Products Regulatory Agency (MHRA) in collaboration with other organizations in the United Kingdom (UK) periodically evaluate the performances of new CR systems in the market. [17],[18] In such assessment, the performance specification data of individual CR system or a group of CR systems is evaluated to provide technical information to prospective buyers. There is scanty information on the evaluations of CR systems based on AAPM Report 93 methodology. One study [19] evaluated the performance of CR systems using a method similar to that of AAPM Report 93 but the majority tests were subjective tests. In this study, one quantitative test of exposure indicator calibration accuracy was performed. The values for this test varied from 6% to 58% for Kodak CR 400 and from -3% to 22% for Agfa Diagnostic Centre (ADC) compact. In the present study, the exposure indicator calibration accuracy varied from -34.5% to -39.7% (Agfa CR) [Table 4] and from -18% to 13.5% (Kodak 1 and Kodak 2) [Table 5] and [Table 6]. Therefore the results of the two studies are nearly similar for Agfa CR systems but different for Kodak CR systems. The differences for the Kodak CR systems are probably due to technological improvements in the present imaging systems.

During this study, three main aspects were experienced and are worthy discussing. First, it was difficult to confirm the status of raw images even though the protocol identifies the test conditions that usually lead to raw data. The pixel spacing on DICOM header remained constantly independent of any exposure conditions suggesting that post processing existed to some extent. Second, in some installations, it was not possible to meet all test conditions such as the 180 cm SID setup, beam quality, beam intensity, level setting of "4096-EI" in or the "default or equivalent to 1 log(E)" window setting noise measurement for Kodak CR systems. For example, attempts to set the recommended level "4096-EI" resulted to dark image while "default" window setting retained PVs much higher than the test criteria. The setting of 1 log(E) window could not be done as related information was not available in user operational manual Third, the usefulness of excel spreadsheet for acceptance testing of CR systems developed by Ehsan Samei of Duke University ( http://www.toodoc.com/Ehsan-excel.html ) is part of the positive experience from this study. The use of worksheets minimized errors and formed an important self-audit check mechanism to achieve desirable quality evaluations.

 Conclusions



The overall functioning of CR systems involves complex mechanisms that necessitate regular quality control (QC). [20],[21] In this study, the QC tests have been performed for three CR systems in clinical practice using six methods recommended in AAPM Report 93. The results have indicated varied performances between the CR systems. However, comparable contrast detail scores were achieved for two CR systems, implying similar perceived image quality of test images. The non-compliance with performance criteria in some situations has mainly been attributed to different detector formulations, plate readers' characteristics as well as aging effects. The study has demonstrated the need to implement QC program regularly.

 Acknowledgements



The work was supported by joint program of the International Centre for Theoretical Physics and the International Atomic Energy Agency (421.IAEA.02G) as well as the project on Safety and Efficacies for New Technology and Imaging using New equipment to Support European Legislation (SENTINEL FP-012909).The permission granted by the authority of the University Hospital (Udine) to undertake this study using hospital facilities is specially acknowledged. The technical assistance of M. Florean and E. Cragnolini on computed radiography facility is also appreciated.

References

1American Association of Physicists in Medicine. Acceptance testing and quality control of photostimulable storage phosphor imaging systems, AAPM Report No. 93, Report of AAPM Task Group 10, AAPM 2006.
2Rowlands JA. The physics of computed radiography. Phys Med Biol 2002;47:R123-66.
3Charnock P, Connolly PA, Hughes D, Moores BM. Evaluation and testing of computed radiography systems. Radiat Prot Dosim 2005;114:201-7.
4Walsh C, Gorman D, Byrne P, Larkin A, Dowling A, Malone JF. Quality assurance of computed and digital radiography systems. Radiat Prot Dosim 2008;129:271-5.
5King′s Centre for the Assessment of Radiological Equipment (KCARE), "Protocol for the QA of computed radiography systems, commissioning and annual QA tests". London, UK: KCARE; 2005. Available from: http://www.kcare.co.uk/Education/protocols.htm [Last accessed on 2009 15 Aug].
6Institute of Physics and Engineering in Medicine (IPEM), "Recommended standards for routine performance testing of diagnostic X-ray imaging systems", IPEM Report 91, York, UK 2005.
7Rasband WS. Image J. U. S. National Institutes of Health, Bethesda, Maryland, USA, Available from: http://rsb.info.nih.gov/ij/, [Last accessed on 1997-2006].
8Abramoff MD, Magelhaes PJ, Ram SJ. Image Processing with Image. J Biophotonics Int 2004;11:36-42.
9Bradford DC, Peppler WP, Dobbins 3 rd JT. Performance characteristics of a Kodak computed radiography system. Med Phys 1999;26:27-31.
10Fetterly KA, Hanqiandreou NJ. The effects of x-ray spectra on the DQE of computed radiography system. Med Phys 2001;28:241-9.
11Fauber TL, Legg JS, Quinn M. Variation in CR imaging plate readers. Radiol Technol 2002;74:15-23.
12Flynn MJ, Samei E. Experimental comparison of noise and resolution for 2k and 4k storage phosphor radiography systems. Med Phys1999;26:1612-23.
13Nakano Y, Girdo T, Honda S, Akihiro M, Wakamatsu H, Yanagita T. Improved computed radiography image quality from BaFI: Eu photostimulable phosphor plate. Med Phys 2002;29:592-7.
14Van Metter R, YorkSton Y. Factors influencing image quality in digital radiographic systems, Proc SPIE 2001;4320:244-56.
15Tapiovaara M. Relationships between physical measurements and user evaluation of image quality in medical radiology-A review. Radiation and Nuclear Safety Authority (STUK), Helsinki, Finland, A 219, 2006.
16Lu ZF, Nickoloff EL, So JC, Dutta AK. Comparison of computed radiography and film/screen combination using a contrast-detail phantom. J Appl Clin Med Phys 2003;4:91-8.
17Medicines and Healthcare Products Regulatory Agency. Comparative specifications of computed radiography systems for general radiography, 2 nd ed. United Kingdom: MHRA 03140; 2003.
18Centre for Evidence Based Radiology. Computed radiography system for general radiography, Agfa Healthcare, Report number 06004, United Kingdom: 2006.
19Tucker JE, Contreras M, Wider RJ, Radvany MG, Chacko AK, Shah RB. Photostimulable storage phosphor image acquisition: Evaluation of three commercially available state-of-the-art systems. J Digit Imaging 1999;12:54-8.
20Krupinski EA, Jiang Y. Anniversary paper: Evaluation of medical imaging systems. Med Phys 2008;35:645-59.
21Gur D. Imaging technology and practice assessment studies: Importance of the baseline or reference performance level. Radiology 2008;247:8-11.