Comparison of an objective method of measuring bulbar redness to the use of traditional grading scales☆

1. Introduction and objectives 

The clinical judgment of ocular redness is complex and poorly understood. In clinical research and practice, grading scales are commonly used to categorize the severity and advancement of clinical conditions. It is important, in clinical decision-making, to use grading systems that possess both high discrimination and reliability, and are quick and simple to use.

While there is no standard grading scale amongst eye care professionals, a few scales are used on a regular basis in many parts of the world [1]. Typically, the eye is judged with the use of a scale, which implies measurement—although some scales are used descriptively. Some qualitative integer grading systems lack the sensitivity needed to detect clinically meaningful change [2]. The most widely adopted method of assigning a grade to a patient is through a comparison of the individual's eye and a set of illustrations or pictures typifying each grade [1], [3]. This method entails a judgment call on the part of the practitioner and often does not represent the full range of possible grades, because the range of assignable grades is limited by the number of pictures that can be conveniently displayed on a reference chart. These scales often have unevenly spaced reference benchmarks [4] and thus lack the standardized reference criteria that would increase inter- and intra-observer reliability [3], [5]. Given this subjectivity, there exists a lack of consistency in assigning these grades. Thus, grade assignments may vary substantially between practitioners. At present, the subjective classification of clinical signs observed with the use of a slit lamp, such as conjunctival hyperemia [2] and cataract [6], has been improved by the use of standardized photographic systems instead of the traditional integer scaling systems.

The current image-based grading uses either medical illustrations or actual pictures as a baseline. The resolution of these grades is limited by the number of pictures provided by the source material. Smaller scales (0–4 or 0–5) are more common because they are easier to display. As a result these grades are discrete, and each practitioner must discriminate between fractions of a grade (i.e., what constitutes a 2.5 versus a 2.25 on a 0–4 scale). One scale even suggests counting the number of filled blood vessels to enhance repeatability [7]. More recently practitioners have used visual analogue scales to rate handling of lenses [8] and have suggested that they may be a repeatable tool for measuring subjective responses. Little is known about how clinicians perform these types of judgments and with few exceptions, nothing is known about the performance of the scales used to assign clinical grades [9], [10].

There have been attempts to perform and evaluate clinical redness grading using automated methods [11], [12], [13], [14], [15], [16]. These typically involve examining the ocular surface in a particular area to determine the characteristics of the vessels [17], [18], [19] in order to measure local variations in luminance. The results of these studies were compared to traditional clinical grading in addition to being assessed for repeatability. However, the validity of this grading method has never been tested. Guillon and Shah [17] measured the number of blood vessels found at a fixed distance from the limbus, by assuming that the level of redness is proportional to the presence of blood vessels. However, very fine vessels in the sclera that are normally invisible can also become irritated and are not accounted for with this method of measurement. Other attempts have tried measures such as the normalized ratio of red to total intensity [19], width of blood vessel, number of vessels and proportion of vessel area [18] and total blood vessel area [17], [18].

Attempts have been made to create an automated grading system by morphing video clips showing the diminishing redness of an ocular reaction to create a continuous grading scale [2]. Another method in particular involved automated measurement of scales and image analysis [20]. This study developed an automated method of redness measurement by comparing redness and blood vessel appearance from several images to the redness values assigned by clinicians, on a 100-point scale, in a web survey. Although the methods used in these studies are still regarded as being highly variable, it was determined from the results, that the important characteristics in determining the severity of conjunctival redness are blood vessel area, overall redness intensity, vessel diameter, and tortuosity [21]. Since there are a large number of features that are taken into account with a clinician's grade of an image, it would be interesting to compare these with a photometric measure of redness [5], [10].

Simpson et al. [22] have developed a photometric method of objectively measuring bulbar conjunctival redness involving the measurement of chromaticity, a physical analogue of redness, greenness and blueness in the image using the Spectrascan650® Photometer [20]. Although other objective methods such as image analysis using edge detection [14], [15], [16], [22], [23] have been used, this paper investigates the correlation between this photometric method and the commonly used CCLRU scale.

The CCLRU scale has been reported to have high correlations between colouration image analysis and scale grades (r2=0.98) and the study investigators were experienced in its use [16].

Scientific investigations of treatments using contact lenses that cause very low amounts of hyperaemia would be improved by an objective and sensitive measure of hyperaemia. This study compares the clinical judgement of a single investigator using the photographic reference scale of ocular hyperaemia to a photometric technique that measures chromaticity of a distinct area of the conjunctiva. In a clinical research setting there is a need to better judge ocular redness on an objective and continuous scale.

A pilot study (Part 1) was performed to show that this method provides a repeatable and reliable measure of bulbar redness.

The purpose of the clinical trial (Part 2), was to analyze and compare two clinical grading systems: an objective system using photometric measures of redness and a subjective system based on a 0–100 grading scale. In addition to comparing the two grading methods, the redness over time while wearing contact lenses overnight was also assessed for those two methods.

2. Materials and methods 

3. Instrumentation 

The Spectrascan650® Photometer by Photoresearch® Inc. (Chatsworth, VA) was used to measure redness under fixed scotopic illumination.

For Part 1 of the study the instrument was mounted at a fixed distance of 30cm from a computer screen. Room conditions, screen settings and instrumental set-up were kept stable throughout the measurements. The images representing grades 1–4 of the CCLRU grading scale were displayed on the screen and separated into five regions of interest that covered the largest part of the conjunctiva possible (see Fig. 1).

View Large Image Download to PowerPoint

Fig. 1. Repeatability study set-up with designated areas on the bulbar conjunctiva.

For Part 2 of the study, the instrument was mounted on a base, and included a chin and forehead rest that was at a distance of 37cm from the eye. A photo (Fig. 2) of the set-up of the spectrophotometer is included. The measuring area of the spectrophotometer is a circular area with a diameter of approximately 5mm.

View Large Image Download to PowerPoint

Fig. 2. Spectrascan650® Photometer by Photoresearch® Inc. (Chatsworth, VA).

4. Participants 

Twenty-four subjects wearing a high-Dk silicone hydrogel lens in one eye for a 6-month period were studied. Lenses were replaced on a monthly basis. The subjects had no contact lens experience in the previous 5 years. Subjects included 18 females and six males ranging in age from 21 to 49 years and averaging 32.6±10.8 years.

5. Data collection 

5.3. Part 2: Subjective grading of bulbar redness on participants 

The investigator graded the nasal and temporal bulbar conjunctivae of each image on a 100-point scale using the CCLRU bulbar conjunctiva redness scale as reference [1] (see Fig. 3). Grading was derived from the overall impression of the anterior bulbar area. The participant was asked to look to the left and right side to give the investigator a good impression of the whole nasal and temporal bulbar region.

View Large Image Download to PowerPoint

Fig. 3. CCLRU bulbar conjunctival redness scale (subjective). Reproduced courtesy of CCLRU.

Fig. 4a and b provide examples of low amounts of bulbar redness that were graded using both subjective and objective means. Subjectively they may be graded similarly, but, when measured objectively, they are different.

View Large Image Download to PowerPoint

Fig. 4. (a and b) Examples of low amounts of bulbar hyperemia that were graded using both subjective and objective means.

The ocular surface was examined at baseline and after the lens was removed at the 6-month EW visit. A single investigator assessed the bulbar conjunctival redness (nasal and temporal quadrants) using the CCLRU grade scale converted to a 0–100 incremental grading scale with a comparison to the four photos provided (Fig. 3). This scale has been previously suggested since a strong linear relationship exists between an ordinal graded scale and a continuous one [21] and also in order to increase the ability to detect small differences between experimental treatments [17].

6. Data analysis 

For analysis of the repeatability study, limits of agreement [25] and the correlation coefficient of concordance, CCC, were used [26]. CCC describes concordance between repeated measurements by analyzing the deviation of test and re-test measures from a perfect 45°-line through the origin (i.e., CCC=1). CCCs<1 represent deviations from this perfect line and correspond to a weaker repeatability.

Comparisons were made over time and between the two methods of grading. Between method and visit comparisons were made using paired t-tests and a two-way ANOVA for method and time interactions. Measurement methods were compared using Pearson correlation. p-Values less than 0.05 were considered to indicate a statistically significant difference.

7. Results 

7.1. Part 1: Repeatability study 

The Spectrascan650® proved to be highly repeatable. As an example, Fig. 5 shows the limits of agreement for analysis of test/re-test discrepancies with respect to u′ for the reference image representing grade 2 (or 25–50 on the 0–100 scale) in the CCLRU scale. The bold horizontal line near zero represents the mean of all differences between test and re-test; the upper and lower horizontal lines show the limits of agreement. Almost perfect concordance between test and re-test measurements (CCC=0.989) was found for all reference grades in the scale.

View Large Image Download to PowerPoint

Fig. 5. The limits of agreement. The bold horizontal line near zero represents the mean of all differences between test and re-test; the upper and lower horizontal lines show the limits of agreement .

7.2. Part 2: Bulbar redness over time, objective and subjective results 

Fig. 6 illustrates the relationship between wearing time (horizontal axis) and objective grading and subjective grading, respectively, for the 24 participants (mean±standard error). There was no change from baseline (mean and standard deviation, 11.6±6.0) to 6 months of overnight wear (11.7±5.3) in redness when measured with the CIEu* method (objective) (p=0.226). Whereas, there was a significant change from baseline (32.3±8.7) compared to 6 months of overnight wear (35.6±9.3) that is demonstrated with the subjective grading (p=0.001).

View Large Image Download to PowerPoint

Fig. 6. Objective and subjective measurement of bulbar redness over time (p=0.226 and 0.001, respectively).

8. Correlation between objective and subjective bulbar redness 

Fig. 7 shows the relationship between subjective clinical redness and objective redness for all subjects. The correlation between the objective and subjective scales was R=0.795 (p=0.000). Though significant, this moderate correlation value can be accounted for by the fact that the range of bulbar redness in this study was low and its subjective graded results were somewhat variable (±S.D.).

View Large Image Download to PowerPoint

Fig. 7. Correlation between objective and subjective bulbar redness (p=0.000).

9. Discussion 

The grading of bulbar redness (erthyma or hyperaemia) is an important tool in estimating the efficacy, for instance, of drugs for the management of allergy or judging the efficacy of certain contact lenses (CL) on their own or in combination with care systems in reducing the inflammatory response associated with CL wear and the symptom of dryness. Hence the purpose of the study to look at an objective measure of evaluating redness after wearing silicone hydrogel lenses in order to assess the efficacy of this lens material in regards to this parameter.

In light of evidence [20] that clinicians grade bulbar redness quite inconsistently even when photographic scales are presented with a continuous numerical basis, this study proposes the use of a relatively inexpensive, easy to use and readily available photometer that measures redness (chromaticity) and is independent of luminance. Other objective analysis techniques have been developed over the past decade and their repeatability and sensitivity have been assessed [15]. Now the repeatability of using a photometric technique is reported here.

The high levels for repeatability expressed by the limits of agreement (Fig. 5) and CCC emphasize the ability of the photometer to repeatedly and reliably measure varying degrees of redness. If photometric measurements reveal changing values for the redness in a patient's eye, these can likely be attributed to actual changes and not to variability of the measurement itself.

Measurements taken with the photometer appeared to follow the same trends as the subjectively graded data, indicating that bulbar redness can be accurately determined objectively as seen in Fig. 6. Using the objective photometric method, there was no significant increase in redness over the 6-month lens wear period (p>0.05). In contrast to the photometric results, the bulbar redness ratings recorded by the investigator revealed a significant increase in redness from the baseline to the 6-month EW visit (p=0.001). This result further strengthens the need to develop objective methods to avoid any human bias or error. This method of measuring redness objectively has the potential to replace subjective scales in multi-centre studies, where variability by investigator or among several investigators may occur.

Perhaps the subjective grading of ocular redness has less to do with very specific judgments about local variations in redness (e.g., vessel thickness and tortuosity). Subjective grading appears to have more to do with a combination of background redness (microvascular engorgement) as well as the characteristics of large vessels, which could explain the moderate correlation found when we compared this method with the objective method. This moderate correlation between the two methods is seen in Fig. 7.

In this study, we found low amounts of bulbar redness with the use of silicone hydrogel lenses (Fig. 4a and b): since there is a higher sensitivity of printed grading scales at the low end of the scale [16], there may be more variability in grading ocular redness using subjective grading scales. A higher correlation to photometric scales may have been evident if there were larger amounts of bulbar redness detected. In summary, the measurement of bulbar redness using CIEu* chromaticity compared well to the use of standard photographic grading scales. Its future use will minimize the need for more subjective scales, offering accurate and more sensitive measurements of redness, particularly for low levels of redness.