Translation and validation to Portuguese of a 60-item questionnaire
to evaluate theoretical knowledge in fundus examination
Blindness from reversible causes has been increasing globally. In 2020, 237 million
people were estimated to have a moderate or severe visual impairment, while another
38 million were considered blind  . Among irreversible causes of blindness, like diabetic retinopathy
and glaucoma, an early diagnosis may slow down progression since treatment has been
instituted to prevent a severe visual impairment. In this context, fundoscopy
emerges as a fast, cheap, and effective strategy for tracking down eye
pathologies. Despite the importance of fundus examination, many doctors currently lack knowledge
of the technique and/or self-confidence in performing the examination  . The
International Council of Ophthalmology  recommends
knowledge of the basic use and handling of fundoscopy equipment and how to examine
and identify the normal and pathological structures. In consonance, the Association
of University Professors of Ophthalmology  highlights the importance of fundoscopy in primary care and
its various applications in prevention and health promotion. Recently, a questionnaire to measure self-confidence in performing fundoscopy was
translated and validated into Portuguese  . However, at the time, there is no available instruments in
Portuguese to measure physicians’ and medical students’ knowledge regarding fundus
examination. A questionnaire that evaluates several aspects of fundoscopy such as
technique and normal and abnormal findings would help to standardize future research
and to propose future interventions to improve medical education in ophthalmology.
This study aimed to translate and validate a questionnaire into Portuguese to
evaluate theoretical knowledge regarding fundus examination. 

This study was approved by the Ethics Committee of the University of Campinas and
conducted in compliance with the Declaration of Helsinki. All procedures were fully
explained and informed consent was obtained from all participants. Translation, cross-cultural adaptation, and validation process The questionnaire developed by Jørgensen et al.  which was used as a reference, consists of 60
multiplechoice questions with 3 answers, of which only 1 is the correct answer.
This questionnaire investigates the various aspects of knowledge in the
fundoscopy examination, such as the details of the correct technique (items 1-6,
12, 13, 21-26, 43, 45, 56-58, 60), anatomical structures, and pathological
findings (items 7-11, 14-20, 27-42, 44, 46-55, 59). Among the 60 questions, 18
were considered easy and 42 to be of moderate difficulty level. The translation,
cross-cultural adaptation, and validation process were followed as described in
the literature  . The reference questionnaire was translated from
English to Portuguese by two independent translators, both with high proficiency
in English and one with an experience in the fundus examination and
ophthalmology. Both the translations were then synthesized and evaluated by an
expert committee, which evaluated the equivalence of the translation and made
the necessary modifications. The back-translation from Portuguese to English was
made by two independent translators with high proficiency in the English
language. Both back translations were then compared to the original version of
the questionnaire to evaluate the consistency of the translation process.
Finally, the expert committee evaluated the final version of the questionnaire
(available as Supplemental Material) to verify the semantic, idiomatic, and
conceptual equivalence relative to the original questionnaire. Test of the translated questionnaire The resulting questionnaire was applied to final-year medical students and
ophthalmology residents. Each subject answered the questionnaire twice, at an
interval of 1 week between each application (test-retest). The participants who
did not complete the questionnaire at both times were excluded. Internal
consistency, test-retest reliability, inter-rater reliability, and percentage
agreement were also calculated. Statistical analysis The statistical analysis was performed with the Statistical Package for Social
Sciences-SPSS (IBM Corporation, Armon NY, USA, version 22.0). The normality of
the mean score of the groups was calculated by Shapiro-Wilk test. The comparison
was performed with Mann-Whitney U-test. P<0.05 was considered to indicate
statistical significance. The pass-fail cutoff was calculated with contrasting group methods as described
by Jørgensen et al.  . Internal consistency, to evaluate the inter-correlation of
the questionnaire items, was calculated by using coefficient alpha (Cronbach’s
alpha-α), which varied from 0 to 1, while a value of >0.7 was
considered adequate  . Test-retest reliability was calculated by Pearson’s correlation (R), and
inter-rater reliability was calculated by Cohen’s Kappa (κ), which varied
from 0 to 1, and was classified as <0.2, poor agreement; 0.2-0.4, week
agreement, 0.4-0.6, moderate agreement; 0.6-0.8, good agreement; >0.8
excellent agreement  . Percentage agreement was
calculated by the number of agreements divided by the total number of
answers  . 

The questionnaire developed by Jørgensen et al.  which was used as a reference, consists of 60
multiplechoice questions with 3 answers, of which only 1 is the correct answer.
This questionnaire investigates the various aspects of knowledge in the
fundoscopy examination, such as the details of the correct technique (items 1-6,
12, 13, 21-26, 43, 45, 56-58, 60), anatomical structures, and pathological
findings (items 7-11, 14-20, 27-42, 44, 46-55, 59). Among the 60 questions, 18
were considered easy and 42 to be of moderate difficulty level. The translation,
cross-cultural adaptation, and validation process were followed as described in
the literature  . The reference questionnaire was translated from
English to Portuguese by two independent translators, both with high proficiency
in English and one with an experience in the fundus examination and
ophthalmology. Both the translations were then synthesized and evaluated by an
expert committee, which evaluated the equivalence of the translation and made
the necessary modifications. The back-translation from Portuguese to English was
made by two independent translators with high proficiency in the English
language. Both back translations were then compared to the original version of
the questionnaire to evaluate the consistency of the translation process.
Finally, the expert committee evaluated the final version of the questionnaire
(available as Supplemental Material) to verify the semantic, idiomatic, and
conceptual equivalence relative to the original questionnaire. 

The resulting questionnaire was applied to final-year medical students and
ophthalmology residents. Each subject answered the questionnaire twice, at an
interval of 1 week between each application (test-retest). The participants who
did not complete the questionnaire at both times were excluded. Internal
consistency, test-retest reliability, inter-rater reliability, and percentage
agreement were also calculated. 

The statistical analysis was performed with the Statistical Package for Social
Sciences-SPSS (IBM Corporation, Armon NY, USA, version 22.0). The normality of
the mean score of the groups was calculated by Shapiro-Wilk test. The comparison
was performed with Mann-Whitney U-test. P<0.05 was considered to indicate
statistical significance. The pass-fail cutoff was calculated with contrasting group methods as described
by Jørgensen et al.  . Internal consistency, to evaluate the inter-correlation of
the questionnaire items, was calculated by using coefficient alpha (Cronbach’s
alpha-α), which varied from 0 to 1, while a value of >0.7 was
considered adequate  . Test-retest reliability was calculated by Pearson’s correlation (R), and
inter-rater reliability was calculated by Cohen’s Kappa (κ), which varied
from 0 to 1, and was classified as <0.2, poor agreement; 0.2-0.4, week
agreement, 0.4-0.6, moderate agreement; 0.6-0.8, good agreement; >0.8
excellent agreement  . Percentage agreement was
calculated by the number of agreements divided by the total number of
answers  . 

A total of 30 participants were included in the validation process, of which 25 were
medical students and 5 were ophthalmology residents. The baseline characteristics
are presented in . While the overall mean score was 39.83 ± 8.11 (66.38%), medical students’ mean
score was 37.48 ± 6.6 (62.46%) and ophthalmology residents’ mean score was
51.6 ± 2.7 (86%) (p<0.0001). The pass-fail cutoff was calculated as 46,
the theoretical false positives were 8.7%, and the theoretical false negatives were
2.8% . At this score, the observed
rates of false positive and false negative were 0%. Among the 60 items, test-retest reliability was strong in 17 items having negative
correlation, moderate in 14 items having negative correlation and weak in 29 items.
The inter-rater reliability was <0.4 in 34 items, between 0.4 and 0.6 in 17
items, and >0.6 in 8 items. Only one item had negative inter-rater reliability.
Among the percent agreement, 10 items were between 40%-60% agreement, 50 items were
above 60% agreement, and 18 items were above 80% agreement. Cronbach’s alpha was
calculated as 0.674. The details of each item are presented in . 

The current study successfully translated a questionnaire into Portuguese that
comprehensively evaluated the theoretical knowledge regarding fundus examination.
The questionnaire developed by Jørgensen et al.  recruited 30 participants for the validation
process (20 medical students and 10 ophthalmology specialists). The mean score of
the first group was 30.0 ± 4.3 and the mean score of the second group was
57.4 ± 1.6 (p<0.0001). The pass-fail cutoff was calculated as 49.7, which
represented the point that no medical student passed the test (no false positive)
and no ophthalmology resident failed the test (no false negative). Internal
consistency was calculated as 0.95. The validation process revealed that there was a
significant difference in the scores between the two groups. A lower pass-fail
cutoff (46 vs 49.7) and a lower internal consistency (0.674 vs 0.95) was found in
the current study. Further, reliability is a method to estimate the random errors in an assessment. One
way to evaluate reliability is through internal consistency, represented by
Cronbach’s alpha. The level of accepted reliability for an instrument depends on its
application. For instruments that assess subjects with major consequences for the
society such as obtaining a medical license, a value >0.9 is desired; for
assessments with moderate consequences such as end-of-year or end-of-course
examinations, a minimum of 0.8 is acceptable, and a value <0.7 is acceptable when
the instrument aims to assess classroom proficiency  . In the current study, the reliability was 0.674,
close to the minimum 0.7 acceptable for the aim of the questionnaire, which was to
evaluate theoretical proficiency in one specific item of the physical
examination. Another method to verify the validity of training and instruments is the proposed
contrasting groups’ method  which consists of a comparison between two groups with significantly different
expertise in the subject, a group of novices and a group of experts. The calculation
of the cutoff point and the theoretical false positives and false negatives englobes
the mean and standard deviation of the score of each group, which are considered as
a normal distribution. This reduces the main issue in validation studies, which is
the small sample size, mainly due to the reduced availability of experts. The cutoff
point is considered the intersection between the curves of both groups, as shown in , whereas the theoretical false
positive rate is calculated based on the novices who scored higher than the cutoff
point, and the false negative rate as the experts who scored lower than the
pass-fail point  . The
differences between observed and theoretical false positives and false negatives are
due to the assumption of the normal distribution of the values, which may be
considered a limitation of the study. The small samples are not always normally
distributed, and so the outliers may have a strong influence on the
calculations  . In our
casuistic, low rates of theoretical false positives and false negatives were
observed, which reflected a good representation of novices with low proficiency, and
experts with high proficiency. Both Cronbach’s alpha and contrasting groups are
valid methods to calculate reliability  . Despite the successful translation and cross-cultural adaptation to Portuguese, the
validation process had a few issues. One method of measuring inter-rater reliability
was percentage agreement, which was calculated by the number of agreements divided
by the total number of answers  . However, this method did not take into account the chance of
agreement, which is a random agreement caused by guessing that resulted in a false
agreement. Cohen’s kappa  was introduced to calculate the
agreement including this chance, which varied between -1 and +1, and a value closer
to +1 represented a perfect agreement while a value closer to 0 represented no
agreement. A negative value represented a disagreement between the observers.
Kappa’s correlation also had a few limitations. Despite the minimally accepted
values of 0.40-0.59, it represented only a 15%-35% reliability of the
data  , which may be
critical values depending on the subject. In our study, we obtained a <0.4
correlation in 35 items and a negative correlation in one item, which represented a
limitation of the validation process. However, among the remaining 24 items, a
moderate to strong kappa was calculated, which indicated a good agreement and a
reliable validation process. The current study had a few limitations. The population included in the validation
process was composed mainly of medical students with moderate to low levels of
proficiency in fundoscopy  which could explain the lower level of agreement by
a higher chance of guessing. Also, the extension of the questionnaire composed of
60-items would have influenced the concordance of the answers with a higher rate of
concordance in the initial questions and a lower level in the following questions. A
shorter questionnaire or a shorter version of the same instrument evaluating the
same points would have improved the validation process. In conclusion, the experts translated and validated the English questionnaire into
Portuguese that comprehensively evaluated the knowledge regarding fundoscopy among
medical students and ophthalmology residents. Further research is needed to measure
the theoretical proficiency regarding fundus examination, which would help to
propose effective interventions to improve medical education in ophthalmology. 

