Essay Grade

Windle et al. Health and Quality of Life Outcomes 2011, 9:8
http://www.hqlo.com/content/9/1/8
RESEARCH
Open Access
A methodological review of resilience
measurement scales
Gill Windle1*, Kate M Bennett2, Jane Noyes3
Abstract
Background: The evaluation of interventions and policies designed to promote resilience, and research to
understand the determinants and associations, require reliable and valid measures to ensure data quality. This
paper systematically reviews the psychometric rigour of resilience measurement scales developed for use in
general and clinical populations.
Methods: Eight electronic abstract databases and the internet were searched and reference lists of all identified
papers were hand searched. The focus was to identify peer reviewed journal articles where resilience was a key
focus and/or is assessed. Two authors independently extracted data and performed a quality assessment of the
scale psychometric properties.
Results: Nineteen resilience measures were reviewed; four of these were refinements of the original measure. All
the measures had some missing information regarding the psychometric properties. Overall, the Connor-Davidson
Resilience Scale, the Resilience Scale for Adults and the Brief Resilience Scale received the best psychometric
ratings. The conceptual and theoretical adequacy of a number of the scales was questionable.
Conclusion: We found no current ‘gold standard’ amongst 15 measures of resilience. A number of the scales are
in the early stages of development, and all require further validation work. Given increasing interest in resilience
from major international funders, key policy makers and practice, researchers are urged to report relevant validation
statistics when using the measures.
Background
International research on resilience has increased substantially over the past two decades [1], following dissatisfaction with ‘deficit’ models of illness and psychopathology
[2]. Resilience is now also receiving increasing interest
from policy and practice [3,4] in relation to its potential influence on health, well-being and quality of life
and how people respond to the various challenges of
the ageing process. Major international funders, such
as the Medical Research Council and the Economic
and Social Research Council in the UK [5] have identified resilience as an important factor for lifelong health
and well-being.
Resilience could be the key to explaining resistance to
risk across the lifespan and how people ‘bounce back’
* Correspondence: g.windle@bangor.ac.uk
1
Dementia Services Development Centre, Institute of Medical and Social
Care Research, Bangor University, Ardudwy, Holyhead Road, Bangor, LL56
2PX Gwynedd, UK
Full list of author information is available at the end of the article
and deal with various challenges presented from childhood to older age, such as ill-health. Evaluation of interventions and policies designed to promote resilience
require reliable and valid measures. However the complexity of defining the construct of resilience has been
widely recognised [6-8] which has created considerable
challenges when developing an operational definition of
resilience.
Different approaches to measuring resilience across
studies have lead to inconsistencies relating to the nature of potential risk factors and protective processes,
and in estimates of prevalence ([1,6]. VanderbiltAdriance and Shaw’s review [9] notes that the proportions found to be resilient varied from 25% to 84%. This
creates difficulties in comparing prevalence across studies, even if study populations experience similar adversities. This diversity also raises questions about the
extent to which resilience researchers are measuring
resilience, or an entirely different experience.
© 2011 Windle et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons
Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in
any medium, provided the original work is properly cited.
Windle et al. Health and Quality of Life Outcomes 2011, 9:8
http://www.hqlo.com/content/9/1/8
One of the main tasks of the Resilience and Healthy
Ageing Network, funded by the UK Cross-Council programme for Life Long Health and Wellbeing (of which
the authors are members), was to contribute to the
debate regarding definition and measurement. As part
of the work programme, the Network examined how
resilience could best be defined and measured in order
to better inform research, policy and practice. An extensive review of the literature and concept analysis of resilience research adopts the following definition.
Resilience is the process of negotiating, managing and
adapting to significant sources of stress or trauma.
Assets and resources within the individual, their life and
environment facilitate this capacity for adaptation and
‘bouncing back’ in the face of adversity. Across the life
course, the experience of resilience will vary [10].
This definition, derived from a synthesis of over 270
research articles, provides a useful benchmark for
understanding the operationalisation of resilience for
measurement. This parallel paper reports a methodological review focussing on the measurement of resilience.
One way of ensuring data quality is to only use resilience measures which have been validated. This requires
the measure to undergo a validation procedure, demonstrating that it accurately measures what it aims to do,
regardless of who responds (if for all the population),
when they respond, and to whom they respond. The
validation procedure should establish the range of and
reasons for inaccuracies and potential sources of bias. It
should also demonstrate that it is well accepted by
responders and that items accurately reflect the underlying concepts and theory. Ideally, an independent ‘gold
standard’ should be available when developing the questionnaire [11,12].
Other research has clearly demonstrated the need for
reliable and valid measures. For example Marshall et al.
[13] found that clinical trials evaluating interventions for
people with schizophrenia were almost 40% more likely
to report that treatment was effective when they used
unpublished scales as opposed to validated measures.
Thus there is a strong case for the development, evaluation and utilisation of valid measures.
Although a number of scales have been developed for
measuring resilience, they are not widely adopted and
no one scale is preferable over the others [14]. Consequently, researchers and clinicians have little robust evidence to inform their choice of a resilience measure and
may make an arbitrary and inappropriate selection for
the population and context. Methodological reviews aim
to identify, compare and critically assess the validity and
psychometric properties of conceptually similar scales,
and make recommendations about the most appropriate
use for a specific population, intervention and outcome.
Fundamental to the robustness of a methodological
Page 2 of 18
review are the quality criteria used to distinguish the
measurement properties of a scale to enable a meaningful comparison [15].
An earlier review of instruments measuring resilience
compared the psychometric properties and appropriateness of six scales for the study of resilience in adolescents [16]. Although their search strategy was thorough,
their quality assessment criteria were found to have
weaknesses. The authors reported the psychometric
properties of the measures (e.g. reliability, validity, internal consistency). However they did not use explicit quality assessment criteria to demonstrate what constitutes
good measurement properties which in turn would
distinguish what an acceptable internal consistency
co-efficient might be, or what proportion of the lowest
and highest scores might indicate floor or ceiling effects.
On that basis, the review fails to identify where any of
the scales might lack specific psychometric evidence, as
that judgement is left to the reader.
The lack of a robust evaluation framework in the work
of Ahern et al. [16] creates difficulties for interpreting
overall scores awarded by the authors to each of the
measures. Each measure was rated on a scale of one to
three according to the psychometric properties presented, with a score of one reflecting a measure that is
not acceptable, two indicating that the measure may be
acceptable in other populations, but further work is
needed with adolescents, and three indicating that the
measure is acceptable for the adolescent population on
the basis of the psychometric properties. Under this criteria only one measurement scale, the Resilience Scale
[17] satisfied this score fully.
Although the Resilience Scale has been applied to
younger populations, it was developed using qualitative
data from older women. More rigorous approaches to
content validity advocate that the target group should be
involved with the item selection when measures are being
developed[11,15]. Thus applying a more rigorous criterion
for content validity could lead to different conclusions.
In order to address known methodological weaknesses
in the current evidence informing practice, this paper
reports a methodological systematic review of resilience
measurement scales, using published quality assessment
criteria to evaluate psychometric properties[15]. The
comprehensive set of quality criteria was developed for
the purpose of evaluating psychometric properties of
health status measures and address content validity,
internal consistency, criterion validity, construct validity,
reproducibility, responsiveness, floor and ceiling effects
and interpretability (see Table 1). In addition to
strengthening the previous review, it updates it to the
current, and by identifying scales that have been applied
to all populations (not just adolescents) it contributes an
important addition to the current evidence base.
Windle et al. Health and Quality of Life Outcomes 2011, 9:8
http://www.hqlo.com/content/9/1/8
Page 3 of 18
Table 1 Scoring criteria for the quality assessment of each resilience measure
Property
1
2
3
4
5
Definition
Quality criteria
Content validity The extent to which the domain of interest is
+
comprehensively sampled by the items in the
2
questionnaire (the extent to which the measure represents
all facets of the construct under question).
Internal
consistency
The extent to which items in a (sub)scale are
intercorrelated, thus measuring the same construct
Criterion validity The extent to which scores on a particular questionnaire
relate to a gold standard
Construct
validity
The extent to which scores on a particular questionnaire
relate to other measures in a manner that is consistent
with theoretically derived hypotheses concerning the
concepts that are being measured
A clear description of measurement aim, target population,
concept(s) that are being measured, and the item selection
AND target population and (investigators OR experts) were
involved in item selection
?
1
A clear description of above-mentioned aspects is lacking OR
only target population involved OR doubtful design or
method
0
No target population involvement
0
0
No information found on target population involvement
+
2
Factor analyses performed on adequate sample size (7*
#items and > = 100) AND Cronbach’s alpha(s) calculated per
dimension AND Cronbach’s alpha(s) between 0.70 and 0.95
?
1
No factor analysis OR doubtful design or method
0
Cronbach’s alpha(s) 0.95, despite adequate design
and method
0
0
No information found on internal consistency
+
2
Convincing arguments that gold standard is “gold” AND
correlation with gold standard > = 0.70
?
1
No convincing arguments that gold standard is “gold” OR
doubtful design or method
0
Correlation with gold standard = 0.70
?
1
Doubtful design or method
0
ICC or weighted Kappa < 0.70, despite adequate design and
method
0
0
No information found on reliability
Reproducibility
5.1 Agreement
5.2 Reliability
The extent to which the scores on repeated measures are
close to each other (absolute measurement error)
The extent to which patients can be distinguished from
each other, despite measurement errors (relative
measurement error)
Windle et al. Health and Quality of Life Outcomes 2011, 9:8
http://www.hqlo.com/content/9/1/8
Page 4 of 18
Table 1 Scoring criteria for the quality assessment of each resilience measure (Continued)
6
7
8
Responsiveness
Floor and
ceiling effects
Interpretability
The ability of a questionnaire to detect clinically important +
changes over time
2
The number of respondents who achieved the lowest or
highest possible score
The degree to which one can assign qualitative meaning
to quantitative scores
SDC or SDC < MIC OR MIC outside the LOA OR RR > 1.96 OR
AUC > = 0.70
?
1
Doubtful design or method
0
SDC or SDC > = MIC OR MIC equals or inside LOA OR RR <
= 1.96 or AUC 0.70. For the RSA, two separate analyses
report that one of the six subscales to be 0.70 for all the subscales.
The ICC was 0.87 for the CD-RISC, but the sample size

Purchase answer to see full
attachment