Nursing Education and Validity or Reliability of Evaluation Instruments Part-III
Validity and Reliability of Evaluation Instruments In Nursing Education
Portfolio
A portfolio is used to provide evidence of program learning outcomes through the accumulation of artifacts collected throughout the program. An artifact is the production of instruction such as an assignment, PowerPoint presentation, concept map, care plan, and so on. A guide for collecting artifacts is provided to students at the start of their program to facilitate successful completion.
Portfolios are stored either
electronically or in an organized paper version. Often reflections are added to
each artifact to show how the artifact meets program learning outcomes.
Reflections enhance student learning and ensure progress toward meeting program
learning outcomes.
Reliability and Validity of Evaluation Instruments Used in Nursing Education
Whenever any instrument is used, its validity and reliability for evaluation should be ensured. Special procedures can be used to determine the reliability and validity of instruments used for clinical evaluation, program evaluation, and examinations given to measure classroom achievement.
Specific procedures are discussed in appropriate
chapters of this book. A general overview of the concepts of validity and
reliability is provided here .
Validity
Measurement validity verifies that faculty are in fact collecting and analyzing results they intend to measure. Measurement validity, particularly in the area of educational assessment and evaluation, has attributes of relevance, accuracy, and utility (Prus & Johnson, 1994; Wholey, Haltry , & Newcomer, 2004).
Relevance of an instrument is achieved when the instrument measures the educational objective as directly as possible. The instrument is accurate if it measures the educational objective precisely. The instrument is useful if it provides formative and summative results that have implications for evaluation and improvement.
As a result, valid evaluation instruments have relevance for the local program or curriculum and can provide meaningful results that indicate directions for change (Prus & Johnson, 1994). Although there are several types of validity, measurement validity is viewed as a single concept.
Content-related evidence, criterion-related evidence, and construct-related evidence are considered categories of validity. For interpretation, evidence from all categories is ideal. The validity of an instrument can best be determined when faculty understand the nature of the content and specifications in the evaluation design, the relationship of the instrument to the significant criterion, and the constructs or psychological characteristics being measured by the instrument (Waugh & Gronlund, 2012).
Content-related evidence refers to the extent to which the instrument is representative of the larger domain of the behavior being measured. Content-related evidence is particularly important to establish for clinical evaluation instruments and classroom tests. For example, with classroom tests, the following question is raised: “Does the sample of test questions represent all content described in the course?”
In clinical evaluation, the question posed is: “Does the instrument measure attitudes, behaviors, and skills representative of the domain of being a nurse?” Criterion-related evidence refers to the relationship of a score on one measure (test, clinical performance appraisal) to other external measures. There are two ways to establish criterion-related evidence: concurrent and predictive.
Concurrent evidence is the correlation of one score with another measure that occurs at the same time. The most common example of concurrent validity is the correlation of clinical course grades with didactic course grades. Concurrent validity of the instrument is said to occur, for example, when there is a high correlation between clinical evaluation and examination scores in a class of students.
Predictive study, on the other hand, is a correlation with measures obtained after completion of an event or intervention, such as a course or lesson. For example, there may be predictive validity between course grades and licensing examination or certification examination scores. Criterion-related evidence is used to relate the outcomes of one instrument to the outcomes of another.
In this sense, it is used to predict success or establish the predictability of one measure with another. Criterion-related evidence is established by using correlation measures. One example is the correlation between grade point average and scores on licensing or credentialing examinations.
When there is a high positive correlation between the grade point average and the examination score, there is said to be criterion-related evidence. Construct-related evidence is a relationship of one measure (eg, examination) to other learner variables such as learning style, IQ, clinical competence, or job experience.
Construct-related evidence is used
to infer the relationship between a test instrument and student traits or
qualities in order to identify what factors might be influencing performance.
Examples include the relationship of IQ scores, Scholastic Aptitude Test
scores, and other test scores or working for pay as a student nurse and
clinical performance.
Reliability
Reliability is the extent to which an instrument (self-report examination, observation schedule, or checklist) is dependable, precise, predictable, and consistent. Pedhazur and Schmelkin (1991) refer to reliability as the degree to which test scores are free from errors of measurement.
Reliability answers the question: “Will the same instrument yield the same results with different groups of students or when used by different raters?” According to Newby (1992), “Reliability in testing refers to the idea that tests should be consistent in the way that they measure performance” (p. 253).
Several types of reliability—stability reliability, equivalence reliability, and internal consistency reliability—are relevant to evaluation instruments and achievement examinations. Stability reliability of an instrument is the perceived consistency of the instrument over time. An assumption of stability in results is assumed.
Equivalence reliability means the degree to which two different forms of an instrument can be used to obtain the same results. For example, when two forms of a test are used, both tests should have the same number of items and the same level of difficulty. The test is given to the group at the same time as the equivalent test is given or the equivalent test is administered at a later date.
Internal consistency reliability is associated with the extent to which all items on an instrument measure the same variable and with the homogeneity of the items. This reliability is considered only when the instrument is used to measure a single concept or construct at a time. Because the validity of findings is threatened when an instrument is unreliable, faculty should use measures to ensure instrument reliability.
Give your opinion if have any.