Collection and Interpretation of Data In Nursing Education Part-IV
Collection and Interpretation of Data In Evaluation Process
Collecting Data
The next step in the evaluation
process is to use the evaluation instrument to gather data. Although the
instrument will determine to some extent what data are collected and how,
several other factors should be considered at this time. These include the data
collector, the data sources, amount of data, timing of data collection, and
informal versus formal data collection.
Data Collector
Consideration must be given to whoever is collecting the data. For example, the evaluator who gathers the data might be the faculty member evaluating the clinical performance of the students. In other situations, students or research assistants may administer instruments.
If the data collectors are not familiar with the data collecting
procedures, they should be oriented to the task. Interrater reliability must be
ensured when more than one person is collecting data.
Data Source
Before evaluation, the evaluator must identify sources from which the data will be collected. Will the data be observed (as in clinical evaluation), archival (as when grade point average is obtained from student records), or reported (as obtained from a longitudinal questionnaire of graduates)?
At this time in the evaluation process, it is
important to determine whether it is possible to have access to records,
particularly if permission must be obtained from the participants
Amount of Data
The amount of data to be collected must also be determined and specified. All data may be collected or a sample may be sufficient, but a decision must be made. For example, in clinical evaluation or classroom testing it is impossible to collect data about each instance of clinical performance or knowledge gained from the classroom experience.
In this instance, a sampling procedure is used and guided by the
clinical evaluation protocol, blueprint, or plan for the classroom test. It is
important to note that the sampling plan must be established at this stage of
the evaluation process.
Timing of data collection
When is the best time to collect the data? An understanding of the context of evaluation is helpful here. Should the data be collected at the beginning, middle, or end of the activity being evaluated?
When gathering data from students, it is important to allow adequate
time and to gather data when students are able to give unbiased responses. (For
example, course evaluation data collected immediately after test results have
been given may not yield the most reflective responses ).
Formal versus
Informal Data Collection Decisions
about use of formal and informal data must also be made. Data can be obtained
in a formal manner, such as by using a structured evaluation tool. Data can
also be collected using informal methods, such as in the form of spontaneous
comments made by students. The evaluator must decide whether both formal and
informal data will be used in the plan.
Interpreting Data
The interpretation step of the
evaluation process involves translating data to answer the evaluation questions
established at the beginning of the evaluation process. This involves putting
the data into usable form, organizing data for analysis, and interpreting the
data against pre-established criteria. When data are interpreted, the context,
frame of reference, objectivity, and legal and ethical issues must also be
considered.
Frame of Reference
Frame of reference refers to the
reference point used for interpretation of data. Two frames of reference are
discussed here: norm-referenced interpretation and criterion-referenced
interpretation.
Norm-Referenced Interpretation
Norm-referenced interpretation refers to interpreting data in terms of the norms of a group of individuals who are being evaluated. The scores of the group form a basis for comparing each individual with the others. In norm-referenced evaluation, there will always be an individual who has achieved at the highest level, as well as one who has achieved at the lowest level.
Norm-referenced interpretation allows evaluators to compare achievement of students in several ways. Students in the same group can be compared and ranked. Students can be compared with students in another group or class section or with national group norms, as in the case of licensing examinations or nursing specialty certification examinations.
Consequently, an advantage of norm referenced interpretation is the ability to
make comparisons within groups or with external groups and to use the data for
predictive purposes, such as admission criteria. A disadvantage of norm
referenced interpretation is the focus on comparison, which may foster a sense
of competitiveness among students.
Criterion Referenced Interpretation
In criterion-referenced interpretation, on the other hand, results are judged against preestablished criteria and reflect the degree of criteria attainment. Criterion referenced interpretation is typically used in competence-based learning models in which the goal is to assist the learner to achieve competence in or mastery of specified learning outcomes.
Because students are compared with the outcomes
and not each other, all students can achieve competence. The advantages of
criterion-referenced interpretation include the following: emphasis on mastery
and the potential for all learners to achieve increased learner motivation,
sharing and collaboration among students, and ability to give clear progress
reports to learners. Disadvantages of criterion-referenced interpretation
include the inability to compare students with each other or with other groups.
Issues of Objectivity and Subjectivity
The issues of objectivity and subjectivity in evaluation always arise when data are interpreted. Different evaluators can look at the same data yet render different judgments. The differences may be a result of evaluator bias or degrees of difference in objectivity.
Studies of performance appraisals in work settings have shown the effects of regency interpreting findings when other favorable findings have preceded the evaluation (Polit & Beck, 2013).
In some ways, faculty need to accept that
there is a certain amount of subjectivity in evaluation; After all, this is
“evaluation” and not “measurement.” However, faculty should recognize
subjectivity and the role it may play in interpretation of findings.
Legal Considerations
There may be legal aspects involved in the interpretation of findings. Legal consideration is particularly important in the area of student rights. How will the results of evaluations be shared? What data about students can be collected? Does evaluation involve protection of human subjects? Will there be moral or ethical dilemmas in reporting the data? Who is affected by evaluations?
How will they respond to
the results? What effect will evaluation have on a student, a program, or a
curriculum? Accordingly, the evaluator and the audience must be aware of the
context of the evaluation because these elements can influence how the
evaluation is conducted, how results are reported, what will change because of
the evaluation, and how due process will be handled.
Give your opinion if have any.