| |
The Master of Library & Information Science (MLIS) degree at the University
of Pittsburgh is an established program with an international reputation for
excellence. The University's Center for Instructional Development & Distance
Education (CIDDE) provided support for this program including help with evaluation.
The purpose of the evaluation was to determine the effectiveness of the accredited
online FastTrack (FT) program leading to a MLIS degree. The two-year program,
designed to meet the needs of adult learners, is cohort-based and consists
of 36 credits with an on-campus weekend learning experience every semester.
Program evaluation was a meta-activity that incorporated all aspects of the
learning experience. Both the learning process and outcomes were assessed qualitatively
and quantitatively. The overall program effectiveness was determined by:
- Informal
Discussion Board analysis - The Discussion Board was a critical component
of the FT programs as students respond to specific questions and issues.
The analysis identified postings where the learner achieved higher levels
of thinking (i.e., application, synthesis, and evaluation).
- Assignments - The course instructors assessed students' work. The results
of these assessments yielded quantitative data for formative program
evaluation.
- Focus groups - During the first semester of the six-semester
program, the evaluator conducted on-campus focus groups. And during the
second and fifth semester,
she facilitated on-line focus groups. The purpose of all the
groups was to determine student satisfaction with the program.
- Faculty
perceptions - Anecdotal reports compared the face-to-face and the online
courses.
- Usage statistics - The Blackboard course management system
provides user statistics on the number of accesses per
area of the course,
the number of accesses over
time, accesses by hour of the day, and total accesses by
user.
- End of course
evaluations - End of term evaluations are administered by the Office of Measurement & Evaluation
of Teaching. Instructors have the option of adding additional questions
specific to Blackboard
or their
courses.
- Course outcomes - The grades from final projects
and papers provide quantitative, summative assessments.
Faculty
taught
their courses
in the face-to-face delivery
format prior to teaching online. Therefore, they had
data that allowed them to compare delivery formats.
- End of program
survey - Upon completion of the program, students completed
a summative Blackboard survey. Likert
scale and
open-ended questions
measured learner satisfaction with instruction, the
on-campus component of the program,
and student services.
- Employer analysis - Six months
after graduation, a survey evaluating employers' satisfaction with
the
recent graduates'
knowledge
and skills will be mailed.
- Department analysis - The department analysis consists of a review of records noting demographic
and retention
information
about
the students.
- Institutional
analysis - This analysis
includes a summary of University publications and comments
from
administrators/departments.
Evaluation is paramount to program success. Throughout the FT program,
multiple sources of data have yielded a comprehensive overview of
the program's effectiveness.
In addition, the consistent evaluation process has provided faculty
and administrators with valuable data to make informed decisions. |
