Why evaluate?

There are various reasons or purposes why the process of evaluation occurs. The person attempting to suggest change must first recognize who will be the receivers of the findings and if conclusions must be drawn. The purpose of the evaluation may include one or more of the following:

to confirm learners' needs- this may suggest the direction of the formation of course/program objectives,

to mobilize learner participation- this may assist in identifying and targeting learners' interests which may initiate a motivational response,

to document learning outcomes- the most utilized approach in classroom evaluation, this may suggest change has occurred due in part to participation,

to identify needed program changes- which can be utilized to re-direct the program or lesson to the needs and interests of the learner while still achieving previously set objectives and

to communicate accountability to funders, administrators and others. (Case, 0.3,1988).

Identifying "why" the evaluation will take place will also suggest the model selected.

Are there evaluative models ?

Dr. Robert Reineke, Associate Professor, University of Nebraska, presented an overview from the Worthen and Sanders text, Educational Evaluation: Alternate Approaches and Practical Guidelines, as a presentation to a Michigan State University, distance education graduate class, Program Evaluation in Adult Education. Six suggested general evaluation models/frameworks were discussed.

Six General Evaluation Models/Frameworks:

Objectives Oriented: where the focus is on specifying goals and objectives and determining the extent to which they have been attained.

Management Oriented: where the central concern is on identifying and meeting the informational needs of managerial decision makers.

Consumer Oriented: where the central issue is developing evaluative information on educational "products," broadly defined, for use by educational consumers in choosing among competing curricula, instructional products, and the like.

Expertise-Oriented: which depend primarily on the direct application of professional expertise to judge the quality of educational endeavors.

Adversary-oriented: where planned opposition in points of view of different evaluators (pro and con) is the central focus of the evaluation.

Naturalistic and participant-oriented: where naturalistic inquiry and involvement of participants (stakeholders in that which is evaluated) are central in determining values, criteria, needs and data for the evaluation.

In most program and classroom evaluations the objectives oriented model tends to be utilized. Program or lesson objectives are determined and then are used as the comparative criteria to attempt to suggest that change has or has not occurred due to participation.

When To Evaluate?

Evaluation is an ongoing process which should include all stakeholders, i.e., program planners, learners, administrators, instructors and funders. The successful evaluation process should be implemented at the initial development phase, during the implementation/delivery phase and at the conclusion of the activity in order to demonstrate or suggest change and outcomes due to participation. The real challenge is when to begin collecting evidence. Research and Development in Global Studies suggests there are six appropriate phases to consider collecting evidence for evaluative purposes:

Project Design: The most frequent form of evaluation at this point is a needs assessment. Program directions can be influenced by the needs of the participants.

Resource Development: This involves testing learning materials and delivery strategies before the start of the program.

Program start-up: To establish a baseline to judge changes in knowledge, behavior or practices of the participants. Another reason is to assess whether resources are being deployed properly.

In-progress: To monitor the actual operations in case adjustments are required and to prepare for summative evaluation.

Program wrap-up: Explores end of program reactions and if objectives were adequately met.

Follow-up: The period after the program has ended. This technique may suggest long term benefits of the program as evidenced by practice and behavior changes. (Case, 0.5, 1988).

No program or classroom activity is of the perfect design, therefore, evaluators must be prepared to collect evidence at any point of their effort. The evaluation should begin ideally at the initial design stages which suggests communication with all stakeholders. Information collected may also demonstrate a baseline from which to compare and make assessment of outcomes/changes as the project progresses and concludes.

When planning an evaluation, there are three basic evaluation questions which must be asked:

        Can the program/lesson be evaluated?

        What information is needed?
 
        How do we get the information?

The previous questions are an elementary approach to designing the evaluation. The questions also lead to selection of the steps of evaluation:

        Identify purpose & receivers of results.

        Review purpose with stakeholders.

        Identify data needed to be collected.

        Select data collection methods.

        Select sample/population.

        Develop data collection instruments.

        Gather data.

        Analyze and summarize results.

        Present the information (Case, 0.8 , 1988).

Each listed step is a process of it's own. Much time and effort must be exerted at each step in order to achieve a viable evaluative effort. Involve all the stakeholders concerned throughout the evaluative process.

Evaluating students in the classroom.

The classroom teacher faces a variety of challenges when confronted with the evaluation of students in the classroom. McKeachie, the author of Teaching Tips, suggests, "evaluation is a great deal more than giving a grade. In teaching the major part of evaluation should be in the form of comments on papers, responses to student statements, conversations, and other means of helping students understand where they are and how to do better." The educator role is not only to impart knowledge but also to provide leadership and direction in helping the student learn how to learn and move towards their potential. The previous statements suggest a somewhat objective approach to evaluating a student. Educators must move towards a subjective process while empowering the student with the opportunity of achieving their own evaluative marks.

The educator must set and communicate to the learners an absolute standard. These standards must be developed in coordination with the tested goals and objectives developed for the specific class. In 1950, Travers proposed one set of absolute standards:

         A: All major and minor goals achieved.

         B: All major goals achieved; some minor ones not.

         C: All major goals achieved; many minor ones not.

         D: A few major goals achieved, but student is not prepared for advanced work.

         E or F: None of major goals achieved (McKeachie, 1994).

This set of absolute standards enables the educator to develop goals and objectives which may empower the learners to make decisions regarding the grade earned. Assigning of grades may consist of a summation of instructor required and student selected activities, i.e.,

         an ungraded test initially to orient students to educator's testing style and provide educator  with student needs,

         examinations which should consist of a variety of questioning designs, i.e., problem solving, short answer, essay, true/false, and multiple choice,

         group activities,

         outside research projects,

         student journal,

         research paper and

         outside reading and reporting assignments (McKeachie, 1994, pp. 117-22).

This combination of instructor required and student selected activities appropriately empowers the student to achieve their set goals by the selection of activities to complete, a learning contract. This process suggests the contract grading approach where the student and the instructor enter into an agreed upon contract with specific activities, which when completed, will provide the subjective results to compare to the classroom's previously set absolute standards. Contract grading moves the responsibility for evaluation towards the accomplishments of the student, thus, enabling the educator to become more subjective in the evaluative process. Other possibilities of assigning grades may be selected. Competency-based grading, grading group projects, cooperative grading by peers and grading on the curve are also utilized by various educators. Contract grading may suggest a more subjective approach and empowers the student with the developmental process of responsibility, accountability and learning how to achieve goals.

An educator must continue to learn, not only about their respective discipline, but also focus on the learner's needs. In order to achieve the goal of an educator, which is to assist the learner in how to learn, not only what to learn, one must explore various instructional strategies, evaluative processes and attempt to provide learning opportunities for all students at all learning levels.

Dr. Frank Bobbitt, Professor, Michigan State University, stated, "the most diverse characteristic of an individual in society today is not race, religion, gender, or socioeconomic status.....it is an individual's learning style." This is a very profound statement. Educators must focus on this challenge if our "leaders of tomorrow" are to develop and become accountable for their choices, actions and educational development.
 

Bibliography

Case, Roland, Andrews, Mary and Werner, Walter . 1988. How Can We Do It? An Evaluation Training Package for Development Educators. East Lansing: Michigan State University.

Cookson, Peter S., 1996. Program Planning for Lifelong Education [Draft]. State College: Pennsylvania State University.

McKeachie, Wilbert J. 1994. Teaching Tips. Lexington, MA: D.C. Heath and Company.
 
Reineke, Robert. 1995. "Worthen & Sanders: Six General Evaluation Models/Frameworks," Program Evaluation in Adult Education and Training, AEE 891. East Lansing: Michigan State University.