Since evaluation helps find the deficiencies in curriculum design in order to restore them, it is considered as one of vital parts of a fine curriculum design. Evaluation provides the right information to do adjustments in favor of changing needs and environment. That’s why a well-planned evaluation allows teachers to make progress professionally, feeling that they have the complete control of the course.
To check whether the course is flawless, should the most extensive evaluation be applied to cover every aspect of curriculum design. In evaluating, both planning and running a course and results of it, are necessary to be looked at. However, most evaluations in reality are focused more narrowly and might answer the following questions:
- Is the course satisfactory from learners' view?
- Does the course be taught considering highly suitable standards?
- Will the learners receive proper education to obtain the use of English they want by the end of the course (for example to be able to study in an English-medium university, to be able to work as a tour leader, to pass the IELTS test)?
- Considering its cost, is the course gainful?
Following are all the early steps for setting evaluation goals at whether evaluation is possible to carry out and why it's being done:
- Figure out for whom is the evaluation and what sort of information they are after.
- In order to make decisions, it's necessary to figure how to use evaluation results (e.g. whether to apply improvements and keep the course or get rid of it).
- Find if the money and time are available to carry out the evaluation.
- Is it necessary to do evaluation or the required information is already obtainable?
- Determine what sorts of information will be assembled.
- To whom it's satisfactory: learner, teacher or sponsor.
- If the course is financially profitable.
- Amount of learning
- The qualities that need to be taken to consideration: quality of curriculum design, learning, teaching, course administration.
- Amount of learning
- Later success of those who graduate from the course.
- Decide who will involve in gathering the information and how it will be gathered.
- Decide how the findings should be presented.
- To get the implementation of the findings checked, decide whether it's necessary to plan a follow-up evaluation.
- Don't forget that obtaining the support of those people who are involved in the evaluation is important
A useful three-way scope is distinguished by Kiely and Rea-Dickens (2006: 225–271): (1) evaluations that are led by teachers (2) evaluations which are led by managers and (3) in order to concentrate on major educational innovations that have underlying agenda and extensive financial backing, large-scale evaluation is used. Evaluating a course can be a time-consuming and expensive procedure. for example, to gather an evaluation team for a university department, some outside evaluators were also involved. On the other hand, paying their accommodation and travel expenses plus a fee for their services had to be considered. It is of great importance to organize a well-motivated and well-focused evaluation since such a huge investment of money and time is proposed.
Most vital works in an evaluation is done before beginning to collect the data. The first crucial step to take is to find out for whom is the evaluation and they value what sort of information. Several reasons are provided to clarify the importance of this step: first, it helps settle the grade of confidentiality of the evaluation; Whether the report of the evaluation is accessible to all involved or it only goes to the group or person who commissioned the evaluation.
Second, it helps determine to gather what kind of information. The group or person that commissions the evaluation may put great focus on learner satisfaction or on economic issues, or may think that these factors are irrelevant. In the primary stages of an evaluation, in order to make clear the aims and type of the data which are supposed to be obtained through evaluation, the evaluator should talk with the person who is commissioning it at length. An effective way to show the person who is commissioning the evaluation is to prepare a “mock” report based on false data. Those who are interested in commissioning an evaluation for a language course can simply include the teachers, the learners and the owner/director of the language center. These interested groups will value different kinds of evidence and will be having different attitude about what a “good” course is.
Third, to determine if the gathered data will be provided reluctantly or willingly, it is useful to know for whom is the evaluation.
Meanwhile, it is important to be aware of why the evaluation is being carried out. Is it done to guiding a decision (whether to get rid of the course or maintain it) or is it done for improving the course? This is where the evaluator ought to be most skeptical to figure whether there exists a hidden purpose to the evaluation which is not clarified to the evaluator. For example, is the actual goal of evaluation to make an excuse to dispose of a course, or to get rid of an unwanted staff member (the decision to dispose of it have already been made secretly)?
The evaluator ought to be able to tell these statements to the group/person commissioning the evaluation by the end of the preparatory stages of the evaluation:
- Whether it's possible to do the evaluation
- Is the evaluation worth doing?
- How much does it cost?
- How long it may take
- The evaluator is willing to do it or not
- What sort of evidence will be gathered by the evaluation?
On the basis of the purpose of the evaluation, two types of evaluations are to be introduced: summative evaluation and formative evaluation. A summative evaluation is for making a summary or judgement on the adequacy or quality of the course so it can be compared with previous summative evaluations or other courses, or judged as being up to a certain criterion or not. A formative evaluation has the intention of shaping or forming the course in order to make improvements. Basically, evaluators are able to use most data for either of these two purposes but we have to bear in mind that these different purposes may affect the way the conclusions are presented and when and what type of data should be gathered. The distinction of summative/formative is important in guiding the evaluator decide what sort of information will be most useful to collect, informing those who are the main focus of an evaluation about the purpose of it, and using the gathered information.
Classifying evaluation as formative or summative is one decision. Another decision is whether it should be short term or long term (Beretta, 1986a, b). Evaluations are mostly short term and can be conducted in few days. If long-term evaluation is planned to be a part of curriculum design, it will be done most economically. we will discuss it in this chapter later. However, in a short-term evaluation some critical features of a course can't be validly evaluated, such as learner achievement and quality of teaching. By watching just one or two lessons, a teacher's quality can't be assessed reliably or validly. The teacher could be anxious because of having a bad day or on the contrary, for doing great in evaluation, the lessons might be especially well prepared. Two or three isolated observation is not enough to evaluate a teacher's performance since teaching involves planning a program and carrying it through to its conclusion. Stenhouse (1975) emphasized the significance of “illuminative evaluation”. This kind of evaluation helps those who are involved understand what is happening in the program, so this means teachers have to be an active program evaluator.
Along with long term/short term and summative/formative, there's a third distinction and that is product/process (Long, 1984). On one hand, an evaluation can concentrate on the result or product of teaching or learning; on the other hand, it can focus on the process of teaching and learning. It is product observations of learning that views how much was learned and what was learned. To have a look at how learners are engaged in their tasks, the quality and quantity of the language used, and the quality of communication between the learners and the teacher and between themselves. Data provided by process and product observations are different but views of the same thing will hopefully come together.
At this point, the last set of distinctions would be: if the evaluation will include affective, cognitive and resource factors. Affective factors deal with attitudes and feelings of satisfaction. Questions typically would be: “does the teacher think the course is effective?”, “are the staff working well together or not?”, “are the learners satisfied with the course?”. Cognitive factors deal with teaching and learning and obtaining knowledge, and applying that knowledge after the course has finished. Some typical questions would be:
“How much has been learned?”,
“How much has been taught?”,
“Has the course enhanced learners' study or work performance?”. Resource factors care for profit, costs, quality and availability of teaching and learning resources like: classrooms, visual aids, books, computers, social services, tape recorders, library facilities and administrative support. Questions typically would be: “are the classrooms spacious?”, “is the library adequate for answering learners needs?”, “will the course make financial profit?”.
Snice this brief survey made it clear that running a full-scale evaluation could be a huge operation, the importance of deciding what the evaluation should concentrate on is clarified. Essentially, this decision should be based on the sort of information that is needed to reach the goal of evaluation, so that's why instead of having a large amount of data (which do not address the primary concerns of the evaluation), it's better to have a small amount of relevant data.
Achieving Support for Evaluation
Evaluating a course brings the weaknesses and strengths to light. However, what mainly causes concern are the weaknesses. Finding weaknesses means that something or someone is to be held responsible for the weaknesses and this is evidently a threatening situation. It is crucial to have honest data if an evaluation is expected to proceed effectively. That means it is vital for those involved in the evaluation, especially who are sources of information, to know that the evaluation is worth it and it's not threatening their job security and their “face”. This requires meeting with those involved and including them in the planning and undertaking the evaluation. As a consequence, some evaluations bring in a respected outsider who tries to achieve the agreement and collaboration of the staff in favor of doing the evaluation.
