A critical view to
assessment of Web-based instruction
Metaphor of medicial treatment
- Once upon a time a man got a cold and visited a doctor. The doctor gave him some cold medicine, of course. A few days later, the patient was recovered. The physician claimed that his treatment was effective.
- In a scientific viewpoint, this claim is not valid. First, the doctor did not know whether the patient took the medicine or not. Second, even if the patient took the drug, he could not tell what effects the drug made to the patient's body. As a matter of fact, a cold could be healed without any intervention. To validate his claim, the doctor should sample and analyze the patient*s blood right after he took the medicine.
In my experience, today quite a few assessments of Web-based instruction have the same problem. The instructors deliver contents over the Web and then students performance are evaluated. A typical approach is to conduct a dependent t-test of pretest-posttest-difference. But even if performance gain is found, can we claim that the improvement is caused by the Web? This input-output evaluation should be replaced by an input-process-output evaluation. To check the effect of a drug, a doctor should track down the chemical reaction of the drug inside the patient's blood. By the same token, a cyber-teacher should analyze the usage of webpages by learners and correlated the user activities with the test scores. Frequent users of the Web may have better test performance. But what if infrequent Interent users also have high performance? If so, the change in performance may not be caused by the use of the Web.
Metaphor of magicial healing
- Once upon a time a magician handed a bottle of "holy water" to a patient and told him, "This bottle of water is not ordinate water. It possesses the divine power from God. If you have faith on it, it will heal your illness." The patient drank the water and his disease disappeared. The magician claimed that his "holy water" has healing power.
- Again, in a scientific viewpoint this claim has no merit. The recovery might be caused by the patient*s belief and thus he got what he expected to see. In psychology it is called "placebo effect" and in sociology it is called "self-fulfilling prophecy."
- Some assessments of Web-based instruction has the same potential flaw as the above. In their classic book "Experimental Design," Cook and Campbell explicitly warned researchers about this kind of possible bias of experiments. A well-thought researcher should hide the true intention of the experiment in order to avoid the subjects to behave in "expected" manner. However, not only some instructors do not "blind" the subjects, they "evangelize" that the Web is a wonderful and revolutionary learning tool. Hence, the performance increase of students may be contributed by motivational variables rather than technological variables. Nonetheless, this weakness is understandable. It is common that instructors are excited by new technologies.
- Further, some researchers are aware of this matter, but they do not want to lie to students such as saying "We deliver Web-based instruction because there are not enough parking lots on campus. So you should stay home to attend online classes." This involves an ethical issue.
Table of Contents