Let's say that you have decided to take a course in introductory psychology but are not sure which instructor to choose. You probably want an instructor who presents the material in an interesting and understandable way, but also one who does not make unreasonable academic demands on students. How would you go about making a decision? If you are like most people, you probably will ask friends or acquaintances who have taken the course before. Let's say that one of these people warns you "not to take Dr. Flanagan because she's really boring and way too hard." It's very likely that you will give this comment some weight. But should you?

As stated in Critical Thinking Lesson 1, whenever we are trying to answer a question, such as deciding which instructor to choose for a course, we must be empirical: we must make direct observations that will allow us to choose among various possible answers to the question. But, if we want to choose the best answer, we shouldn't simply go out into the world and collect whatever observations happen our way. Instead, we need to make systematic observations--observations that are performed in an orderly way according to a plan. For example, in trying to decide which instructor to choose, it would help a great deal if you could obtain student-evaluation records collected by the school for each instructor. In this way, you could see how students evaluated each instructor, on average, with respect to a number of characteristics. These observations would be much more systematic than accidentally running into a few students and asking them about instructors they have had. In the Critical Thinking Application at the end of Chapter 2, it was stated that the "critical thinker understands the need to seek sound evidence to guide decisions in everyday life" (p. 71). In order for one's evidence to be sound, one needs to make observations in a systematic manner.

Making Systematic Observations

Most of you probably have wondered how much you need to study in order to do well in your courses. Some of you may have heard about the rule-of-thumb that states that you should study two hours outside of class for every hour you spend in class (a "rule-of-thumb" tells you, in an approximate way, how to achieve a a particular goal). We'll refer to this as the "2-for-1 Rule." According to this rule, if you go to class for three hours every week, you should study six hours outside of class during that week. But is there any evidence that confirms (or disconfirms) the 2-for-1 Rule?

Perhaps you decide to test the rule by remembering courses you have taken in the past. You might remember that you took an American history course last semester and received an "A" even though you rarely opened the textbook. Instead, you simply listened carefully in class and took good notes, which you reviewed the morning of the tests. In fact, you now recall that you received all A's and B's last semester without studying outside of class very much at all.

Do these observations show that the 2-for-1 Rule is wrong? Not necessarily. It could be that the particular courses you took last semester were not very demanding. Perhaps most college courses would require much more studying outside of class than these courses did. Or it could be that you are misremembering how much you actually did study for your courses. Memories tend to fade with time. In other words, you were not making systematic observations when you simply tried to recall what happened in a few courses that you took last semester. But what exactly does it mean to say that we are observing in a systematic manner?

In order to answer this question, let's imagine that a group of researchers wanted to determine how important the number of hours spent studying is for test scores by recruiting 80 students to take a week-long course that met every day for one hour, with a test on the last day (Friday). Thus, the students spent a total of four hours in class receiving instruction (Monday through Thursday). The researchers split the students into four groups (20 students in each group), and asked each group to study a different number of hours for the test according to the following schedule (adapted from Goodwin, 1995, pp. 135-36).

Is this a reasonable conclusion to make? Although it may seem as if the researchers made systematic observations that supported their conclusion, you may have noticed a problem with the study. The four groups of students differed not only in the total number of hours that they studied, but also in the number of days between the last time that they studied and the time that they took the test (which is called the retention interval). Because of this, the researchers could not know if the test-score differences observed among the groups were due to the different amounts of time spent studying, to the different retention intervals, or to both.

When scientists make systematic observations, they attempt to control for the effects of various factors (a factor is an event or condition that causes something to occur) that would make it difficult to reach a firm conclusion as to what happened. In the example above, the researchers were unable to make a firm conclusion about the effect on test scores of the factor they were investigating--the amount of time spent studying--because they did not control for the effect of a second factor--retention interval. When we control a research situation, we attempt to regulate the research situation in such a way that we can exclude the effects of all factors but one. In other words, when we exert control, we want to be left with only one possible explanation for the results of a study. In this example, however, there are three possible explanations for the results, none of which can be ruled out:

1. Spending more time studying causes higher test scores.
2. Studying closer to the time of a test causes higher test scores.
3. Spending more time studying and studying closer to the time of a test together cause higher test scores.

In order to make systematic observations in this study, the researchers needed to control for the extraneous variable of retention interval.

Controlling for the Effects of Extraneous Variables

A variable is an event, situation, or condition that takes on different values that can be measured. The factors mentioned in the research example are variables because they represent units of time that can differ (for instance, you can study 8 hours, 6 hours, 4 hours, and so on). Eye color is a variable since there is a large number of different eye colors that can be measured. Height also is a variable since individuals can range from very short to very tall. On the other hand, noses would not represent a variable since everyone has only one nose.

In the study described above, the researchers were trying to determine the causal effects of number of study hours (the first variable) on test scores (the second variable). They were not able to do this, however, because they hadn't controlled for the effects of retention interval, which represents an "extraneous variable" in this study. As you learned in Chapter 2 of the textbook, an extraneous variable is a variable, other than the one being investigated, that also may be having causal effects on a second variable.

If we want to reach a firm conclusion about the causal effects of one variable on another, we must control for the effects of extraneous variables. How could we have controlled for the effects of retention interval in the study described above? Perhaps we could have had the groups study according to the following schedule:

Anecdotes tell of events in which there was no attempt to control for the effects of important extraneous variables on the phenomenon being discussed. In the hypnotherapy example, your friend also may have started a rigorous exercise program at the same time that he received the hypnotherapy treatments. Or, it could have been that his belief in the power of hypnosis finally gave him the motivation he needed to stick to a diet-and-exercise program (which is related to the "placebo effect"; see Critical Thinking Lesson 2B). So, although anecdotes often seem to be very compelling evidence, they do not include systematic observations and, thus, cannot be used to justify a particular claim.

CRITICAL THINKING QUESTIONS FOR LESSON 2A

Question 2A-1
The "2-for-1 rule" was not specifically tested in the example presented above. Instead, the study examined only the question of whether or not more time spent studying led to higher test scores. Design a study testing the "2-for-1 rule" against other possible rules of thumb--a study that attempts to control for the effects of important extraneous variables.
Suggested Answer

Question 2A-2
The United States Census Bureau (2002) reported the following annual average earnings for people 25 years and older who had completed various years of schooling:

The table shows that people with more education make more money, on average, than people with less education. Some people have used this fact to argue that, if you want a job that pays good money, you need to get as much education as you possibly can. In other words, they are arguing that this table shows that one variable, number of years of education, has a causal effect on a second variable, average annual earnings. What are some extraneous variables that would need to be controlled for before one could make this argument?
Suggested Answer

Question 2A-3
For each of the following, think of an uncontrolled extraneous variable that may have led to the reported results.

(a) Some people have argued that being President of the United States is such a stressful job that it causes a noticeable physical deterioration of the body over time. Evidence that the severe stress associated with the position is causing bodily changes can be seen by comparing photographs of presidents taken near the beginning of their time in office with photographs taken near the end of their time in office. The former photographs typically show an energetic and youthful-looking person, whereas the latter photographs typically show a more worn-out, haggard, and tired-looking individual. Upon looking at these "before" and "after" photographs, the claim that stress has caused bodily changes in these men seems compelling.
Suggested Answer

(b) My cat took part in a double-blind, placebo-controlled study of a new kidney medication. We did not know whether she was taking a medication thought to improve kidney functioning or a placebo that should have no effect on kidney functioning. On the first day of the study, her blood pressure was taken, which was discovered to be very high. But a few weeks after the study had begun, her blood pressure had decreased to a normal level, and remained at this level thereafter. The veterinarian told us that our cat must be taking the kidney medication and not the placebo because her blood pressure had decreased so dramatically.
Suggested Answer

(c) A horse was discovered in the early twentieth century that was able to add numbers and tap out the correct response with his hoof. If you asked him the answer to, say, 4 + 2, he would tap his foot six times. He could do this even when his owner was asked to leave the room, so that fraud on the part of the owner was ruled out. Is it any wonder that this horse was referred to as "Clever Hans"? The only possible explanation seemed to be that Hans was able to understand the concept of numbers and to add them together in his head.
Suggested Answer

(d) An experimental study examined the speed with which rats ran through a maze (McGuigan, 1997, pp. 74-75). The researcher predicted that the rats in the experimental group, which received a particular type of training thought to improve maze running, would run through the maze faster than the rats in the control group, which did not receive the training. In selecting animals for each group before beginning the experiment, the researcher reached into the cages that contained the rats and placed the ones that ran into his hands into the experimental group. The rats that were left in the cage were placed into the control group. The researcher found that, after receiving the training, the rats in the experimental group did indeed run through the maze faster than did the rats in the control group. He concluded that the particular type of training used increased the speed with which rats will run through a maze.
Suggested Answer

Question 2-4
In most classes that I have taught, the students who sit in the center seats in the first few rows tend to do better on tests than do the students who sit in other areas of the room. Let's say that a student who is failing one of my classes comes to me and asks what he can do to improve his grade, and I answer: "Your grade probably will improve if you sit in the center seat of the first row." Would this be a reasonable suggestion to make? Why or why not?
Suggested Answer

Bibliography and References

Goodwin, C. J. (1995). Research in psychology: Methods and design. New York: Wiley & Sons.

McGuigan, F. J. (1997). Experimental psychology: Methods of research (7th ed.). Upper Saddle River, NJ: Prentice Hall.

Pfungst, O. (1911/1965). Clever Hans (the horse of Mr. Von Osteen): A contribution to experimental, animal, and human psychology. Translated by C. L. Rahn. New York: Holt, Rinehart, and Winston. Originally published in 1911.

Ricker, J. P. (2002). An introduction to the science of psychology. Boston: Pearson Custom Publishing.

Wozniak, R. H. (1999). Oskar Pfungst: Clever Hans (The Horse of Mr. von Osten) (1907; English 1911). Retrieved April 19, 2003, from http://www.thoemmes.com/psych/pfungst.htm