Statistics Primer

INTRODUCTION

THE DANGER OF SUCCESS IN MATH

DESCRIPTIVE STATISTICS
Data Reduction
Measures of Association
Regression Analysis

OTHER MULTIVARIATE TECHNIQUES
Path Analysis
Time-Series Analysis
Factor Analysis

INFERENTIAL STATISTICS
Univariate Inferences
Tests of Statistical Significance
The Logic of Statistical Significance
Chi Square

MAIN POINTS

KEY TERMS

REVIEW QUESTIONS AND EXERCISES

ADDITIONAL READINGS

*Thanks to the many contributors to humor lists on the Internet.

Many people are intimidated by quantitative research because they feel uncomfortable with mathematics and statistics. And indeed, many research reports are filled with unspecified computations. The role of statistics in social research is often important, but it is equally important to see this role in its proper perspective.

Empirical research is first and foremost a logical rather than a mathematical operation. Mathematics is merely a convenient and efficient language for accomplishing the logical operations inherent in quantitative data analysis. Statistics is the applied branch of mathematics especially appropriate to a variety of research analyses.

This discussion begins with an informal look at one of the concerns many people have when they approach statistics. I hope this exercise will make it easier to understand and feel comfortable with the relatively simple statistics introduced in the remainder of the discussion. We'll be looking at two types of statistics descriptive and inferential. Descriptive statistics is a medium for describing data in manageable forms. Inferential statistics, on the other hand, assists researchers in drawing conclusions from their observations; typically, this involves drawing conclusions about a population from the study of a sample drawn from it.

THE DANGER OF SUCCESS IN MATH
Since I began teaching research methods that include at least a small amount of statistics, I've been struck by the large number of students who report that they are "simply no good at math." Just as some people are reported to be inherently tone-deaf and others unable to learn foreign languages, about 90 percent of my college students have seemed to suffer from congenital math deficiency syndrome (CMDS). Its most common symptoms are frustration, boredom, and drowsiness. I'm delighted to report that I have finally uncovered a major cause of the disease and have brewed up a cure. In the event that you may be a sufferer, I'd like to share it with you before we delve into the statistics of social research.

You may be familiar with the story of Typhoid Mary, whose real name was Mary Mallon. Mary was a typhoid carrier who died in 1938 in New York. Before her death, she worked as a household cook, moving from household to household and causing ten outbreaks of typhoid fever. Over 50 people caught the disease from her, and 3 of them died.

The congenital math deficiency syndrome has a similar cause. After an exhaustive search, I've discovered the culprit, whom I'll call Mathematical Marvin, though he has used countless aliases. If you suffer from CMDS, I suspect you've met him. Take a minute to recall your years in high school. Remember the person your teachers and your classmates regarded as a "mathematical genius." Getting A's in all the math classes was only part of it; often the math genius seemed to know math better than the teachers did.

Now that you have that math genius in mind, let me ask you a few questions. First, what was the person's gender? I'd guess he was probably male. Most of the students I've asked in class report that. But let's consider some other characteristics

I've been asking students (including some in adult classes) these questions for several years, and the answers I've gotten are amazing. Marvin is usually unathletic, often either very skinny or overweight. He usually wears glasses, and he seems otherwise rather delicate. During his high school years, he was invited to an average (mean) of 1.2 parties, and nobody talked to him. His complexion was terrible. Almost nobody ever wanted to change places with him; he was a social misfit, to be pitied rather than envied.

As I've discussed Marvin with my students, it has become increasingly clear that most of them have formed a subconscious association between mathematical proficiency and Marvin's unenviable characteristics. Most have concluded that doing well in math and statistics would turn them into social misfits, which they regard as too high a price to pay.

Everything I've said about Mathematical Marvin represents a powerful stereotype that many people still seem to share, but the fact is that it's only a social stereotype, not a matter of biology. Women can excel in mathematics; attractive people can calculate as accurately as unattractive ones. Yet the stereotype exercises a powerful influence on our behavior, as evidenced in the tragic examples of young women, for example, pretending mathematical impotence in the belief that they will be seen as less attractive if they're gifted in that realm.
So if you're one of those people who's "just no good at math," it's possible you carry around a hidden fear that your face will break out in pimples if you do well in statistics in this course. If so, you're going to be reading the rest of this discussion in a terrible state wanting to understand it at least until the next exam and, at the same time, worrying that you may understand it too well and lose all your friends.

There is no cause for concern. The level of statistics contained in the rest of this discussion has been proved safe for humans. There has not been a single documented case of pimples connected to understanding lambda, gamma, chi square, or any of the other statistics discussed in the pages that follow. In fact, this level of exposure has been found to be beneficial to young social researchers.

By the way, uncovering Marvin can clear up a lot of mysteries. It did for me. (In my high school class, he didn't wear glasses, but he squinted a lot.) In the first research methods book I wrote, I presented three statistical computations and got one of them wrong. In the first edition of this book, I got a different one wrong. Most embarrassing of all, however, the first printing of the earlier book had a unique feature. I thought it would be fun to write a computer program to generate my own table of random numbers rather than reprinting one that someone else had created. In doing that, I had the dubious honor of publishing the world's first table of random numbers that didn't have any nines! It was not until I tracked Marvin down that I discovered the source of my problems, and statistics has been much more fun (and trouble-free) ever since. So enjoy.

DESCRIPTIVE STATISTICS
As I've already suggested, descriptive statistics present quantitative descriptions in a manageable form. Sometimes we want to describe single variables, and sometimes we want to describe the associations that connect one variable with another. Let's look at some of the ways to do these things.

Data Reduction
-no indent Scientific research often involves collecting large masses of data. Suppose we surveyed 2,000 people, asking each of them 100 questions, not an unusually large study. We would then have a staggering 200,000 answers! No one could possibly read all those answers and reach any meaningful conclusion about them. Thus, much scientific analysis involves the reduction of data from unmanageable details to manageable summaries.
To begin our discussion, let's look briefly at the raw data matrix created by a quantitative research project. Table 17-1 presents a partial data matrix. Notice that each row in the matrix represents a person (or other unit of analysis), each column represents a variable, and each cell represents the coded attribute or value a given person has on a given variable. The first column in Table 17-1 represents a person's gender. Let's say a "1" represents male and a "2" represents female. This means that persons 1 and 2 are male, person 3 is female, and so forth.
**[Table 17-1 about here; pickup from 8e p. 407]**

In the case of age, person 1's "3" might mean 30?39 years old, person 2's "4" might mean 40-49. However age has been coded, the code numbers shown in Table 17-1 describe each of the people represented there.

Notice that the data have already been reduced somewhat by the time a data matrix like this one has been created. If age has been coded as suggested previously, the specific answer "33 years old" has already been assigned to the category "30-39." The people responding to our survey may have given us 60 or 70 different ages, but we have now reduced them to 6 or 7 categories.

Measures of Association
-no indent The association between any two variables can also be represented by a data matrix, this time produced by the joint frequency distributions of the two variables. Table 17-2 presents such a matrix. It provides all the information needed to determine the nature and extent of the relationship between education and prejudice.
**[Table 17-2 about here; pickup from 8e p. 407]**

Notice, for example, that 23 people (1) have no education and (2) scored high on prejudice; 77 people (1) had graduate degrees and (2) scored low on prejudice.

Like the raw-data matrix in Table 17-1, this matrix provides more information than can easily be comprehended. A careful study of the table shows that as education increases from "None" to "Graduate Degree," there is a general tendency for prejudice to decrease, but no more than a general impression is possible. For a more precise summary of the data matrix, we need one of several types of descriptive statistics. Selecting the appropriate measure depends initially on the nature of the two variables.

We'll turn now to some of the options available for summarizing the association between two variables. Each of these measures of association is based on the same model, proportionate reduction of error (PRE).

To see how this model works, let's assume that I asked you to guess respondents' attributes on a given variable for example, whether they answered yes or no to a given questionnaire item. To assist you, let's first assume you know the overall distribution of responses in the total sample, say, 60 percent said yes and 40 percent said no. You would make the fewest errors in this process if you always guessed the modal (most frequent) response yes.
Second, let's assume you also know the empirical relationship between the first variable and some other variable say, gender. Now, each time I ask you to guess whether a respondent said yes or no, I'll tell you whether the respondent is a man or a woman. If the two variables are related, you should make fewer errors the second time. It's possible, therefore, to compute the PRE by knowing the relationship between the two variables the greater the relationship, the greater the reduction of error.

This basic PRE model is modified slightly to take account of different levels of measurement, nominal, ordinal, or interval. The following sections will consider each level of measurement and present one measure of association appropriate to each. Bear in mind, though, that the three measures discussed are only an arbitrary selection from among many appropriate measures.

Nominal Variables: If the two variables consist of nominal data (for example, gender, religious affiliation, race), lambda (l) would be one appropriate measure. (Lambda is a letter in the Greek alphabet corresponding to l in our alphabet. Greek letters are used for many concepts in statistics, which perhaps helps to account for the number of people who say of statistics, "It's all Greek to me.") Lambda is based on your ability to guess values on one of the variables the PRE achieved through knowledge of values on the other variable.

Imagine this situation. I tell you that a room contains 100 people and I would like you to guess the gender of each person, one at a time. If half are men and half women, you will probably be right half the time and wrong half the time. But suppose I tell you each person's occupation before you guess that person's gender.

What gender would you guess if I said the person was a truck driver? Probably you would be wise to guess "male"; although there are now plenty of women truck drivers, most are still men. If I said the next person was a nurse, you'd probably be wisest to guess "female," following the same logic. While you would still make errors in guessing genders, you would clearly do better than you would if you didn't know their occupations. The extent to which you did better (the proportionate reduction of error) would be an indicator of the association that exists between gender and occupation.

Here's another simple hypothetical example that illustrates the logic and method of lambda. Table 17-3 presents hypothetical data relating gender to employment status. Overall, we note that 1,100 people are employed, and 900 are not employed. If you were to predict whether people were employed, knowing only the overall distribution on that variable, you would always predict "employed," since that would result in fewer errors than always predicting "not employed." Nevertheless, this strategy would result in 900 errors out of 2,000 predictions.
**[Table 17-3 about here; pickup from 4e p. 408]**

Let's suppose that you had access to the data in Table 17-3 and that you were told each person's gender before making your prediction of employment status. Your strategy would change in that case. For every man, you would predict "employed," and for every woman, you would predict "not employed." In this instance, you would make 300 errors, the 100 men who were not employed and the 200 employed women, or 600 fewer errors than you would make without knowing the person's gender.

Lambda, then, represents the reduction in errors as a proportion of the errors that would have been made on the basis of the overall distribution. In this hypothetical example, lambda would equal .67; that is, 600 fewer errors divided by the 900 total errors based on employment status alone. In this fashion, lambda measures the statistical association between gender and employment status.

If gender and employment status were statistically independent, we would find the same distribution of employment status for men and women. In this case, knowing each person's gender would not affect the number of errors made in predicting employment status, and the resulting lambda would be zero. If, on the other hand, all men were employed and none of the women were employed, by knowing gender you would avoid errors in predicting employment status. You would make 900 fewer errors (out of 900), so lambda would be 1.0, representing a perfect statistical association.

Lambda is only one of several measures of association appropriate to the analysis of two nominal variables. You could look at any statistics textbook for a discussion of other appropriate measures.

Ordinal Variables: If the variables being related are ordinal (for example, social class, religiosity, alienation), gamma (g) is one appropriate measure of association. Like lambda, gamma is based on our ability to guess values on one variable by knowing values on another. However, whereas lambda is based on guessing exact values, gamma is based on guessing the ordinal arrangement of values. For any given pair of cases, we guess that their ordinal ranking on one variable will correspond (positively or negatively) to their ordinal ranking on the other.

Let's say we have a group of elementary students. It's reasonable to assume that there is a relationship between their ages and their heights. We can test this by comparing every pair of students Sam and Mary, Sam and Fred, Mary and Fred, and so forth. Then we ignore all the pairs in which the students are the same age and/or the same height. We then classify each of the remaining pairs (those who differ in both age and height) into one of two categories those in which the older child is also the taller ("same" pairs) and those in which the older child is the shorter ("opposite" pairs). So, if Sam is older and taller than Mary, the Sam-Mary pair is counted as a "same." If Sam is older but shorter than Mary, then that pair is an "opposite." (If they're the same age and/or same height, we ignore them.)

To determine whether age and height are related to one another, we compare the number of same and opposite pairs. If the same pairs outnumber the opposite pairs, we can conclude that there is a positive association between the two variables, as one increases, the other increases. If there are more opposites than sames, we can conclude that the relationship is negative. If there are about as many sames as opposites, we can conclude that age and height are not related to each another, that they're independent of each other.

Here's a social science example to illustrate the simple calculations involved in gamma. Let's say you suspect that religiosity is positively related to political conservatism, and if Person A is more religious than Person B, you guess that A is also more conservative than B. Gamma is the proportion of paired comparisons that fits this pattern.

Table 17-4 presents hypothetical data relating social class to prejudice. The general nature of the relationship between these two variables is that as social class increases, prejudice decreases. There is a negative association between social class and prejudice.
**[Table 17-4 about here; pickup from8e p. 409]**

Gamma is computed from two quantities (1) the number of pairs having the same ranking on the two variables and (2) the number of pairs having the opposite ranking on the two variables. The pairs having the same ranking are computed as follows. The frequency of each cell in the table is multiplied by the sum of all cells appearing below and to the right of it, with all these products being summed. In Table 17-4, the number of pairs with the same ranking would be 200(900 + 300 + 400 + 100) + 500(300 + 100) + 400(400 + 100) + 900(100), or 340,000 + 200,000 + 200,000 + 90,000 = 830,000.

The pairs having the opposite ranking on the two variables are computed as follows The frequency of each cell in the table is multiplied by the sum of all cells appearing below and to the left of it, with all these products being summed. In Table 17-4, the numbers of pairs with opposite rankings would be 700(500 + 800 + 900 + 300) + 400(800 + 300) + 400(500 + 800) + 900(800), or 1,750,000 + 440,000 + 520,000 + 720,000 = 3,430,000. Gamma is computed from the numbers of same-ranked pairs and opposite-ranked pairs as follows **[Set as in 8e p. 410]**
 same ? opposite
 gamma =  ---------------------
  same + opposite

In our example, gamma equals (830,000 ? 3,430,000) divided by (830,000 + 3,430,000), or ?.61. The negative sign in this answer indicates the negative association suggested by the initial inspection of the table. Social class and prejudice, in this hypothetical example, are negatively associated with each other. The numerical figure for gamma indicates that 61 percent more of the pairs examined had the opposite ranking than the same ranking.
Note that whereas values of lambda vary from 0 to 1, values of gamma vary from ?1 to +1, representing the direction as well as the magnitude of the association. Because nominal variables have no ordinal structure, it makes no sense to speak of the direction of the relationship. (A negative lambda would indicate that you made more errors in predicting values on one variable while knowing values on the second than you made in ignorance of the second, and that's not logically possible.)

Table 17-5 is an example of the use of gamma in social research. To study the extent to which widows sanctified their deceased husbands, Helena Znaniecki Lopata (1981) administered a questionnaire to a probability sample of 301 widows. In part, the questionnaire asked the respondents to characterize their deceased husbands in terms of the following semantic differentiation scale **[set as in 8e p. 410]**

Characteristic
Positive Extreme Negative Extreme
Good 1 2 3 4 5 6 7 Bad
Useful 1 2 3 4 5 6 7 Useless
Honest 1 2 3 4 5 6 7 Dishonest
Superior 1 2 3 4 5 6 7 Inferior
Kind 1 2 3 4 5 6 7 Cruel
Friendly 1 2 3 4 5 6 7 Unfriendly
Warm 1 2 3 4 5 6 7 Cold
**[Table 17-5 about here; pickup from 8e p. 410]**

Respondents were asked to describe their deceased spouses by circling a number for each pair of opposing characteristics. Notice that the series of numbers connecting each pair of characteristics is an ordinal measure.

Next, Lopata wanted to discover the extent to which the several measures were related to each other. Appropriately, she chose gamma as the measure of association. Table 17-5 shows how she presented the results of her investigation.

The format presented in Table 17-5 is called a correlation matrix. For each pair of measures, Lopata has calculated the gamma. Good and Useful, for example, are related to each other by a gamma equal to .79. The matrix is a convenient way of presenting the intercorrelations among several variables, and you'll find it frequently in the research literature. In this case, we see that all the variables are quite strongly related to each other, though some pairs are more strongly related than others.

Gamma is only one of several measures of association appropriate to ordinal variables. Again, any introductory statistics textbook will give you a more comprehensive treatment of this subject.

Interval or Ratio Variables: If interval or ratio variables (for example, age, income, grade point average, and so forth) are being associated, one appropriate measure of association is Pearson's product-moment correlation (r). The derivation and computation of this measure of association are complex enough to lie outside the scope of this book, so I'll make only a few general comments here.

Like both gamma and lambda, r is based on guessing the value of one variable by knowing the other. For continuous interval or ratio variables, however, it is unlikely that you could predict the precise value of the variable. But on the other hand, predicting only the ordinal arrangement of values on the two variables would not take advantage of the greater amount of information conveyed by an interval or ratio variable. In a sense, r reflects how closely you can guess the value of one variable through your knowledge of the value of the other.

To understand the logic of r, consider the way you might hypothetically guess values that particular cases have on a given variable. With nominal variables, we've seen that you might always guess the modal value. But for interval or ratio data, you would minimize your errors by always guessing the mean value of the variable. Although this practice produces few if any perfect guesses, the extent of your errors will be minimized. Imagine the task of guessing peoples' incomes and how much better you would do if you knew how many years of education they had as well as the mean incomes for people with 0, 1, 2 (and so forth) years of education.

In the computation of lambda, we noted the number of errors produced by always guessing the modal value. In the case of r, errors are measured in terms of the sum of the squared differences between the actual value and the mean. This sum is called the total variation.
To understand this concept, we must expand the scope of our examination. Let's look at the logic of regression analysis and discuss to correlation within that context.

Regression Analysis
At several points in this text, I have referred to the general formula for describing the association between two variables Y = f(X). This formula is read "Y is a function of X," meaning that values of Y can be explained in terms of variations in the values of X. Stated more strongly, we might say that X causes Y, so the value of X determines the value of Y. Regression analysis is a method of determining the specific function relating Y to X. There are several forms of regression analysis, depending on the complexity of the relationships being studied. Let's begin with the simplest.

Linear Regression: The regression model can be seen most clearly in the case of a linear regression analysis, where there is a perfect linear association between two variables. Figure 17-1 is a scattergram presenting in graphic form the values of X and Y as produced by a hypothetical study. It shows that for the four cases in our study, the values of X and Y are identical in each instance. The case with a value of 1 on X also has a value of 1 on Y, and so forth. The relationship between the two variables in this instance is described by the equation Y = X; this is called the regression equation. Because all four points lie on a straight line, we could superimpose that line over the points; this is the regression line.
**[Figure 17-1 about here; pickup from 8e p. 411]**
Figure 17-1

Simple Scattergram of Values of X and Y
The linear regression model has important descriptive uses. The regression line offers a graphic picture of the association between X and Y, and the regression equation is an efficient form for summarizing that association. The regression model has inferential value as well. To the extent that the regression equation correctly describes the general association between the two variables, it may be used to predict other sets of values. If, for example, we know that a new case has a value of 3.5 on X, we can predict the value of 3.5 on Y as well.

In practice, of course, studies are seldom limited to four cases, and the associations between variables are seldom as clear as the one presented in Figure 17-1.

A somewhat more realistic example is presented in Figure 17-2, representing a hypothetical relationship between population and crime rate in small- to medium-sized cities. Each dot in the scattergram is a city, and its placement reflects that city's population and its crime rate. As was the case in our previous example, the values of Y (crime rates) generally correspond to those of X (populations), and as values of X increase, so do values of Y. However, the association is not nearly as clear as it is in Figure 17-1.
**[Figure 17-2 about here; pickup from 8e p. 412]**
Figure 17-2 A Scattergram of the Values of Two Variables with Regression Line Added (Hypothetical)

In Figure 17-2 we can't superimpose a straight line that will pass through all the points in the scattergram. But we can draw an approximate line showing the best possible linear representation of the several points. I've drawn that line on the graph.

You may (or may not) recall from algebra that any straight line on a graph can be represented by an equation of the form Y = a + bX, where X and Y are values of the two variables. In this equation, a equals the value of Y when X is 0, and b represents the slope of the line. If we know the values of a and b, we can calculate an estimate of Y for every value of X.

We can now say more formally that regression analysis is a technique for establishing the regression equation representing the geometric line that comes closest to the distribution of points on a graph. The regression equation provides a mathematical description of the relationship between the variables, and it allows us to infer values of Y when we have values of X. Recalling Figure 17-2, we could estimate crime rates of cities if we knew their populations.
To improve your guessing, you construct a regression line, stated in the form of a regression equation that permits the estimation of values on one variable from values on the other. The general format for this equation is Y ' = a + b(X), where a and b are computed values, X is a given value on one variable, and Y ' is the estimated value on the other. **[Y ' is Y prime, as in 4e p. 413]**The values of a and b are computed to minimize the differences between actual values of Y and the corresponding estimates (Y ') based on the known value of X. The sum of squared differences between actual and estimated values of Y is called the unexplained variation because it represents errors that still exist even when estimates are based on known values of X.

The explained variation is the difference between the total variation and the unexplained variation. Dividing the explained variation by the total variation produces a measure of the proportionate reduction of error corresponding to the similar quantity in the computation of lambda. In the present case, this quantity is the correlation squared r2. Thus, if r = .7, then r2 = .49, meaning that about half the variation has been explained. In practice, we compute r rather than r2, because the product-moment correlation can take either a positive or a negative sign, depending on the direction of the relationship between the two variables. (Computing r2 and taking a square root would always produce a positive quantity.) You can consult any standard statistics textbook for the method of computing r, although I anticipate that most readers using this measure will have access to computer programs designed for this function.
Unfortunately, or perhaps fortunately, social life is so complex that the simple linear regression model often does not sufficiently represent the state of affairs. It's possible, using percentage tables, to analyze more than two variables. As the number of variables increases, such tables become increasingly complicated and hard to read. But the regression model offers a useful alternative in such cases.

Multiple Regression: Very often, social researchers find that a given dependent variable is affected simultaneously by several independent variables. Multiple regression analysis provides a means of analyzing such situations. This was the case when Beverly Yerg (1981) set about studying teacher effectiveness in physical education. She stated her expectations in the form of a multiple regression equation
**[Set as in 8e p. 413; see addition of comma and "where" after first equation]**
@EX F = b0 + b1I + b2X1 + b3X2 + b4X3 + b5X4 + e, where
F  = Final pupil-performance score
I  = Initial pupil-performance score
X1 = Composite of guiding and supporting practice
X2 = Composite of teacher mastery of content
X3 = Composite of providing specific, task-related feedback
X4 = Composite of clear, concise task presentation
b  = Regression weight
e  = Residual

(ADAPTED FROM YERG 1981 42)
Notice that in place of the single X variable in a linear regression, there are several X's, and there are also several b's instead of just one. Also, Yerg has chosen to represent a as b0 in this equation but with the same meaning as discussed previously. Finally, the equation ends with a residual factor (e), which represents the variance in Y that is not accounted for by the X variables analyzed.

Beginning with this equation, Yerg calculated the values of the several b's to show the relative contributions of the several independent variables in determining final student-performance scores. She also calculated the multiple-correlation coefficient as an indicator of the extent to which all six variables predict the final scores. This follows the same logic as the simple bivariate correlation discussed earlier, and it is traditionally reported as a capital R. In this case, R = .877, meaning that 77 percent of the variance (.8772= .77) in final scores is explained by the six variables acting in concert.

Partial Regression: In exploring the elaboration model in your textbook, we paid special attention to the relationship between two variables when a third test variable was held constant. Thus, we might examine the effect of education on prejudice with age held constant, testing the independent effect of education. To do so, we would compute the tabular relationship between education and prejudice separately for each age group.

Partial regression analysis is based on this same logical model. The equation summarizing the relationship between variables is computed on the basis of the test variables remaining constant. As in the case of the elaboration model, the result may then be compared with the uncontrolled relationship between the two variables to clarify further the overall relationship.
Curvilinear Regression Up to now, we have been discussing the association among variables as represented by a straight line. The regression model is even more general than our discussion thus far has implied.

You may already know that curvilinear functions, as well as linear ones, can be represented by equations. For example, the equation X2 + Y2 = 25 describes a circle with a radius of 5. Raising variables to powers greater than 1 has the effect of producing curves rather than straight lines. And in the real world there is no reason to assume that the relationship among every set of variables will be linear. In some cases, then, curvilinear regression analysis can provide a better understanding of empirical relationships than can any linear model.
Recall, however, that a regression line serves two functions. It describes a set of empirical observations, and it provides a general model for making inferences about the relationship between two variables in the general population that the observations represent. A very complex equation might produce an erratic line that would indeed pass through every individual point. In this sense, it would perfectly describe the empirical observations. There would be no guarantee, however, that such a line could adequately predict new observations or that it in any meaningful way represented the relationship between the two variables in general. Thus, it would have little or no inferential value.

Earlier in this book, we discussed the need for balancing detail and utility in data reduction. Ultimately, researchers attempt to provide the most faithful, yet also the simplest, representation of their data. This practice also applies to regression analysis. Data should be presented in the simplest fashion (thus, linear regressions are most frequently used) that best describes the actual data. Curvilinear regression analysis adds a new option to the researcher in this regard, but it does not solve the problems altogether. Nothing does that.
Cautions in Regression Analysis The use of regression analysis for statistical inferences is based on the same assumptions made for correlational analysis simple random sampling, the absence of nonsampling errors, and continuous interval data. Because social scientific research seldom completely satisfies these assumptions, you should use caution in assessing the results in regression analyses.

Also, regression lines, linear or curvilinear, can be useful for interpolation (estimating cases lying between those observed), but they are less trustworthy when used for extrapolation (estimating cases that lie beyond the range of observations). This limitation on extrapolations is important in two ways. First, you are likely to come across regression equations that seem to make illogical predictions. An equation linking population and crimes, for example, might seem to suggest that small towns with, say, a population of 1,000 should produce 123 crimes a year. This failure in predictive ability does not disqualify the equation but dramatizes that its applicability is limited to a particular range of population sizes. Second, researchers sometimes overstep this limitation, drawing inferences that lie outside their range of observation, and you'd be right in criticizing them for that.

The preceding sections have introduced some of the techniques for measuring associations among variables at different levels of measurement. Matters become slightly more complex when the two variables represent different levels of measurement. Though we aren't going to pursue this issue in this textbook, the box "Measures of Association and Levels of Measurement," by Peter Nardi, may be a useful resource if you ever have to address such situations.
**[Box "Measures of Association and Levels of Measurement" about here; pickup from 8e p. 415]**

OTHER MULTIVARIATE TECHNIQUES
For the most part, this book has focused on rather rudimentary forms of data manipulation, such as the use of contingency tables and percentages. The elaboration model of analysis was presented in this form, as were the statistical techniques presented so far in this discussion.

This section of the discussion presents a cook's tour of three other multivariate techniques from the logical perspective of the elaboration model. This discussion isn't intended to teach you how to use these techniques but rather to present sufficient information so that you can understand them if you run across them in a research report. The three methods of analysis that we'll examine, path analysis, time-series analysis, and factor analysis, are only a few of the many multivariate techniques used by social scientists.

Path Analysis
Path analysis is a causal model for understanding relationships between variables. Though based on regression analysis, it can provide a more useful graphic picture of relationships among several variables than other means can. Path analysis assumes that the values of one variable are caused by the values of another, so it is essential to distinguish independent and dependent variables. This requirement is not unique to path analysis, of course, but path analysis provides a unique way of displaying explanatory results for interpretation.
Recall for a moment one of the ways I represented the elaboration model in your textbook. Here's how we might diagram the logic of interpretation **[Set the following as in 8e p. 416; lc "v"]**

Independent variable Æ Intervening variable Æ Dependent variable
The logic of this presentation is that an independent variable has an impact on an intervening variable, which, in turn, has an impact on a dependent variable. The path analyst constructs similar patterns of relationships among variables, but the typical path diagram contains many more variables than shown in this diagram.

Besides diagramming a network of relationships among variables, path analysis also shows the strengths of those several relationships. The strengths of relationships are calculated from a regression analysis that produces numbers analogous to the partial relationships in the elaboration model. These path coefficients, as they are called, represent the strengths of the relationships between pairs of variables, with the effects of all other variables in the model held constant.

The analysis in Figure 17-3, for example, focuses on the religious causes of anti-Semitism among Christian church members. The variables in the diagram are, from left to right, (1) orthodoxy, or the extent to which the subjects accept conventional beliefs about God, Jesus, biblical miracles, and so forth; (2) particularism, the belief that one's religion is the "only true faith"; (3) acceptance of the view that the Jews crucified Jesus; (4) religious hostility toward contemporary Jews, such as believing that God is punishing them or that they will suffer damnation unless they convert to Christianity; and (5) secular anti-Semitism, such as believing that Jews cheat in business, are disloyal to their country, and so forth.
**[Figure 17-3 about here; pickup from 8e p. 416]**
Figure 17-3: Diagramming the Religious Sources of Anti-Semitism

To start with, the researchers who conducted this analysis proposed that secular anti-Semitism was produced by moving through the five variables Orthodoxy caused particularism, which caused the view of the historical Jews as crucifiers, which caused religious hostility toward contemporary Jews, which resulted, finally, in secular anti-Semitism.

The path diagram tells a different story. The researchers found, for example, that belief in the historical role of Jews as the crucifiers of Jesus doesn't seem to matter in the process that generates anti-Semitism. And, although particularism is a part of one process resulting in secular anti-Semitism, the diagram also shows that anti-Semitism is created more directly by orthodoxy and religious hostility. Orthodoxy produces religious hostility even without particularism, and religious hostility generates secular hostility in any event.

One last comment on path analysis is in order. Although it is an excellent way of handling complex causal chains and networks of variables, path analysis itself does not tell the causal order of the variables. Nor was the path diagram generated by computer. The researcher decided the structure of relationships among the variables and used computer analysis merely to calculate the path coefficients that apply to such a structure.

Time-Series Analysis
The various forms of regression analysis are often used to examine time-series data, representing changes in one or more variables over time. As I'm sure you know, U.S. crime rates have generally increased over the years. A time-series analysis of crime rates could express the long-term trend in a regression format and provide a way of testing explanations for the trend, such as population growth or economic fluctuations, and could permit forecasting of future crime rates.

In a simple illustration, Figure 17-4 graphs the larceny rates of a hypothetical city over time. Each dot on the graph represents the number of larcenies reported to police during the year indicated.
**[Figure 17-4 about here; pickup from 8e p. 417]**
Figure 17-4: The Larceny Rates over Time in a Hypothetical City

Suppose we feel that larceny is partly a function of overpopulation. You might reason that crowding would lead to psychological stress and frustration, resulting in increased crimes of many sorts. Recalling the discussion of regression analysis, we could create a regression equation representing the relationship between larceny and population density, using the actual figures for each variable, with years as the units of analysis. Having created the best-fitting regression equation, we could then calculate a larceny rate for each year, based on that year's population density rate. For the sake of simplicity, let's assume that the city's population size (and hence density) has been steadily increasing. This would lead us to predict a steadily increasing larceny rate as well. These regression estimates are represented by the dashed regression line in Figure 17-4.

Time-series relationships are often more complex than this simple illustration suggests. For one thing, there can be more than one causal variable. For example, we might find that unemployment rates also had a powerful impact on larceny. We might develop an equation to predict larceny on the basis of both of these causal variables. As a result, the predictions might not fall along a simple, straight line. Whereas population density was increasing steadily in the first model, unemployment rates rise and fall. As a consequence, our predictions of the larceny rate would similarly go up and down.

Pursuing the relationship between larceny and unemployment rates, we might reason that people do not begin stealing as soon as they become unemployed. Typically, they might first exhaust their savings, borrow from friends, and keep hoping for work. Larceny would be a last resort.

Time-lagged regression analysis could be used to address this more complex case. Thus, we might create a regression equation that predicted a given year's larceny rate based, in part, on the previous year's unemployment rate or perhaps on an average of the two years' unemployment rates. The possibilities are endless.

If you think about it, a great many causal relationships are likely to involve a time lag. Historically, many of the world's poor countries have maintained their populations by matching high death rates with equally high birthrates. It has been observed repeatedly, moreover, that when a society's death rate is drastically reduced, through improved medical care, public sanitation, and improved agriculture, for example, that society's birthrate drops sometime later on, but with an intervening period of rapid population growth. Or, to take a very different example, a crackdown on speeding on a state's highways is likely to reduce the average speed of cars. Again, however, the causal relationship would undoubtedly involve a time lag, days, weeks, or months, perhaps, as motorists began to realize the seriousness of the crackdown.

In all such cases, the regression equations generated might take many forms. In any event, the criterion for judging success or failure is the extent to which the researcher can account for the actual values observed for the dependent variable.

Factor Analysis
Factor analysis is a unique approach to multivariate analysis. Its statistical basis is complex enough and different enough from the foregoing discussions to suggest a general discussion here.

Factor analysis is a complex algebraic method used to discover patterns among the variations in values of several variables. This is done essentially through the generation of artificial dimensions (factors) that correlate highly with several of the real variables and that are independent of one another. A computer must be used to perform this complex operation.
Let's suppose that a data file contains several indicators of subjects' prejudice. Each item should provide some indication of prejudice, but none will give a perfect indication. All of these items, moreover, should be highly intercorrelated empirically. In a factor analysis of the data, the researcher would create an artificial dimension that would be highly correlated with each of the items measuring prejudice. Each subject would essentially receive a value on that artificial dimension, and the value assigned would be a good predictor of the observed attributes on each item.

Suppose now that the same study provided several indicators of subjects' mathematical ability. It's likely that the factor analysis would also generate an artificial dimension highly correlated with each of those items.

The output of a factor analysis program consists of columns representing the several factors (artificial dimensions) generated from the observed relations among variables plus the correlations between each variable and each factor, called the factor loadings.
In the preceding example, it's likely that one factor would more or less represent prejudice, and another would more or less represent mathematical ability. Data items measuring prejudice would have high loadings on (correlations with) the prejudice factor and low loadings on the mathematical ability factor. Data items measuring mathematical ability would have just the opposite pattern.

In practice, factor analysis does not proceed in this fashion. Rather, the variables are input to the program, and the program outputs a series of factors with appropriate factor loadings. The analyst must then determine the meaning of a given factor on the basis of those variables that load highly on it. The program's generation of factors, however, has no reference to the meaning of variables, only to their empirical associations. Two criteria are taken into account (1) a factor must explain a relatively large portion of the variance found in the study variables, and (2) every factor must be more or less independent of every other factor.
Here's an example of the use of factor analysis. Many social researchers have studied the problem of delinquency. If you look deeply into the problem, however, you'll discover that there are many different types of delinquents. In a survey of high school students in a small Wyoming town, Morris Forslund (1980) set out to create a typology of delinquency. His questionnaire asked students to report whether they had committed a variety of delinquent acts. He then submitted their responses to factor analysis. The results are shown in Table 17-6.
**[Table 17-6 about here; pickup from 8e p. 419]**

As you can see in this table, the various delinquent acts are listed on the left. The numbers shown in the body of the table are the factor loadings on the four factors constructed in the analysis. You'll notice that after examining the dimensions, or factors, Forslund labeled them. I've bracketed the items on each factor that led to his choice of labels. Forslund summarized the results as follows

For the total sample four fairly distinct patterns of delinquent acts are apparent. In order of variance explained, they have been labeled 1) Property Offenses, including both vandalism and theft; 2) Incorrigibility; 3) Drugs/Truancy; and 4) Fighting. It is interesting, and perhaps surprising, to find both vandalism and theft appear together in the same factor. It would seem that those high school students who engage in property offenses tend to be involved in both vandalism and theft. It is also interesting to note that drugs, alcohol and truancy fall in the same factor.
(1980 4)

Having determined this overall pattern, Forslund reran the factor analysis separately for boys and for girls. Essentially the same patterns emerged in both cases.
This example shows that factor analysis is an efficient method of discovering predominant patterns among a large number of variables. Instead of being forced to compare countless correlations, simple, partial, and multiple, to discover those patterns, researchers can use factor analysis for this task. Incidentally, this is a good example of a helpful use of computers.
Factor analysis also presents data in a form that can be interpreted by the reader or researcher. For a given factor, the reader can easily discover the variables loading highly on it, thus noting clusters of variables. Or, the reader can easily discover which factors a given variable is or is not loaded highly on.

But factor analysis also has disadvantages. First, as noted previously, factors are generated without any regard to substantive meaning. Often researchers will find factors producing very high loadings for a group of substantively disparate variables. They might find, for example, that prejudice and religiosity have high positive loadings on a given factor, with education having an equally high negative loading. Surely the three variables are highly correlated, but what does the factor represent in the real world? All too often, inexperienced researchers will be led into naming such factors as "religio-prejudicial lack of education" or something similarly nonsensical.

Second, factor analysis is often criticized on basic philosophical grounds. Recall that to be useful, a hypothesis must be disprovable. If the researcher cannot specify the conditions under which the hypothesis would be disproved, the hypothesis is in reality either a tautology or useless. In a sense, factor analysis suffers this defect. No matter what data are input, factor analysis produces a solution in the form of factors. Thus, if the researcher were asking, "Are there any patterns among these variables?" the answer always would be yes. This fact must also be taken into account in evaluating the results of factor analysis. The generation of factors by no means ensures meaning.

My personal view of factor analysis is the same as that for other complex modes of analysis. It can be an extremely useful tool for the social science researcher. Its use should be encouraged whenever such activity may assist researchers in understanding a body of data. As in all cases, however, such tools are only tools and never magical solutions.
Let me reiterate that the analytical techniques we've touched on are only a few of the many techniques commonly used by social scientists. As you pursue your studies, you may very well want to study this subject in more depth later.