Having subjects respond reliably on a measure is a great start, but there is another concept you need to get down really well. That’s validity. There are many kinds of validity, but they all refer to whether or not what you are manipulating, or what you are measuring, truly reflects the concept you think it does.

Here’s a crazy (but true) example:
Many years ago, people believed that if you had a large brain then you were intelligent. Suppose you went around and measured the circumference of your friends' heads because you also believed this theory (they’d know for sure that you're a psychology major now).

Is the size of a person’s head a reliable measure (Think first!)? The answer is YES. If I measured the size of your head today and then next week, I would get the same number. Therefore, it is reliable. However, the whole idea is wrong! Because we now know that larger headed people are not necessarily smarter than smaller headed ones, we know that the theory behind the measure is invalid.