Experimental Design
The basic idea of experimental design involves formulating a question and hypothesis, testing the question, and analyzing data. Though the research designs available to educational researchers vary considerably, the experimental design provides a basic model for comparison as we learn new designs and techniques for conducting research.
Note: This review is similar to the overview of significance testing, so you will see some of the introductory material on the scientific method repeated in both places.
Part I: The Scientific Method
We start with familiar territory, the scientific method. To illustrate, we’ll look at a basic research question: How does one thing (variable A) affect another (variable B)? You may have seen variable A referred to as the treatment, or independent, variable, and variable B as the outcome, or dependant, variable. Let’s call variable A parental involvement and lets call variable B a test score. The traditional way to test this question involves:
Step 1. Develop a research question.
Step 2. Find previous research to support, refute, or suggest ways of testing the question.
Step 3. Construct a hypothesis by revising your research question:
| Hypothesis | Summary | Type |
| H1: A = B | There is no relationship between A and B | Null |
| H2: A ≠ B | There is a relationship between A and B. Here, there is a relationship, but we don’t know if it is positive or negative. | Alternate |
| H3: A < B | There is a negative relationship between A and B. Here, the < suggests that the less A is involved, the better B. | Alternate |
| H4: A > B | There is a positive relationship between A and B. Here, the > suggests that the more B is involved, the better A. | Alternate |
Step 4. Test the null hypothesis. To test the null hypothesis, A = B, we use a significance test. The italicized lowercase p you often see, followed by > or < sign and a decimal (p ≤ .05) indicate significance. In most cases, the researcher tests the null hypothesis, A = B, because is it easier to show there is some sort of effect of A on B, than to have to determine a positive or negative effect prior to conducting the research. This way, you leave yourself room without having the burden of proof on your study from the beginning.
Step 5. Analyze data and draw a conclusion. Testing the null leaves two possibilities:
| Outcome | Wording | Type |
| A = B | Fail to reject the null. We find no relationship between A and B. | Null |
| A =, <, or> B | Reject the null. We find a relationship between A and B. | Alternate |
Step 6. Communicate results. Now, we communicate our findings.
Part II: Control
One of the most important concepts to understand when about learning research methods is control. Control differentiates experimental from all other types of research. Recall the previous example of parental involvement (the treatment, or independent, variable) affecting test score (the outcome, or dependant, variable). As educational researchers, we know that many other things (extraneous variables) affect a student’s test-taking ability. Some of these might include how much the student studied for the test, IQ, test anxiety, etc. We could spend hours thinking of effects. That’s where control comes in. If the researcher could control for all these variables, which are extraneous to what he or she really wants to know about (parental involvement), the researcher could make a prediction about the effects of A on B. As you have seen, however, its not that simple when working with people.
In a pure experimental design, the researcher can control for all factors related to the outcome. In other words, we can image it possible to have students in our study who were exactly alike except for level of parent involvement. Then, we could say that parental involvement affected test scores. Since we have to control many things, social science research can never be experimental (well, maybe with clones. . .). Thus, the example we’ve been using is quasi-experimental, which acknowledges that we will try to control for as many extraneous variables as we can, but acknowledge that that is virtually impossible in experimental research.
Part III: Control vs Experimental Group
Finally, you may see research groups defined as control group vs. experimental group. The control group will not receive a treatment. The experimental group will receive a treatment. We can’t use the previous example, because it is likely (and perhaps slightly unethical when considering possible psychological effects on children) to assign students to a parental involvement group (control) versus no parental involvement group (treatment). To try another example, let’s say you teach the same lesson in the same day to two different groups of students. You want to know if letting the students work individually (the traditional method) is more effective than letting them work in teams for learning biology. In this case, individual work is the control, while teams work is the treatment. We can then test for differences between groups using the following null hypothesis: H1: individual work = group work.
Research Rundowns 