# Lesson 1

Being Skeptical

## 1.1: Notice and Wonder: Headlines (10 minutes)

### Warm-up

The purpose of this warm-up is to elicit the idea that there are different ways to collect and represent data. This will be useful when students examine statistical designs in a later activity. While students may notice and wonder many things about these images, how the data are represented (or misrepresented) is the important discussion point.

Through articulating things they notice and things they wonder about the headlines and graph, students have an opportunity to attend to precision in the language they use to describe what they see (MP6). They might first propose less formal or imprecise language, and then restate their observation with more precise language in order to communicate more clearly.

### Launch

Display the image and headlines for all to see. Ask students to think of at least one thing they notice and at least one thing they wonder. Give students 1 minute of quiet think time, and then 1 minute to discuss the things they notice and wonder with their partner, followed by a whole-class discussion.

### Student Facing

A graph and 2 headlines from a website are shown.

What do you notice? What do you wonder?

1. “80% of Dentists Recommend Acme Toothpaste”
2. “Pythagoras Brand Rulers Measure 20% Better”

### Activity Synthesis

Ask students to share the things they noticed and wondered. Record and display their responses for all to see. If possible, record the relevant reasoning on or near the image or statement. After all responses have been recorded without commentary or editing, ask students, “Is there anything on this list that you are wondering about?” Encourage students to respectfully disagree, ask for clarification, or point out contradicting information.

Display a wider view of the graph that shows more of the story.

Ask, “How does having the whole graph tell a different story than just having the part of the graph from 1998 to 2012?” (In the whole graph, you can see that the slope of the line is positive over the 112 years represented in the graph, and you could find other 12-year periods in which the slope is negative, positive, or close to zero.) If time allows, follow up by asking, “Does the dashed line appear to be a line of best fit or is it something else? Explain your reasoning.” (The dashed line does not appear to be a line of best fit. It just seems to connect the first point in the graph to the last point. This could be misleading since the line of best fit would have a greater slope than the dashed line, and would be a little lower on the graph.)

Here are some questions for discussion:

• “Why is the dentist headline potentially misleading?” (It does not mention if they recommend other brands as well. For example, dentists might just recommend using any toothpaste at all and this brand works just as well as any other brand.)
• “Why is the ruler headline misleading?” (It does not mention what it is that Pythagoras rulers measure better than.)

## 1.2: Take Turns: Statistical Design (25 minutes)

### Activity

In this activity, students take turns with a partner classifying studies as surveys, observational studies, or experimental studies, and critiquing the design of the study. Students trade roles explaining their thinking and listening, providing opportunities to explain their reasoning and critique the reasoning of others (MP3). Taking turns classifying studies gives students opportunities to analyze statements and structures closely and make connections (MP2, MP7).

### Launch

Arrange students in groups of 2. Display these statements for all to see:

• Surveys are sets of questions given to people to seek their responses.
• Observational studies collect data without influencing things directly.
• Experimental studies collect data by directly influencing something to determine how another thing is changed.

Identify students who sort any of the studies incorrectly.

Representation: Develop Language and Symbols. Create a display of important terms and vocabulary. Invite students to suggest language or diagrams to include that will support their understanding of surveys, observational studies, and experimental studies. Encourage students to create an icon for each term which they can explain to someone, or to sketch each scenario in a setting of their choice (What would a survey, experiment, or observation look like at a swimming pool? Restaurant? Baseball game? Family reunion?) What are key generalizations about each?
Supports accessibility for: Conceptual processing; Language

### Student Facing

One partner reads the statistical question and study design to the other partner. The reader should then sort the item into the type of study described:

• Survey
• Observational study
• Experimental study

The other partner should decide whether the study is good or bad and explain their reasoning. After the discussion, switch roles and have the other partner read the next statistical question and study design aloud. Take turns until no questions remain.

1. Why are students missing so much school in the district? A district administrator selects 300 student names at random from the enrollment list and sends a letter to each student’s home. The letter includes a page to be returned to their school signed by a parent or guardian. The page asks, “How many days has your student missed school this year?” and “What are the reasons for missing school on those days?”
2. Why are students missing so much school in the district? A district administrator chooses one of the elementary schools in the district and asks the principal to provide information about the number of absences and the excuse notes provided to the school.
3. What type of sweetener do flies prefer? A scientist puts the same amount of each sweetener into different bowls of water and counts the number of flies that drink from each bowl in 4 hours.
4. What type of sweetener do flies prefer? A scientist divides the flies into different groups and gives each group only water with a certain type of sweetener in it for 3 days. The scientist then does a test on each group to see how well the flies can fly through a maze.
5. Is there a link between dark chocolate and weight loss when compared to milk chocolate? A nutritionist asks 5 friends to eat dark chocolate along with their usual food for 6 months and 5 other friends to eat milk chocolate along with their usual food for 6 months. The nutritionist then compares their weight after the 6 months to their weight before eating the different chocolates.
6. Is there a link between dark chocolate and weight loss when compared to milk chocolate? A nutritionist gathers 60 people, selected at random, then randomly assigns half of the group to eat a single dark chocolate bar after dinner each night and the other half to eat a single milk chocolate bar after dinner each night for 6 months. Everyone is to keep track of the other food they eat in an app provided. The nutritionist then compares each person’s weight after the 6 months to their weight before eating the different chocolates accounting for the other calories consumed.
7. Do voters in the district favor a sales tax increase of 1% to fund the parks and recreation department? A politician sends a letter to 300 voters in the district asking, “Would you pay extra money on your essential groceries to hire more government workers to plant flowers around the town?”
8. Do voters in the district favor a sales tax increase of 1% to fund the parks and recreation department? A politician sends a letter to 300 voters selected at random in the district asking, “Would you be in favor of a 1% increase in sales tax to fund the parks and recreation department in town?”

### Student Facing

#### Are you ready for more?

A college student wants to study how sleep impacts college students.

### Anticipated Misconceptions

Some students may have trouble determining whether a study is an experiment or not. Ask students to recall the definition of an experiment. Point out that experiments generally have 2 groups of subjects with only 1 variable changed between the groups.

### Activity Synthesis

The purpose of this discussion is for students to classify each study by study type, and to informally critique the design of each study. For previously identified students, ask, “What misconception did you have that caused you to sort one of the studies the way you did?” (I thought the sweetener and fly study was experimental because it seemed like an experiment. I realized it was observational because all the researcher did was observe several different bowls with the same type of liquid in it.)

Here are some questions for discussion:

• “For one of the studies, how did you determine what type of study it was?” (I decided that the missing school example in which the letter sent home was a survey because the participants had to respond to a question.)
• “What were some reasons that you said some of the studies were good?” (There was random assignment in some of the designs. Some of the questions asked in surveys asked about the facts. Some of the studies had a large sample size.)
• “What were some reasons that you said some of the studies were bad?” (The sample size was small. The questions seemed to lead the people being surveyed to give a particular answer. Some of the experiments did not test for what the research question was asking.)
• “What is important to think about when designing a study?” (It is important to think about whether or not the design will answer the question being asked, and whether you are influencing the results of the study by the way you ask the questions.)
• “How did you determine if a study was observational or experimental?” (I looked for the researcher influencing a variable for the subjects in the study. If this was due to the design of the study, then I knew it was an experiment.)
• “A survey is a specific type of observational study. How did you determine if a situation should be called a survey or observational study?” (In a survey, the researcher asks the participants directly to respond to a set of questions. In a general observational study, the researcher does not need to ask questions of the participants.)
Conversing: MLR2 Collect and Display. During the synthesis, listen for and collect language students use to classify and critique each study. Record informal student language alongside the mathematical terms (survey, observational study, experimental study) on a visual display of the three study types and update it throughout the remainder of the lesson. Consider grouping words and phrases students use to critique the studies in a “good study” or “bad study” section. Remind students to borrow language from the display as needed. This will provide students with a resource to draw language from during small-group and whole-group discussions.
Design Principle(s): Maximize meta-awareness; Support sense-making

## Lesson Synthesis

### Lesson Synthesis

Here are some questions for discussion.

• “Why is it important for a study to be designed well?” (If the study is not designed well, then it will not answer the question being asked.)
• “If you wanted to estimate the percentage of people who plan to watch a major sporting event, why would it be better to select people at random than to select people who play the sport?” (If you only select people who play the sport, then you would only have one part of the population. Selecting people at random allows your sample to be more representative of the whole population who might watch the event.)
• “What type of study do you think you would conduct to estimate the percentage of people who plan to watch the major sporting event? Explain your reasoning.” (I would conduct a survey because I would need people to respond to a question to know what their plans are.)
• “Come up with a topic and a study design. Explain why it is a survey, observational, or experimental study.” (I am going to study how drinking energy drinks impacts the ability to follow directions. I will randomly assign students to two different groups. One group will drink an energy drink and the other group will not. Then I will have them complete a task that requires them to follow directions. I will record the number of mistakes each person makes. This is an experimental study because I am directly influencing who drinks an energy drink and who does not so that I can determine the effect it has on following directions.)
• “Why do we collect data when conducting a study?” (Collecting data when conducting a study allows you to make inferences and draw conclusions about population parameters, and it can help you answer statistical questions.)
• “An experiment that studies how energy drinks impact following directions has two groups, one group of 25 people that is given an energy drink to drink and one group of 25 people that is given water to drink. Why is it important to the study for the participants to be randomly assigned to each group?” (It is important because you want to make sure that other factors are not responsible for any differences observed in the results. For example, if you did not assign them randomly, you might end up with people who prefer energy drinks in one group and people who don’t in the other group. Then you would not know if your results were related to their preference for energy drinks or water, or to their drinking the energy drink or drinking water.)

## Student Lesson Summary

### Student Facing

There are many things a researcher should consider when collecting data about a question they are interested in. How the subjects of the study are selected as well as the details of how the study is conducted are very important in getting useful data to answer the question at hand. In particular, the researcher should consider:

• selecting subjects that are representative of the larger population
• how subjects are selected for a study or assigned to groups within a study
• making sure that the question does not lead subjects to answer a certain way
• making sure that data is collected and analyzed fairly
• using a sample that is large enough to detect differences in the presence of variability
• collecting data directly related to the question being asked

Without directly addressing these concerns, the data collected might lead to misleading conclusions. Three common types of studies are surveys, observational studies, and experimental studies.

• A survey is a set of questions given to people to seek their responses.
• An observational study collects data without influencing the subjects directly.
• An experimental study collects data by directly influencing something to determine how another thing is changed.