A homework activity to prepare for a short course on Bayesian Thinking

Samantha Low-Choy, Griffith Social & Behavioural Research College, with input from Chris Bigum.

Developed with, and for, “Nexus of Research & Method” subgroup, in Special Interest Group “Sociology of Education”, Griffith University

*12 April 2016*

In discussion with the SIG, these activities have been designed to prepare you to learn about Bayesian statistics. The first activity helps you find your motivation – *Why do you want to learn about (Bayesian) statistics*? The second activity helps you revisit/uproot any misconceptions about statistics, e.g.: statistics is in stasis … statistics is just a type of mathematics … there’s only one way to do statistics.

It is therefore strongly recommended that you complete Activities 1 and 2 to prepare for a short course on Bayesian Thinking.

The third activity is for those wanting a numerical introduction.

If you are time-poor, then please do Activity 1(a)-(c) and Activity 2(a) and (c).

## ACTIVITY 1 - Understanding statistics in the research you read

### EXERCISE

Find a statistical statement in a research article, where you would like to know what it means. For example, it may contain words like "p-value", "null hypothesis", "hypothesis test", "significant", "Bayesian" or "Frequentist".

Questions: send your responses to Sama

a) Provide the citation, page & line number of this motivating statement.

b) Indicate if you are willing to share it (in a googledoc).

c) Write down your immediate interpretation of what you think it means (don't overthink this!)

- There are absolutely NO right or wrong answers here.
- It takes years for some statisticians to understand the nuances, perhaps because they are more focussed on the fun calculations.

d) Jot down, or draw a mindmap, of the questions that arise from your thinking about this statement.

e) Please send these responses to Sama; they will help focus the course.

## ACTIVITY 2

### BACKGROUND

Researchers across every discipline are operating during a period where there are in fact several co-existing statistical paradigms. One of these, the null hypothesis significance testing (NHST) framework was devised in the 1930s (and made good use of the technology at that time). You may have heard of it. It often relies on p-values. Some recent happenings in the "social of statistics" (American Statistical Association, in press) have brought this into the limelight.

This exercise introduces you to this highly topical debate.

#### EXERCISE

Read the hypothesis testing thread, in the history of statistics, below. Follow links you find interesting. Answer the questions below.

##### The hypothesis testing thread, in the history of statistics

**1930s**

Neyman and Pearson famously proposed a Null Hypothesis Significance Testing framework for statistics. The underlying logic exploits what is easily calculated.

If we know what the model is (that’s called a null hypothesis),

then we can work out how surprising the data is. This is measured by the p-value. Then the logic goes:

If the data is really surprising, we should reject our hypothesised model.

**In between**

Shortcomings with NHST were gradually revealed and simmered away, e.g.

A statistical reform proposed avoiding p-values in favour of confidence intervals, e.g.

**12 Feb 2015**

P-values and NHST exploded into the greater consciousness when a journal (Basic and Applied Social Psychology) wrote an editorial that outlawed p-values:

**4 March 2015**

Some eminent statisticians testing-reactions-from-the-statistical-arena blogged about this.

**5 March 2015**

BASP were the first journal to make such a stand, as noted by the UK’s Royal Statistical Society,

**7 March 2016**

Nearly a year later, the American Statistical Association publishes a statement, concentrating on the appropriate use of p-values, devised by a panel of statistical experts.

### QUESTIONS: A REFLECTION

Read the previous potted history, exploring links in the order that interests you.

a) Many qualitative researchers have had little engagement with statistics. Keeping this in mind, and after following the suggested reading above, what questions come to mind, for you? Jot these down.

Follow up some more contributions to this debate. For example, you may wish to continue Googling, follow some of the protagonists on Twitter, or find a textbook and see what it says.

b) Write some words or draw a diagram showing your view on how this debate has evolved, on at least one front.

## ACTIVITY 3: The 3 doors example

### EXERCISE

Suppose you're on a game show, and you're given the choice of three doors. Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?

### QUESTIONS: ANOTHER REFLECTION

We don’t care if you get it right or not … what is important is the journey … for that reason, try really hard to resist the urge to look up the answer. I’m interested in how you interpret and think through the question, your answer, and what stumbling blocks you’ve hit.

a) What’s your gut response to this question? (Squiggle or words).

b) Did you get stuck anywhere? Describe.

c) Talk to someone else to find out how they responded.

d) Now write down (in squiggle or words) how you both have responded.

## Postscript

### FINAL EXERCISE

If you have time, I’d appreciate it if you could take off your “student’s” learning hat and put on your “collegial” teaching hat, to provide feedback on this exercise.

- What was effective or could be improved?
- What types of students would benefit (before or after improvement)?
- Send these comments to Sama!