Deck 3: The Phoenix of Statistics

Question 1

A researcher was interested in students' use of pop-up maths support labs across six campus sites.Each maths support lab collected and analysed its own data and the resultant six reports showed differing p-values.Some sites found a statistical significance between opening hours of labs and student usage and others did not.Which of the following would it useful for her to review?

A)Levels of missing data
B)Confidence intervals
C)Outliers
D)The Null Hypotheses

Answer 1

B

Question 2

A student conducted a Bayesian analysis of increased chocolate consumption and improved test scores.He calculated a Bayes factor of 1.Should he use chocolate consumption as a predictor of improved test scores?

A)No,a Bayes factor of 1 suggests that it is not worth eating more chocolate in order to improve test scores.
B)Yes,a Bayes factor of 1 suggests that it is worth a small increase in chocolate consumption in order to improve test scores.
C)No,a Bayes factor of 1 suggests that the data is corrupted.
D)Yes,a Bayes factor of 1 suggests that it is worth substantially increasing your chocolate consumption in order to improve test scores.

Answer 2

A

Question 3

A head teacher is interested in unconscious bias in teacher recruitment in schools.There were two posts and seven candidates,four men and three women.Theoretically,all the candidates have an equal probability of being hired as they all match the selection criteria.However,the head teacher has data that suggests that it is more likely that women will be hired based on data from across the schools sector and within her own school.However,the head teacher has implemented many equality initiatives within her school and therefore wants to determine the probability that still no men will be hired.What formula could she use to determine this probability and assess the impact of unconscious bias in her school's recruitment?

A)NHST
B)Bayes' theorem.
C)Pearson's r
D)Cronbach's Alpha

Answer 3

B

Question 4

A researcher was interested in comparing maths scores by gender and used a t-test to do so.The mean for males was 66.25 and the mean for females was 78.24,with both groups having a standard deviation of 7.What is the effect size using Cohen's d?

A)1.7
B)0
C)1.712
D)-1.712

Question 5

You are the CEO of a small data forecasting company.You have decided to adopt a Bayesian approach to data analysis and modelling.When you announce this new policy,your staff are unhappy and unconvinced,as they are used to a NHST approach.You stress that the Bayesian approach has several key advantages,including which of the following.

A)You can evaluate complex statistical models without data.
B)You can reject null hypotheses without any data collection.
C)You can evaluate statistical significance using p-values.
D)You can evaluate the likelihood of the null hypothesis being true.

Question 6

Your education lecturer has devoted the past ten weeks to teaching you the Bayesian approach and is now asking that you offer a critique of it.What key criticism could you raise?

A)The reliance on a prior probability is overly objective and therefore not open to a researcher's degrees of freedom.
B)The reliance on a prior probability is overly subjective and therefore can be open to a researcher's degrees of freedom.
C)The lack of reliance on a null hypothesis is overly objective and therefore open to a researcher's degrees of freedom.
D)The lack of reliance on a prior probability is overly subjective and therefore can be open to a researcher's degrees of freedom.

Question 7

A teacher wanted to examine the likelihood that pupils would be more likely to access class materials online as opposed to using print copies.She conducted one study where the probability of making a Type I error was 0.05 and a Type II error was 0.2.Does her research have empirical probability?

A)Yes,to have empirical probability the likelihood of an effect being detected requires a single experiment,where the probability of making a Type I error is above 0.05 and a Type II error is 0.2.
B)No,to have empirical probability the likelihood of an effect being detected requires a series of repeated identical experiments,where the probability of making a Type I error is above 0.05 and a Type II error is 0.2.
C)No,to have empirical probability the likelihood of an effect being detected requires a series of repeated identical experiments,where the probability of making a Type I error is 0.05 and a Type II error is 0.2.
D)No,to have empirical probability the likelihood of an effect being detected requires a single experiment,where the probability of making a Type I error is above 0.05 and a Type II error is 0.1.

Question 8

A trainee government data analyst working in a national education department has just completed an analysis of key factors that may be influencing lower levels of attainment in boys.However,he finds only one statistically significant factor,which he includes in his report but deliberately,omits the other six non-significant findings.What is the term for what the data analyst has done?

A)HARKing
B)p-hacking
C)Mate analysis
D)Bayesian analysis

Question 9

In our previous example,the head teacher had already calculated the probability of no men being hired based on sector wide data.In the Bayesian approach,what sort of probability is this?

A)Positive probability
B)Posterior probability
C)Prior probability
D)Inferior probability

Question 10

You have joined the data modelling team for a national education policy unit.Your boss has decided that from now on the team will adopt a Bayesian approach.However,not all staff understand what this is; your boss asks you to present a training session.How would you explain the Bayesian approach in your session introduction?

A)An approach where you reject your statistical model once data is collected.
B)An approach where you do not modify the likelihood of your statistical model as more data is collected.
C)An approach that allows you to focus on testing the null hypothesis based on data collection.
D)An approach that allows you to update the likelihood of your statistical model as more data is collected.

Question 11

The head of your education research unit has just read a book on criticisms of the NHST and worries that all your unit's data analysis is now flawed.How might you reassure her?

A)NHST is a flawless approach and we need to invest in more training for researchers in our unit.
B)NHST is a flawless approach and the book was probably written by a disciple of the Bayesian approach.
C)NHST does have its flaws but everyone else uses it,therefore we should.
D)NHST does have its flaws but if we incorporate an examination of effect sizes into our analysis,we should be able to trust our research outputs.

Question 12

A researcher presented a recent study,which showed a statistical significance between increased consumption of sugar and increased tests scores in young children.How can she explain to the government minister responsible for education that increasing children's sugar consumption should not be encouraged?

A)A significant result does not mean that the effect is important
B)A significant result means that the effect is strong.
C)A significant result means that the effect is not relevant.
D)A significant result means that the effect is weak.

Question 13

Your head of research has followed your advice and now wants you to measure effect sizes.You report a Pearson's r of 0.50 for the impact of increased consumption of oily fish supplements on student reading scores.Your head of research wants to know if this is bad,as she remembers that a p-value of 0.30 is not good.What do you tell her?

A)You tell her that effect size and p-values are not the same and that a Pearson's r of 0.50 is a large effect,suggesting she should encourage the use of oily fish supplements.
B)You tell her that effect size and p-values are the same and that a Pearson's r of 0.50 means there is no statistically significant effect and oily fish supplements should not be given.
C)You tell her that effect size and p-values are not the same and that a Pearson's r of 0.50 is a small effect,suggesting she should stop the use of oily fish supplements until more data analysis is done.
D)You tell her that effect size and p-values are not the same and that a Pearson's r of 0.50 is a medium effect,suggesting she should encourage the use of oily fish supplements.

Question 14

Which of the following is not a factor in researchers' over-use of p-values and NHST in education research?

A)Statistical software encourages the over-use of p-values.
B)Time constraints within research encourages quick results.
C)Pressure to get a significant result that education policy makers can easily understand and apply.
D)Career bonus structures incentivise 'results'.

Question 15

You have been leading a research team,which has conducted research into a new approach ('Bouncy Phonics')to teaching reading to young children.Your team has conducted forty trials of 'Bouncy Phonics' but you are not sure if the results are conclusive enough to publicise the new approach across all schools nationally.A new data analyst has joined your team suggesting that meta-analysis might be a good idea,do you agree?

A)Yes,because the forty trials were identical and tested the same research question and therefore we can calculate an average effect size for 'Bouncy Phonics'
B)Yes,because the forty trials were identical and tested the same research question and therefore we can calculate an average significance for 'Bouncy Phonics.
C)No,because the forty trials were identical and tested the same research question we cannot calculate an average effect size for 'Bouncy Phonics.
D)No,because the forty trials were identical and tested the same research question we cannot calculate an average significance for 'Bouncy Phonics.

Question 16

A teacher found that students concentrated for longer in class if the teachers wore purple.She had a sample size of 48 and a p-value of 0.20.Does the teacher recommend that all teachers start wearing purple?

A)Yes,because the sample has low confidence levels.
B)No,because the sample size is large and therefore the p-values are accurate.
C)Yes,because statistical significance has nothing to do with sample size.
D)No,because the sample size is small and p-values are easily affected by sample size.