Context: Randomization is ethically or practically possible.

I010 A control group is assigned by randomly dividing subjects into an experimental group(s) and control group.

I012 A control group is constructed (non-randomly) to be as similar to the experimental group as possible by matching each subject in the treatment and control group (case-control).

I013 A control group is constructed (non-randomly) to be as similar to the experimental group as possible.

I017 Data is collected as is (no assignment to treatments). The control group is identified as any group that does not receive the treatment.

I018 Pre-treatment measurements are used in place of a control group.

I019 No control group is considered.

Context: Randomization is not ethically or practically possible.

I150 A control group is constructed (non-randomly) to be as similar to the experimental group as possible by matching each subject in the treatment and control group (case-control).

I152 A control group is constructed (non-randomly) to be as similar, on average, to the experimental group as possible.

I157 A control group is identified as any group that does not receive the treatment.

I158 Pre-treatment measurements are used in place of a control group.

I159 No control group is considered.

Context: Comparing the extremes of two distributions, particularly when one distribution has a larger variance than the other and despite which mean is larger, the distribution with the larger variance is more likely put mass on extreme outcomes.

I020 Both the mean and standard deviation are considered when considering the probability of extreme outcomes for a sample (n>1) of observations. Percentiles are computed.

I022 Both the mean and standard deviation are considered when predicting the extreme outcomes, but the information is not assimilated properly (e.g., they cannot compute percentiles).

I026 Extreme outcomes are not considered. Both the mean and standard deviation are considered and the group with the "best" mean and smallest standard deviation is chosen.

I029 Extreme outcomes are not considered. The group with the "best" mean is chosen.

Context: There are k>2 blocking factors of which only i<k are at least moderately associated with the repsonse.

I030 The study is blocked only on variables that are strongly associated with the response variable.

I032 The study is blocked on a variable strongly associated with they response by considering a single sub-population for the study.

I035 The study is blocked on all possible variables regardless of their association with the response variable.

I037 The study is not blocked because the treatments have been randomized. (Or, in the sampling context, a random sample was taken).

I039 Blocking is not considered.

Context: A randomized experiment is possible to compare one or more treatments and control.

I040 An experiment is performed. Subjects are randomly assigned to treatment(s) and control.

I045 An experiment is performed. Subjects are non-randomly assigned to treatment(s) and control.

I046 An observational study is performed with reference to related randomized experiments.

I048 An observational study is performed.

I049 Anecdotal evidence is presented.

I050 Groups with different sample sizes can be compared.

I055 Groups with different sizes can be compared as long as the size of each group is the same relative to the size of the population.

I058 Groups with different sample sizes cannot be compared because larger groups yield more precise estimates of the mean.

I059 Groups with different sample sizes cannot be compared because you will introduce a bias.

Context: It is practically impossible or very costly to draw a sample directly from the population. A convenient sample frame exists, but may omit some units of the population.

I060 Obtains an SRS from a sampling frame that is a proper subset of the population noting possible generalizability concerns.

I061a Obtains an SRS from the population claiming it is inappropriate or not useful to draw from a sampling frame unequal to the population, but they note practical concerns.

I061b Obtains an SRS from the population of interest with no regard to the practical concerns.

I066 Obtains an SRS from a sampling frame that is a proper subset of the population with no regard to generalizability.

I067 Obtains a non-random sample noting practical concerns.

I069 Obtains a non-random sample.

Context: The sample size is a small fraction of the population size.

I070 The size of the sample, together with the inherent variability in the population from which the sample was drawn, determines how certain we are about our estimates of the population mean.

I072 The size of the sample determines how certain we are about the our estimates of our estimate of the population mean.

I074 The size of the sample relative to the population size, together with the inherent variability in the population from which the sample was drawn, determines how certain we are about our estimates of the population mean.

I075 The size of the sample relative to the population size determines how certain we are about our estimates of the population mean.

I078 The size of the sample has no bearing on our certainty of our estimates for the population mean, only the variability in the population is important.

I079 The size of the sample has no bearing on our certainty of our estimates for the population mean.

I080 The size of the sample does not affect the bias of our estimates of the population mean.

I088 Increasing the sample size reduces the bias of our estimates of the population mean.

I089 Increasing the sample size increases the bias of our estimates of the population mean.

Context: A (1-a )% confidence interval for the population parameter is given to be (a,b).

I090 All possible samples of size n would give us a confidence interval that contains the population parameter.

I098 There is a (1-a )% chance that the population parameter is in the interval (a,b).

I099 (1-a )% of all possible samples of size n would give us a parameter estimates in the interval (a,b).

I100 The observed evidence in favor of Ha, or stronger, would occur by chance alone with probability p if H₀ is true.

I107 There is probability 1–p that Ha is true.

I108 There is probability p that the Ha is true.

I110 Both practical and statistical significance are considered in making a decision.

I117 Only statistical significance is considered in the decision, effect size is ignored.

I119 Only practical significance is considered in the decision; variability is ignored.

Comment: This is a weak cluster. Though choice of hypotheses is a common problem, it’s usually just an issue of vocabulary. However, these facets can affect the interpretation, or even the computation of a p-value (one vs. two-sided, for instance).

I120 The Null Hypothesis is the status quo and is assumed to be true. The Alternate Hypothesis is the claim you wish to prove.

I124 The Null Hypothesis is the status quo and is assumed to be true. The Alternate Hypothesis is any claim that is against the null hypothesis.

I129 The Null Hypothesis is the claim you wish to prove.

Context: Y is the response. X is observed and associated with Y. There are likely latent variables, Z, that are also associated with Y.

I130 Relationships in data can often be explained by other variables (observed or unobserved).

I135 Only observed data is considered when explaining relationships in data. Latent, variables are not considered.

I138 Relationships are ignored when making predictions (e.g., X1 is ignored when making predictions of X2 in the context of regression)

I139 When there is a strong relationship, you can generalize from the data.

I140a Predictions based on a sample are uncertain.

I140b The data’s origins are considered before generalizations are made.

I149 Variability in the data precludes drawing any conclusions.