Research tricks that make methodology click

Hot weather causes both ice cream sales and drowning rates to rise — neither causes the other. To establish causation: need an experiment with random assignment.

p < 0.05 = if the null hypothesis were true, a result at least this extreme would occur less than 5% of the time. It is NOT the probability that the result is due to chance. Does NOT mean the effect is large — just unlikely to be random. Effect size tells you the magnitude.

Random assignment eliminates pre-existing differences. Attrition (dropout) can bias results. Internal validity: did IV cause DV? Experimenter bias: researcher inadvertently influences results. Null hypothesis: default 'no effect' claim.

Reliability: gives same result each time (consistent). Validity: measures what it claims to measure (accurate). A scale that always reads 5 lbs too heavy is reliable but not valid. Both are required for a good study.

Random assignment: participants randomly placed in experimental or control group — controls for pre-existing differences. Control group: doesn't receive treatment — provides baseline. Experimental group: receives the IV manipulation. Double-blind: neither participants nor researchers know who's in which group — prevents bias.

Statistical significance (p-value) only tells you the result probably isn't random — not how important it is. A huge study can find a tiny, meaningless effect at p<0.001. Effect size measures the magnitude. Cohen's d = (mean₁ - mean₂)/pooled SD. Always report effect size alongside p-value.

Simple random: each person equally likely to be selected — best for generalizability. Stratified: divide population into subgroups (strata), randomly sample from each — ensures representation. Cluster: randomly select groups then sample within. Convenience: whoever is available — biased, poor generalizability (WEIRD problem: Western, Educated, Industrialized, Rich, Democratic).

Naturalistic observation: high ecological validity, no cause-effect. Case study: rich detail, poor generalizability. Survey: large samples quickly, self-report bias. Correlational: shows relationships, no causation. Experimental: only method establishing causation, may lack ecological validity. Choose method based on research question.

Operational definition: specifies the exact procedures used to measure or manipulate a variable. 'Stress' is abstract — measured as cortisol level, heart rate, or score on perceived stress scale. Good operational definitions: reliable (consistent), valid (measures what it claims), practical. Allows replication.

Longitudinal: follow same people over years/decades. Strength: sees actual change. Weaknesses: dropout (attrition), time-consuming, expensive, cohort effects. Cross-sectional: compare different age groups at same time. Strength: quick, no attrition. Weakness: cohort effects (different generations, not just age).

Demand characteristics: participants guess the study's purpose and change behavior accordingly. Experimenter bias: researcher unconsciously treats groups differently or interprets results based on expectations. Single-blind eliminates demand characteristics. Double-blind eliminates both. Placebo effect: inert treatment produces real changes because of expectations.

Experimental: random assignment to conditions → can infer causation. Quasi-experimental: no random assignment — groups naturally differ. Cross-sectional: different age groups at one time — fast, but cohort effects confound. Longitudinal: same people over time — shows true development but time-consuming and attrition. Case study: deep individual analysis — rich data, poor generalizability. Naturalistic observation: behavior in natural settings — ecological validity but no control.

P-value: probability of getting results at least this extreme if the null hypothesis is true. p < .05 = less than 5% chance of a false positive (Type I error). Alpha level (α): threshold set before study (usually .05). Type I error (false positive): rejecting true null. Type II error (false negative): failing to reject false null. Effect size (Cohen's d): how large is the difference — small (.2), medium (.5), large (.8). A huge study can make trivial effects statistically significant.

Random sampling: every person in population has equal chance — best for generalizability. Stratified random: sample proportionally from subgroups — ensures minority groups represented. Cluster: randomly select groups (schools, hospitals), then sample within. Convenience: whoever is available — cheap but biased (WEIRD: Western, Educated, Industrialized, Rich, Democratic). Purposive: deliberately select particular individuals — used in qualitative research. Sample size affects statistical power.

Beneficence: research must benefit society and minimize harm. Informed consent: participants must understand and voluntarily agree to procedures (deception requires IRB approval and debriefing). Confidentiality: protect participant data and identity. Right to withdraw: can leave at any time without penalty. Debriefing: explain true purpose after deception. Milgram obedience study and Stanford Prison Experiment sparked ethics reform — IRBs (Institutional Review Boards) now required for all human research.