{ "id": "ep0wbyqv3h", "slug": "post-hoc-analysis", "title": "Post-hoc analysis", "updated": "2026-10-15", "topic_path": [ { "id": "physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical" }, { "id": "physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math" }, { "id": "physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics" }, { "id": "physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference" }, { "id": "physical.math.statistics.stats_statistical_inference.hypothesis_testing", "label": "Hypothesis testing", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.hypothesis_testing" }, { "id": "physical.math.statistics.stats_statistical_inference.sequential_multiplicity.false_discovery_control", "label": "False discovery rate and error-rate control", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.sequential_multiplicity.false_discovery_control" } ], "geo": [ { "id": "geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.hypothesis_testing", "label": "Non-geographic · 1800 to 1945: Hypothesis testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.hypothesis_testing", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1800", "label": "Non-geographic · 1800 to 1945", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800" }, { "id": "geo.nongeo.t1800.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical" }, { "id": "geo.nongeo.t1800.physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math" }, { "id": "geo.nongeo.t1800.physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics" }, { "id": "geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference" }, { "id": "geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.hypothesis_testing", "label": "Hypothesis testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1800.physical.math.statistics.stats_statistical_inference.hypothesis_testing" } ] }, { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.hypothesis_testing", "label": "Non-geographic · 1946 to 2000: Hypothesis testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.hypothesis_testing", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical" }, { "id": "geo.nongeo.t1946.physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math" }, { "id": "geo.nongeo.t1946.physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics" }, { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference" }, { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.hypothesis_testing", "label": "Hypothesis testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference.hypothesis_testing" } ] } ], "excerpt": "A post-hoc analysis is a comparison of treatment means chosen after the data are collected, usually to locate which group differences drive a significant ANOVA result.", "snippet": "A post-hoc analysis is a comparison of treatment means chosen after the data are collected, usually to locate which group differences drive a significant ANOVA result.", "node": "physical.math.statistics.stats_statistical_inference.sequential_multiplicity.false_discovery_control", "markdown": "# Post-hoc analysis\n\nA post-hoc analysis in the design of experiments is a comparison between treatment means that is chosen after the data have been collected and looked at, typically to locate which group differences drive a significant omnibus result such as an ANOVA F-test. The term contrasts with a priori (planned) comparisons, which are chosen before the data are collected; post hoc comparisons are planned after the experimenter has collected the data and looked at the means.[1](https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf) In clinical research the same words carry a second, pejorative sense: an unplanned analysis conceived after the data are examined, which may generate new hypotheses but may also identify false positive findings that exist only in the examined data set.[2](https://pmc.ncbi.nlm.nih.gov/articles/PMC10964884/)\n\n| Key fact | Detail |\n|---|---|\n| What it answers | After a significant omnibus test, which pairs (or contrasts) of group means differ[3](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Introduction_to_Applied_Statistics_for_Psychology_Students_%28Sarty%29/12%3A_ANOVA/12.02%3A_Post_hoc_Comparisons) |\n| Alpha inflation | Six unprotected pairwise tests at 0.05 raise the true family-wise alpha to approximately 0.2649[4](https://pages.uoregon.edu/stevensj/papers/posthoc.pdf) |\n| Basic fix | Bonferroni: divide alpha by the number of comparisons, e.g., 4 comparisons at family-wise 0.05 gives 0.0125 per test[5](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html) |\n| Reported use | In 20 years of environmental and biological literature, Tukey HSD appeared in 30.04% of articles, Duncan's test in 25.41%, and Fisher's LSD in 18.15%; Games-Howell (1.13%), Holm-Bonferroni (1.25%), and Scheffé (2.25%) were least used[6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/) |\n| Unequal sample sizes | The Tukey–Kramer modification, described in Clyde Young Kramer's 1956 Biometrics paper[7](https://doi.org/10.2307/3001469) |\n| Unequal variances | Only four of 18 procedures in SPSS 24, Dunnett's C, Dunnett's T3, Games-Howell, and Tamhane's T2, controlled Type I error when equal variances and sample sizes were violated[8](https://journals.sagepub.com/doi/full/10.1177/2515245918808784) |\n| Main alternative | False discovery rate control via the Benjamini–Hochberg procedure (1995)[9](https://doi.org/10.1111/j.2517-6161.1995.tb02031.x) |\n\n## How it works\n\nEach pairwise test run at level \\( \\alpha' \\) adds a chance of at least one false rejection. With c independent comparisons the family-wise error rate (FWER), the probability of at least one Type I error among all comparisons, is \\( 1-(1-\\alpha')^{c} \\) under the complete null: three comparisons at 0.05 give \\( 1-(1-0.05)^{3} = 0.142625 \\),[10](https://statacumen.com/book/S4R/post-hoc-tests.html) and six independent tests at 0.05 give approximately 0.2649; the six pairwise LSD tests among four means, however, share group data and a common error estimate, so this calculation is only illustrative and their true family-wise rate is lower.[4](https://pages.uoregon.edu/stevensj/papers/posthoc.pdf) With four unplanned contrasts the FWER approaches 0.20.[1](https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf) Looking at the means before choosing comparisons makes a Type I error close to certain under the complete null.[1](https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf)\n\nTwo families of control exist: p-value adjustments (Bonferroni, Holm, Hochberg, Benjamini-Hochberg) and range tests (Tukey HSD, Dunnett, Newman-Keuls, Duncan, Scheffé, Dunn).[11](https://tjmurphy.github.io/jabstb/posthoc.html) The Bonferroni inequality bounds the FWER by \\( \\mathrm{FW} \\le c\\alpha' \\), so setting \\( \\alpha' = \\alpha/c \\) controls it at \\( \\alpha \\);[1](https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf) the Šidák correction of 1967 is slightly less conservative and generally more powerful.[12](https://doi.org/10.1080/01621459.1967.10482935) Range tests instead use the studentized range statistic \\( q = (\\bar{X}_{l} - \\bar{X}_{s})/\\sqrt{MS_{\\mathrm{within}}/n} \\), which assumes equal group sizes; for unequal sizes the Tukey–Kramer statistic uses a pair-specific denominator \\( \\sqrt{MS_{\\mathrm{error}}(1/n_i+1/n_j)/2} \\). Its critical value grows with the number of groups because the gap between extreme means grows.[5](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html) A different target, the false discovery rate, controls the expected proportion of false rejections rather than the chance of any; the [Benjamini–Hochberg procedure](https://www.edgepedia.org/benjamini-hochberg-procedure) of 1995 is the standard tool and is widely used for hit screening in large -omics data sets.[9](https://doi.org/10.1111/j.2517-6161.1995.tb02031.x) • [11](https://tjmurphy.github.io/jabstb/posthoc.html)\n\n## How it is done\n\nThe traditional workflow runs the omnibus ANOVA first; if \\( H_{0} \\) is not rejected, testing stops, and only a rejected \\( H_{0} \\) is followed by pairwise comparisons.[3](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Introduction_to_Applied_Statistics_for_Psychology_Students_%28Sarty%29/12%3A_ANOVA/12.02%3A_Post_hoc_Comparisons) Before choosing a procedure the practitioner checks normality, equality of variance, and group sizes, because a critical review found these assumptions were often not verified in the applied literature.[6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/) The choice matters: in a simulation of the 18 pairwise procedures available in SPSS 24, only the four designed for assumption violations (Dunnett's C, Dunnett's T3, Games-Howell, Tamhane's T2) adequately controlled Type I error when equal sample sizes and variances were violated, and even the conservative Bonferroni and Scheffé tests had inflated error rates when the smallest group had the largest variance.[8](https://journals.sagepub.com/doi/full/10.1177/2515245918808784)\n\nResults should be reported as adjusted p-values with simultaneous confidence intervals. Adjusted p-values can exceed 1 and are capped at 1; R's emmeans package reports FWER-corrected p-values directly comparable to alpha.[5](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html) Reports should state how the FWER was controlled, for example by naming the post-hoc test after the ANOVA, with legends saying \"adjusted for multiple comparisons.\"[11](https://tjmurphy.github.io/jabstb/posthoc.html)\n\n## Origin\n\nThe oldest procedure in common use is Fisher's LSD, essentially a series of multiple t-tests using a pooled standard deviation across all groups; sources credit either 1935, when it is called the first pairwise comparison technique,[13](https://statoberrypapaya.github.io/TextbookofAgstat/multipleComparison.html) or 1940.[14](https://schmidtpaul.github.io/dsfair_quarto/ch/summaryarticles/multiplicityadj.html) The Tukey HSD year is likewise disputed: one tutorial credits [John Tukey](https://www.edgepedia.org/john-tukey) in 1949,[14](https://schmidtpaul.github.io/dsfair_quarto/ch/summaryarticles/multiplicityadj.html) while a review ties the procedure and the familywise-error terminology to Tukey's unpublished 1953 Princeton manuscript \"The Problem of Multiple Comparisons.\"[15](https://journals.sagepub.com/doi/10.3102/00346543066003269) The unequal-sample-size modification appears in a 1956 [Biometrics](https://www.edgepedia.org/biometrics) paper.[7](https://doi.org/10.2307/3001469) • [6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/) Scheffé's method for judging all contrasts appears in his 1953 [Biometrika](https://www.edgepedia.org/biometrika) paper,[16](https://doi.org/10.1093/biomet/40.1-2.87) and the treatments-versus-control procedure appears in Charles W. Dunnett's 1955 paper in the Journal of the American Statistical Association.[17](https://doi.org/10.1080/01621459.1955.10501294) The Bonferroni t test for simultaneous confidence intervals of selected contrasts appears in Olive Jean Dunn's 1961 paper,[18](https://doi.org/10.1080/01621459.1961.10482090) and Sture Holm's 1979 paper presents the sequentially rejective Bonferroni test, in which hypotheses are rejected one at a time until no further rejections can be done.[19](https://doi.org/10.2307/4615733)\n\n## Variants\n\n**Tukey HSD** compares all pairs of means and is the procedure of choice when all pairs are being compared.[4](https://pages.uoregon.edu/stevensj/papers/posthoc.pdf) Its critical difference is \\( \\mathrm{HSD} = q \\cdot \\sqrt{MSE/n^{*}} \\), with q the studentized range critical value; stronger correction widens the interval compared with LSD, while for pairwise comparisons Tukey intervals are generally narrower than Scheffé intervals, which are the most conservative.[4](https://pages.uoregon.edu/stevensj/papers/posthoc.pdf) With k groups there are \\( k(k-1)/2 \\) pairwise comparisons, and the standard error uses the harmonic mean of the two group sample sizes.[20](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Mikes_Biostatistics_Book_%28Dohm%29/12%3A_One-way_Analysis_of_Variance/12.6%3A_ANOVA_post-hoc_tests) The Tukey–Kramer method modifies the q statistic for unbalanced designs;[5](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html) Anthony J. Hayter's 1984 paper proves the procedure is conservative.[21](https://doi.org/10.1214/aos/1176346392)\n\n**Scheffé** covers all contrasts (linear combinations of means), not just pairs, with critical value \\( (k-1) \\) times the ANOVA critical F; it does not require equal group sizes but is the most conservative and least powerful of the common procedures.[16](https://doi.org/10.1093/biomet/40.1-2.87) • [3](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Introduction_to_Applied_Statistics_for_Psychology_Students_%28Sarty%29/12%3A_ANOVA/12.02%3A_Post_hoc_Comparisons) • [5](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html) **Dunnett** compares several treatments only against one control, giving fewer comparisons and narrower intervals, with the standard error changed by multiplying \\( MS_{\\mathrm{error}} \\) by 2.[17](https://doi.org/10.1080/01621459.1955.10501294) • [20](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Mikes_Biostatistics_Book_%28Dohm%29/12%3A_One-way_Analysis_of_Variance/12.6%3A_ANOVA_post-hoc_tests) The three nest: Dunnett (one reference mean versus the rest) inside Tukey (all pairs) inside Scheffé (all contrasts), and the most powerful test consistent with the question should be used.[5](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html)\n\n**Power ordering.** Fisher's LSD is the most powerful of the common procedures, per Carmer and Swanson's 1973 comparison, but it preserves the experiment-wise rate only when there are exactly three treatment means, and it should be used only after a positive ANOVA in a small three-group experiment.[1](https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf) • [13](https://statoberrypapaya.github.io/TextbookofAgstat/multipleComparison.html) • [11](https://tjmurphy.github.io/jabstb/posthoc.html) Holm's stepwise procedure is uniformly more powerful than classical Bonferroni while keeping the FWER at the specified level in the strong sense.[19](https://doi.org/10.2307/4615733) • [6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/)\n\n**Unequal variances.** The Games-Howell procedure, from Paul A. Games and John F. Howell's 1976 paper, incorporates the Welch degrees-of-freedom solution and a weighted [pooled variance](https://www.edgepedia.org/pooled-variance) instead of MSE, and has more power than several tests when assumptions are violated.[22](https://doi.org/10.3102/10769986001002113) • [6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/) The four robust procedures (Games-Howell, Tamhane's T2 from Ajit C. Tamhane's 1979 paper, and Dunnett's T3 and C from Dunnett's 1980 paper) use a sample-size-weighted pooled variance for only the two groups being compared and modify the degrees of freedom; power was similar across the four but slightly higher for Games-Howell.[23](https://doi.org/10.1080/01621459.1979.10482541) • [24](https://doi.org/10.1080/01621459.1980.10477552) • [8](https://journals.sagepub.com/doi/full/10.1177/2515245918808784) Non-parametric post-hoc options include the Dunn, Steel, Nemenyi, and Steel-Dwass tests.[6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/) Duncan's multiple range test and the Newman-Keuls test have been criticized for insufficient protection against alpha slippage.[4](https://pages.uoregon.edu/stevensj/papers/posthoc.pdf)\n\n## Applications\n\nUsage is uneven across fields. Beyond the environmental and biological survey (Tukey 30.04%, Duncan 25.41%, LSD 18.15%, Scheffé 2.25%),[6](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/) a review of ecological journals found more than 40,000 reports of multiple comparison test use over 60 years,[25](https://pubmed.ncbi.nlm.nih.gov/33335808/) and Duncan's test is especially popular in the plant sciences for large numbers of treatments.[13](https://statoberrypapaya.github.io/TextbookofAgstat/multipleComparison.html)\n\nIn R, TukeyHSD(aov(...)) reports 95% family-wise confidence levels and adjusted p-values,[10](https://statacumen.com/book/S4R/post-hoc-tests.html) and the multcomp package's glht with mcp(Label = \"Tukey\") performs Tukey contrasts with single-step adjusted p-values, its default mcp test being Dunnett.[20](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Mikes_Biostatistics_Book_%28Dohm%29/12%3A_One-way_Analysis_of_Variance/12.6%3A_ANOVA_post-hoc_tests) SPSS offers 18 pairwise procedures.[8](https://journals.sagepub.com/doi/full/10.1177/2515245918808784) Two pitfalls: pairwise.t.test produces adjusted p-values but not adjusted confidence intervals, for which a procedure such as TukeyHSD is better; and repeated-measures analyses compare means across related measurements while accounting for within-subject dependence, so ordinary independent-group range tests cannot be applied without an appropriate adjustment for that dependence.[11](https://tjmurphy.github.io/jabstb/posthoc.html)\n\n## Limitations and alternatives\n\nCorrection costs power, and the cheapest alternative is to plan fewer comparisons: planned (a priori) contrasts are more powerful than post-hoc tests because they correct only for the planned number of tests.[20](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Mikes_Biostatistics_Book_%28Dohm%29/12%3A_One-way_Analysis_of_Variance/12.6%3A_ANOVA_post-hoc_tests) Where many hypotheses are screened rather than confirmed, FWER control can be relaxed to false discovery rate control.[9](https://doi.org/10.1111/j.2517-6161.1995.tb02031.x)\n\nIn clinical trials, \"post hoc\" names an unplanned analysis and carries the risks of HARKing, cherry-picking, p-hacking, fishing expeditions, and data dredging, which prespecified analyses reduce.[2](https://pmc.ncbi.nlm.nih.gov/articles/PMC10964884/) A related criticized practice is the post hoc power calculation: power analysis should be used before a study to choose sample size and never to interpret a completed study, because observed power is directly related to the obtained p-value and provides no additional information; the recommended practice is to interpret completed studies with 95% confidence intervals for effect sizes.[26](https://www.jrheum.org/content/49/8/867) A 2019 paper by Yiran Zhang and colleagues, \"Post hoc power analysis: is it an informative and meaningful analysis?\", addresses the same question.[27](https://doi.org/10.1136/gpsych-2019-100069)\n\nRecent work extends the classical toolkit. Group sequential versions of Holm and Hochberg procedures appear in Ajit C. Tamhane, [Dong Xi](https://www.edgepedia.org/dong-xi), and Jiangtao Gou's 2021 paper, where the step-up group sequential Hochberg procedure controlled the FWER more closely and was more powerful.[28](https://doi.org/10.1002/sim.9128)\n\n## References\n\n1. [Chapter 12: Multiple Comparisons Among Treatment Means (Howell, Statistical Methods for Psychology supplement)](https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf)\n2. [Types of Analysis: Planned (prespecified) vs Post Hoc, Primary vs Secondary, Hypothesis-driven vs Exploratory, Subgroup and Sensitivity, and Others (Andrade, Indian Journal of Psychological Medicine, 2023)](https://pmc.ncbi.nlm.nih.gov/articles/PMC10964884/)\n3. [12.02: Post hoc Comparisons (stats.libretexts.org)](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Introduction_to_Applied_Statistics_for_Psychology_Students_%28Sarty%29/12%3A_ANOVA/12.02%3A_Post_hoc_Comparisons)\n4. [Post Hoc Tests in ANOVA (Stevens, 1999 handout)](https://pages.uoregon.edu/stevensj/papers/posthoc.pdf)\n5. [Chapter 17: A Priori and Post-Hoc Comparisons (University of Washington course text)](https://courses.washington.edu/psy524a/_book/apriori-and-post-hoc-comparisons.html)\n6. [On the use of post-hoc tests in environmental and biological sciences: A critical review](https://pmc.ncbi.nlm.nih.gov/articles/PMC11637079/)\n7. [Clyde Young Kramer (1956). Extension of Multiple Range Tests to Group Means with Unequal Numbers of Replications. Biometrics.](https://doi.org/10.2307/3001469)\n8. [An Updated Recommendation for Multiple Comparisons (Advances in Methods and Practices in Psychological Science)](https://journals.sagepub.com/doi/full/10.1177/2515245918808784)\n9. [Yoav Benjamini, Yosef Hochberg (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society Series B (Statistical Methodology).](https://doi.org/10.1111/j.2517-6161.1995.tb02031.x)\n10. [11.5 Post Hoc Tests | Statistical Acumen: Statistics for Research (S4R)](https://statacumen.com/book/S4R/post-hoc-tests.html)\n11. [Chapter 35 ANOVA Posthoc Comparisons | JABSTB (Murphy)](https://tjmurphy.github.io/jabstb/posthoc.html)\n12. [Zbyněk Šidák (1967). Rectangular Confidence Regions for the Means of Multivariate Normal Distributions. Journal of the American Statistical Association.](https://doi.org/10.1080/01621459.1967.10482935)\n13. [Multiple comparison tests – Textbook of Agricultural Statistics](https://statoberrypapaya.github.io/TextbookofAgstat/multipleComparison.html)\n14. [Multiplicity Adjustments: Understanding the Nuance of Post hoc Tests (Paul Schmidt, BioMath GmbH, December 13, 2023)](https://schmidtpaul.github.io/dsfair_quarto/ch/summaryarticles/multiplicityadj.html)\n15. [The Quest for α: Developments in Multiple Comparison Procedures in the Quarter Century Since Games (1971) (Review of Educational Research)](https://journals.sagepub.com/doi/10.3102/00346543066003269)\n16. [HENRY SCHEFFÉ (1953). A METHOD FOR JUDGING ALL CONTRASTS IN THE ANALYSIS OF VARIANCE *. Biometrika.](https://doi.org/10.1093/biomet/40.1-2.87)\n17. [Charles W. Dunnett (1955). A Multiple Comparison Procedure for Comparing Several Treatments with a Control. Journal of the American Statistical Association.](https://doi.org/10.1080/01621459.1955.10501294)\n18. [Olive Jean Dunn (1961). Multiple Comparisons among Means. Journal of the American Statistical Association.](https://doi.org/10.1080/01621459.1961.10482090)\n19. [Sture Holm (1979). A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics.](https://doi.org/10.2307/4615733)\n20. [12.6: ANOVA post hoc tests (stats.libretexts.org)](https://stats.libretexts.org/Bookshelves/Applied_Statistics/Mikes_Biostatistics_Book_%28Dohm%29/12%3A_One-way_Analysis_of_Variance/12.6%3A_ANOVA_post-hoc_tests)\n21. [Anthony J. Hayter (1984). A Proof of the Conjecture that the Tukey-Kramer Multiple Comparisons Procedure is Conservative. The Annals of Statistics.](https://doi.org/10.1214/aos/1176346392)\n22. [Paul A. Games, John F. Howell (1976). Pairwise Multiple Comparison Procedures with Unequal N’s and/or Variances: A Monte Carlo Study. Journal of Educational Statistics.](https://doi.org/10.3102/10769986001002113)\n23. [Ajit C. Tamhane (1979). A Comparison of Procedures for Multiple Comparisons of Means with Unequal Variances. Journal of the American Statistical Association.](https://doi.org/10.1080/01621459.1979.10482541)\n24. [Charles W. Dunnett (1980). Pairwise Multiple Comparisons in the Unequal Variance Case. Journal of the American Statistical Association.](https://doi.org/10.1080/01621459.1980.10477552)\n25. [Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test (PeerJ, 2020)](https://pubmed.ncbi.nlm.nih.gov/33335808/)\n26. [Post Hoc Power Calculations: An Inappropriate Method for Interpreting the Findings of a Research Study (Journal of Rheumatology)](https://www.jrheum.org/content/49/8/867)\n27. [Yiran Zhang and colleagues (2019). Post hoc power analysis: is it an informative and meaningful analysis?. General Psychiatry.](https://doi.org/10.1136/gpsych-2019-100069)\n28. [Ajit C. Tamhane, Dong Xi, Jiangtao Gou (2021). Group sequential Holm and Hochberg procedures. Statistics in Medicine.](https://doi.org/10.1002/sim.9128)\n\n---\n*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Statistical inference, estimation, sampling, and testing › Hypothesis testing › False discovery rate and error-rate control*\n\n*Initially written Sep 29, 2026 · Edited: Oct 15, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [ "https://www.uvm.edu/~statdhtx/methods8/Supplements/OldChapter12.pdf" ], "url": "https://www.edgepedia.org/post-hoc-analysis", "markdown_url": "https://www.edgepedia.org/post-hoc-analysis.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Post-hoc analysis\", Edgepedia (EdgeChat), https://www.edgepedia.org/post-hoc-analysis. Edgepedia Community License 1.0.", "credit_md": "\"[Post-hoc analysis](https://www.edgepedia.org/post-hoc-analysis)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/post-hoc-analysis](https://www.edgepedia.org/post-hoc-analysis). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Post-hoc analysis\", Edgepedia (EdgeChat), https://www.edgepedia.org/post-hoc-analysis. Edgepedia Community License 1.0.", "speakable": "A post-hoc analysis is a comparison of treatment means chosen after the data are collected, usually to locate which group differences drive a significant ANOVA result." }