![]()
Ren B. Kline Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research (2004). American Psychological Association.
AUG/SEPT 2004 |
SRNT NewsletterAugust/September 2004, Volume 10, Number 3 Book Review
This text describes the historical genesis of null hypothesis significance testing (NHST), which has become the modern notion of statistics, and in many people's opinion, the de facto method by which all behavioral and social science should be practiced and pursued. An interesting debate that is currently being discussed in depth on the Chronicle of Higher Education web site will allow readers to better understand the competing approaches that are presented in this book and to contemplate where they stand on some of the issues http://chronicle.com/free/v50/i38/38a01201.htm he NHST approach is ubiquitous throughout higher education curricula and is very nearly the exclusive focus of applied statistical methods as they are taught. NHST is actually a hybrid of two different conceptual approaches that evolved and became institutionalized during the mid-20th century. The text suggests that this approach has become dogma that is largely practiced excessively and uncritically. Many studies are typically underpowered (i.e., 0.50 or less) and often violate assumptions of the statistical procedures used (i.e., assumption of a random sample). Furthermore, research findings are not replicated as often as they should be. Whether the `solution' involves a wholesale minimization or elimination of (NHST) - as the author tends to suggest - or whether the misuses described can be addressed systematically, thus eliminating the `weaknesses', will continue to be a focus of considerable debate and discussion. Quite certainly, this is not a small problem; the following article in The Economist discusses the problem of `sloppy statistics' in the scientific literature http://www.economist.com/science/displayStory.cfm?story_id=2724226. The text presents clear and concise arguments - grounded in a growing literature - about the limitations and misuse of commonly utilized statistical tests. A number of alternative data analytic methods are also presented which readers can ponder and use to stretch their methodological capabilities. These include effect size and confidence interval estimation, meta-analysis, re-sampling techniques like bootstrapping, and an introduction to Bayesian estimation. These alternatives are not presented as magical replacements to traditional statistics, but as possibly more robust approaches to data analysis. The author acknowledges that, while they each have some merit, every method/ approach - even the proposed alternatives - has its own potential problems as well. This book would make a good textbook choice for an intermediate or advanced graduate/professional-level course in which students already have had some solid exposure to statistics and research methods. While the material presented in the first three chapters would be a welcome and beneficial addition to either an undergraduate or graduate curriculum, the remaining six chapters present such a substantial quantity of technical and abstract information that they would be too advanced for students who have not had a good foundation in statistics. A web-based resource page available at http://www.apa.org/books/resources/kline provides excellent examples and answers for Chapters 2 thru 7, as well as links to sites where practice data files or free software may be downloaded. This book will challenge you to think about how you use statistical tests in your approach to conducting good science in the field of nicotine and tobacco research. It will also most certainly force you to re-assess and re-evaluate your interpretation of traditional statistics that are used for null hypothesis significance testing (NHST). This issue will become increasingly important, as we continue to strive to produce the best science possible. See http://www.apa.org/books/4316031.html for more details.
Joseph Bauer, Ph.D.
|
|