Research Methods in AL

Why I Keep Insisting on Equal Group Sizes in ANOVA and MANOVA 🧪📊 Every time I run a workshop on comparing groups, someone asks me the same question: "Does it really matter if my groups are not equal in size?" 🤔 My answer is always the same. It matters, but not for the reason most people assume. It is not about elegance or symmetry. It is about what happens to your Type I error rate when an assumption is violated. ⚠️ Let me explain what I mean. 🗣️ 1️⃣ Equal n protects you when homogeneity is violated 🛡️ We all know ANOVA assumes homogeneity of variances, and MANOVA assumes homogeneity of variance-covariance matrices, which is what Box's M is testing. 📉 What is less widely appreciated is that this assumption is remarkably forgiving when your groups are balanced. The F statistic simply absorbs the violation. 📉 Once your groups are unbalanced, the violation starts to bite, and it bites in a predictable direction: 🦷 * If your larger group has the larger variance, the test becomes conservative. You lose power and you may miss a real effect. ⬇️ * If your smaller group has the larger variance, the test becomes liberal. Your nominal alpha of .05 may in reality be .10 or worse, and you end up reporting an effect that is not there. ⬆️ This is exactly why so many of us are relaxed about Box's M in a balanced design and much more careful in an unbalanced one. Under imbalance, I would move to Pillai's trace for the multivariate test, and to Welch's F or Brown-Forsythe for the univariate follow-ups. 🔄 2️⃣ Unequal cells break the orthogonality of your factorial design 🧱 This one is often missed in our field. In a balanced factorial design, main effects and interactions are independent, and the sums of squares partition cleanly. ✂️ Once cells are unequal, the effects become correlated, and Type I, Type II, and Type III sums of squares no longer agree. 🤝 Your substantive conclusion can therefore depend on a default setting in your software that you never consciously chose. 💻 In a balanced design, this problem simply does not arise. ✅ 3️⃣ You are wasting participants 🏃‍♂️💨 For a fixed total sample size, power is maximized when groups are equal. Effective sample size is driven by the harmonic mean, so a 20 versus 80 split behaves roughly like 32 per group rather than 50 per group. 📉 You collected 100 people and you are getting the power of 64. In our context, where data collection is rarely easy, this is a real cost. 💸 4️⃣ A note specific to MANOVA 📌 MANOVA is even less tolerant, because covariance homogeneity and multivariate normality interact, and multivariate outliers do far more damage in a small cell. 🚫 At minimum, every cell must contain more cases than you have dependent variables, and I would want a comfortable margin beyond that, not a technical minimum. 📏 So how much imbalance is acceptable? 🤷‍♂️ My working rule is to keep the ratio of the largest to the smallest group below 1.5 to 1. Beyond 4 to 1, I would not trust the standard test without serious checking. 🔍 And a plea about reporting 📝 When I review manuscripts, I still see authors report a non-significant Box's M and move straight on, with the cell sizes buried somewhere or absent altogether. 🕵️‍♂️ Please report your cell sizes, your variance ratio, and which multivariate criterion you used and why. It takes two sentences, and it is the difference between a reader being able to evaluate your analysis and simply having to trust it. 🤝 If you are designing your study now, this is one of the few methodological problems you can solve completely in advance, at zero cost, by planning for balanced groups. 🎯 Once the data are collected, all we can do is manage the damage. 🚑 Hessam

Just out: Statistical Modeling for (Actual) Hypothesis Testing (Tove Larsson & Gregory R. Hancock) Free to download until August 11: https://doi.org/10.1017/9781009660877 Keywords: Cumulative knowledge building, empirically informed hypotheses, null hypothesis significance testing, structural equation modeling Abstract: By building knowledge in a deliberate and systematic manner, we can gain a more complete understanding of a given research area relevant to corpus linguists. Specifically, empirically informed hypotheses (i.e., hypotheses that result from a synthesis of findings from all relevant prior studies) play a key role in this endeavor in that they enable us to test to what extent generalizations from previous research are consistent with our results, or if we need to make adjustments to our existing knowledge or theory. In this Element, we aim to provide a practical and accessible introduction to select statistical methods for evaluating such empirically informed hypotheses. In particular, we illustrate techniques from the broader null-hypothesis significance testing framework (e.g., equivalence testing), and structural equation modeling framework (e.g., measured variable path analysis), with the goal of encouraging knowledge building in a more principled and systematic manner in corpus linguistics.

Ethics and positionality are not peripheral sections of qualitative research—they shape every relationship, decision, interpretation, and representation we produce. This poster offers a concise reminder that ethical qualitative inquiry in applied linguistics requires ongoing consent, attention to power and language, contextual privacy, reflexivity, and interpretive care. The central question remains: What am I making visible—and what might I be obscuring?

📚 آکادمی PREET برگزار می‌کند: 🔴 کارگاه *Doing Qualitative Research in Applied Linguistics* 🔹 مدرس: دکتر محمدرضا هاشمی 🔹طول دوره: ۶ هفته (۶ جلسه دو ساعته) 🔹 زمان برگزاری: از ۳۱ تیر ۱۴۰۵، روزهای چهارشنبه ساعت ۱۶:۳۰ الی ۱۸:۳۰ 🔹 دوره در پلتفرم *ادوبی کانکت* برگزار می‌گردد و دارای پشتیبانی می‌باشد. 📚 PREET center presents: 🔴  Workshop on Doing Qualitative Research in Applied Linguistics 🔹 Instructor: Dr. Mohammad R. Hashemi 🔹 Duration: Six weeks (six sessions) 🔹 Schedule: Wednesdays 16:30-18:30 🔹 The workshops will be held on the Adobe Connect platform and will have the support. 📣  Registration has started. (Limited capacity) ▪️ For more information and registration: Telegram: @preetadmin WhatsApp & SMS: +989359289774 ▪️For each participant in the workshop, a digital certificate signed by Dr. Minoo Alemi, Dr. Zia Tajeddin, and Dr. Mohammad R. Hashemi will be issued.

Between-subjects factor × within-subjects factor: technical essentials for Applied Linguistics researchers (By Hessameddin Ghanbar)

Boosting The Quality of Ethics In Applied Linguistics Ghanbar, H., Kim, N., Cinaglia, C., & De Costa, P. I. (2026). The principle-practice gap: A methodological synthesis of discrepancies between narrative inquiry ethical ideals and actual reporting practices. Language Teaching, 1-43.‏

See more in this source: Riazi, M., Ghanbar, H., & Rezvani, R. (2023). Qualitative data coding and analysis: A systematic review of the papers published in the Journal of Second Language Writing. Iranian Journal of Language Teaching Research, 11(1), 25-47.‏ https://ijltr.urmia.ac.ir/article_121271.html

The function of IMPA in SEM and validation

I love complexity and hidden agendas, so I believe, in SEM studies, researchers should not rely solely on direct effects. Here is my point on mediation analysis and indirect effects.