Letter to the Editor

Alpha Psychiatry 2022;23(2):91-92

DOI: 10.5152/alphapsychiatry.2022.2201222

Answer: Instrument Validation Is a Necessary, Comprehensive, and Permanent Process

Mehmet Emrah Karadere

Kürşad Çifteci

Rümeysa Yeni Elbay

Hakan Yılmaz

Hasan Turan Karatepe

Department of Psychiatry, İstanbul Medeniyet University School of Medicine, İstanbul, Turkey

Corresponding author: Kürşad Çifteci ✉ kursadcifteci@gmail.com

Cite this article as: Karadere ME, Çifteci K, Elbay RY, Yılmaz H, Karatepe HT. Answer: instrument validation is a necessary, comprehensive, and permanent process. Alpha Psychiatry. 2022;23(2):91-92.

Dear Editor,

It might have been confusing that concurrent, convergent, and discriminant validity analyses were given under the same heading in the article. Scales other than the Depression Anxiety Stress Scale were used for convergent analyses. It was pointed out that one of the assumptions of the Confirmatory factor analysis (CFA) is the irritability attribute being typically distributed. We agree with that, it should have been reported in the article whether it was typically distributed. Also, using the same sample for the exploratory and confirmatory factor analysis was discussed. This is a perfectly justified criticism. Even though exploratory factor analysis (EFA) was used, we have to note that the eigenvalue was requested during the article evaluation process and the EFA was used to be able to give it.

The brief irritability test is a 6-point Likert-type scale. There is an ongoing debate about whether to treat Likert-type scales as a continuous or ordinal variable.¹ Some writers argue that only non-parametric statistics should be used on Likert scaled data.² There are also other studies suggesting that parametric statistics can be used to analyze Likert-type scale data.³,⁴

Another point made was about the usage of the Shapiro–Wilk test. This is also an interesting topic of discussion. Some researchers recommend using skewness and kurtosis to understand whether the normality is distributed or not.⁵ Some writers suggest Shapiro–Wilk as the most powerful among the other normality tests.⁶ There was also a published study for the Shapiro–Wilk test for sample sizes between 3 and 5000.⁷

Also, it is important to note that for samples with sufficient size, the central limit theorem suggests that the distribution of the mean is always normal.⁸ Therefore, we believe that the normality assumption provided by the Shapiro–Wilk test should not be considered a limitation.

It was also pointed out that correlations greater than 0.30 should be considered significant. This is also another topic that is open for discussion.

Cohen suggests interpreting Pearson correlation coefficients greater than 0.10 as statistically significant. The recommended correlation interpretations are as follows: r = 0.10 to 0.29 or r = –0.10 to –0.29 small ; r = 0.30 to 0.49 or r =–0.30 to –0.4.9 medium; r = 0.50 to 1.0 or r = –0.50 to –1.0 large correlation.⁹ However, all statistical significance does not mean clinical significance.

Additional reliability indicators, such as omega value was suggested. Omega value was already given in the article.

Author Contributions: Concept - M.E.K., K.Ç., R.Y.E., H.Y., H.T.K.; Design - M.E.K., K.Ç.; Supervision - M.E.K., R.Y.E., H.T.K.; Materials - M.E.K.; Data Collection and/or Processing - K.Ç.; Analysis and/or Interpretation - M.E.K., K.Ç., R.Y.E., H.Y., H.T.K.; Literature Review - M.E.K., K.Ç.; Writing - M.E.K., K.Ç., R.Y.E., H.Y., H.T.K.; Critical Review - M.E.K., R.Y.E., H.Y., H.T.K.

Conflict of Interest: The authors have no conflicts of interest to declare.

Financial Disclosure: The authors declared that this study has received no financial support.

References

  1. Carifio, J, & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Med Educ, 42, 12, 1150-1152, (https://doi.org/10.1111/J.1365-2923.2008.03172.X).

  2. Jamieson, S. (2004). Likert scales: how to (ab)use them. Med Educ, 38, 12, 1217-1218, (https://doi.org/10.1111/J.1365-2929.2004.02012.X).

  3. Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Adv. Health Sci. Educ, 15, 625-63, 2, (https://doi.org/10.1007/s10459-010-9222-y).

  4. Sullivan, GM, & Artino, AR. (2013). Analyzing and interpreting data from Likert-type scales. J Grad Med Educ, 5, 4, 541-542, (https://doi.org/10.4300/JGME-5-4-18).

  5. Tabachnick, BG, Fidell, LS, & Ullman, JB. (2007). Using Multivariate Statistics, 5, Boston, MA, , USA, Pearson.

  6. Razali, NM, & Wah, YB. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. J Stat Model Anal, 2, 1, 21-33.

  7. Royston, P. (1992). Approximating the Shapiro-Wilk W-test for non-normality. Stat Comput, 2, 3, 117-119, (https://doi.org/10.1007/BF01891203).

  8. Kwak, SG, & Kim, JH. (2017). Central limit theorem: the cornerstone of modern statistics. Korean J Anesthesiol, 70, 2, 144-156, (https://doi.org/10.4097/kjae.2017.70.2.144).

  9. Cohen, J. (1988). Statistical Power Analysis for the Behavioural Sciences (2nd ed), Hillsdale, NJ, , USA, Lawrence Earlbaum.

Content of this journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Content of this journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.