Letter to the Editor

Alpha Psychiatry 2022;23(2):89-90

DOI: 10.5152/alphapsychiatry.2022.21811

Instrument Validation Is a Necessary, Comprehensive, and Permanent Process

Adalberto Campo-Arias¹

Carlos Alejandro Pineda-Roa²

¹ Programa de Medicina, Universidad del Magdalena, Santa Marta, Colombia

² Programa de Psicología, Universidad del Magdalena, Santa Marta, Colombia

Corresponding author: Adalberto Campo-Arias ✉ acampoa@unimagdalena.edu.co

Cite this article as: Campo-Arias A, Alejandro Pineda-Roa C. Instrument validation is a necessary, comprehensive, and permanent process. Alpha Psychiatry. 2022;23(2):1-2.

Dear Editor,

Recently, we read the work of Karadere et al¹ who presented the psychometric performance of the Turkish version of the brief irritability test (BITe). This is important given the resistance of non-psychometric journals for publishing these articles. However, it is necessary to review the performance of the instruments periodically because these may vary over time from one population to another.²

The article has several technical strengths. However, for lay readers in psychometrics, it is necessary to make some clarifications:

Convergent and discriminant are usually used with other meanings. For example, convergent validation involves applying an instrument that measures the same construct (irritability) and observing the comparability of the scores. Besides, discriminant validity refers to the ability of an instrument to identify the different levels of severity of a trait, that is, the power of the BITe to determine different nuances of irritability unequivocally. The use of the Depression Anxiety Stress Scale-21 possibly only allows us to establish the nomological validity of the observed scores.³

Confirmatory factor analysis works with data assumptions. For example, the irritability attribute must be typically distributed. Therefore, making specific estimates such as estimation by diagonalizable least squares (suitable for non-normal categorical variables) or computation by maximum likelihood is only permissible if there is multivariate normality for confirmatory factor analysis.⁴

Likewise, the authors used the same sample for the exploratory and confirmatory factor analysis. Some authors recommend randomly dividing the total sample into 2. In the present study, the limited size of participants may be a problem.⁵

It is necessary to consider that the BITe has ordinal scores and is not continuous. The authors have had to inform whether they carried out the factorial analyzes on the Pearson matrix or polychoric matrix. This is particularly relevant for exploratory factor analysis.⁴

Likewise, Pearson’s correlations and other estimates require normality. It is pointed out that the usage of the Shapiro–Wilk statistic is erroneous because it applies to samples of less than 50 participants, and in addition, they do not report the significance value.⁴

A correlation should be considered significant if its value is greater than 0.30, with probability values less than .001; a type I error with a sample of more than 100 participants is highly probable.³,⁴

Finally, it is recommended that at least 1 additional reliability indicator, such as McDonald’s omega, be reported, given Cronbach’s alpha limitations.⁵

Peer Review: Externally peer-reviewed.

Author Contributions: Concept - A.CA.; Design - A.CA.; Analysis and/or Interpretation - A.CA., C.A.PR.; Literature Search - A.CA.; Writing - A.CA.; Critical Reviews - C.A.PR.

Conflict of Interest: The authors have no conflict of interest to declare.

Financial Disclosure: The study was funded by Universidad del Magdalena, Santa Marta, Colombia.

References

  1. Karadere, ME, Çifteci, K, Yeni Elbay, R, Yılmaz, H, & Karatepe, HT. (2021). The validity and reliability of the Turkish version of the Brief Irritability Test. Alpha Psychiatry, 22, 6, 318-323, (https://doi.org/10.5152/alphapsychiatry.2021.21370).

  2. Keszei, AP, Novak, M, & Streiner, DL. (2010). Introduction to health measurement scales. J Psychosom Res, 68, 4, 319-323, (https://doi.org/10.1016/j.jpsychores.2010.01.006).

  3. Gregory, RJ. (2014). Psychological Testing: History, Principles, and Applications (7th ed), Boston, Pearson.

  4. Kaplan, RM, & Saccuzzo, DP. (2017). Psychological Testing: Principles, Applications, and Issues (9th ed), Boston, Cengage, , Learning.

  5. Agbo, AA. (2010). Cronbach’s alpha: review of limitations and associated recommendations. J Psychol Afr, 20, 2, 233-239, (https://doi.org/10.1080/14330237.2010.10820371).

Received: December 23, 2021
Accepted: January 23, 2022

Content of this journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Content of this journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.