Change search
ReferencesLink to record
Permanent link

Direct link
Testing rating scale unidimensionality using the Principal Component Analysis (PCA)/t-test protocol with the Rasch Model: the primacy of theory over statistics
Kristianstad University, School of Health and Society, Avdelningen för Hälsovetenskap I. Kristianstad University, Research Environment PRO-CARE. Kristianstad University, Forskningsplattformen Hälsa i samverkan.ORCID iD: 0000-0003-2174-372X
2014 (English)In: Open Journal of Statistics, ISSN 2161-718X, E-ISSN 2161-7198, Vol. 4, no 6, 456-465 p.Article in journal (Refereed) Published
Abstract [en]

Psychometric theory requires unidimensionality (i.e., scale items should represent a common latent variable). One advocated approach to test unidimensionality within the Rasch model is to identify two item sets from a Principal Component Analysis (PCA) of residuals, estimate separate person measures based on the two item sets, compare the two estimates on a person-by-person basis using t-tests and determine the number of cases that differ significantly at the 0.05-level; if ≤5% of tests are significant, or the lower bound of a binomial 95% confidence interval (CI) of the observed proportion overlaps 5%, then it is suggested that strict unidimensionality can be inferred; otherwise the scale is multidimensional. Given its proposed significance and potential implications, this procedure needs detailed scrutiny. This paper explores the impact of sample size and method of estimating the 95% binomial CI upon conclusions according to recommended conventions. Normal approximation, “exact”, Wilson, Agresti-Coull, and Jeffreys binomial CIs were calculated for observed proportions of 0.06, 0.08 and 0.10 and sample sizes from n= 100 to n= 2500. Lower 95%CI boundaries were inspected regarding coverage of the 5% threshold. Results showed that all binomial 95% CIs included as well as excluded 5% as an effect of sample size for all three investigated proportions, except for the Wilson, Agresti-Coull, and JeffreysCIs, which did not include 5% for any sample size with a 10% observed proportion. The normal approximation CI was most sensitive to sample size. These data illustrate that the PCA/t-test protocol should be used and interpreted as any hypothesis testing procedure and is dependent on sample size as well as binomial CI estimation procedure. The PCA/t-test protocol should not be viewed as a “definite” test of unidimensionality and does not replace an integrated quantitative/qualitative interpretation based on an explicit variable definition in view of the perspective, context and purpose of measurement.

Place, publisher, year, edition, pages
2014. Vol. 4, no 6, 456-465 p.
Keyword [en]
Confidence Intervals, Dimensionality, Psychometrics, Rasch Model, Validity
National Category
Probability Theory and Statistics
URN: urn:nbn:se:hkr:diva-12826DOI: 10.4236/ojs.2014.46044OAI: diva2:743817
Available from: 2014-09-05 Created: 2014-09-05 Last updated: 2014-12-07Bibliographically approved

Open Access in DiVA

fulltext(2648 kB)67 downloads
File information
File name FULLTEXT01.pdfFile size 2648 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Hagell, Peter
By organisation
Avdelningen för Hälsovetenskap IResearch Environment PRO-CAREForskningsplattformen Hälsa i samverkan
In the same journal
Open Journal of Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 67 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 171 hits
ReferencesLink to record
Permanent link

Direct link