Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Contributions to Kernel Equating
Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Social Sciences, Department of Statistics.ORCID iD: 0000-0002-9007-2440
2014 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The statistical practice of equating is needed when scores on different versions of the same standardized test are to be compared. This thesis constitutes four contributions to the observed-score equating framework kernel equating.

Paper I introduces the open source R package kequate which enables the equating of observed scores using the kernel method of test equating in all common equating designs. The package is designed for ease of use and integrates well with other packages. The equating methods non-equivalent groups with covariates and item response theory observed-score kernel equating are currently not available in any other software package.

In paper II an alternative bandwidth selection method for the kernel method of test equating is proposed. The new method is designed for usage with non-smooth data such as when using the observed data directly, without pre-smoothing. In previously used bandwidth selection methods, the variability from the bandwidth selection was disregarded when calculating the asymptotic standard errors. Here, the bandwidth selection is accounted for and updated asymptotic standard error derivations are provided.

Item response theory observed-score kernel equating for the non-equivalent groups with anchor test design is introduced in paper III. Multivariate observed-score kernel equating functions are defined and their asymptotic covariance matrices are derived. An empirical example in the form of a standardized achievement test is used and the item response theory methods are compared to previously used log-linear methods.

In paper IV, Wald tests for equating differences in item response theory observed-score kernel equating are conducted using the results from paper III. Simulations are performed to evaluate the empirical significance level and power under different settings, showing that the Wald test is more powerful than the Hommel multiple hypothesis testing method. Data from a psychometric licensure test and a standardized achievement test are used to exemplify the hypothesis testing procedure. The results show that using the Wald test can provide different conclusions to using the Hommel procedure.

Place, publisher, year, edition, pages
Uppsala: Acta Universitatis Upsaliensis, 2014. , 24 p.
Series
Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Social Sciences, ISSN 1652-9030 ; 106
Keyword [en]
observed-score test equating, item response theory, R, equipercentile equating, asymptotic standard errors, non-equivalent groups with anchor test design
National Category
Probability Theory and Statistics
Research subject
Statistics
Identifiers
URN: urn:nbn:se:uu:diva-234618ISBN: 978-91-554-9089-8 (print)OAI: oai:DiVA.org:uu-234618DiVA: diva2:757561
Public defence
2014-12-12, Sal IV, Universitetshuset, Biskopsgatan 3, Uppsala, 10:15 (English)
Opponent
Supervisors
Available from: 2014-11-20 Created: 2014-10-21 Last updated: 2015-02-03
List of papers
1. Performing the Kernel Method of Test Equating with the Package kequate
Open this publication in new window or tab >>Performing the Kernel Method of Test Equating with the Package kequate
2013 (English)In: Journal of Statistical Software, ISSN 1548-7660, E-ISSN 1548-7660, Vol. 55, no 6, 1-25 p.Article in journal (Refereed) Published
Abstract [en]

In standardized testing it is important to equate tests in order to ensure that the test takers, regardless of the test version given, obtain a fair test. Recently, the kernel method of test equating, which is a conjoint framework of test equating, has gained popularity. The kernel method of test equating includes five steps: (1) pre-smoothing, (2) estimation of the score probabilities, (3) continuization, (4) equating, and (5) computing the standard error of equating and the standard error of equating difference. Here, an implementation has been made for six different equating designs: equivalent groups, single group, counterbalanced, non-equivalent groups with anchor test using either chain equating or post-stratification equating, and non-equivalent groups using covariates. An R package for the kernel method of test equating called kequate is presented. Included in the package are also diagnostic tools aiding in the search for a proper log-linear model in the pre-smoothing step for use in conjunction with the R function glm.

Place, publisher, year, edition, pages
American Statistical Association, 2013
Keyword
observed-score test equating, R package, kernel equating, item-response theory
National Category
Probability Theory and Statistics
Research subject
Statistics
Identifiers
urn:nbn:se:uu:diva-208912 (URN)000325948000001 ()
Available from: 2013-10-23 Created: 2013-10-10 Last updated: 2017-12-06Bibliographically approved
2. Improving the Bandwidth Selection in Kernel Equating
Open this publication in new window or tab >>Improving the Bandwidth Selection in Kernel Equating
2014 (English)In: Journal of educational measurement, ISSN 0022-0655, E-ISSN 1745-3984, Vol. 51, no 3, 223-238 p.Article in journal (Refereed) Published
Abstract [en]

We investigate the current bandwidth selection methods in kernel equating and propose a method based on Silverman's rule of thumb for selecting the bandwidth parameters. In kernel equating, the bandwidth parameters have previously been obtained by minimizing a penalty function. This minimization process has been criticized by practitioners for being too complex and that it does not offer sufficient smoothing in certain cases. In addition, the bandwidth parameters have been treated as constants in the derivation of the standard error of equating even when they were selected by considering the observed data. Here, the bandwidth selection is simplified, and modified standard errors of equating (SEEs) that reflect the bandwidth selection method are derived. The method is illustrated with real data examples and simulated data.

Place, publisher, year, edition, pages
Blackwell Publishing, 2014
Keyword
kernel equating, observed-score test equating
National Category
Probability Theory and Statistics
Research subject
Statistics
Identifiers
urn:nbn:se:uu:diva-223988 (URN)10.1111/jedm.12044 (DOI)000341592200001 ()
Available from: 2014-04-29 Created: 2014-04-29 Last updated: 2017-12-05Bibliographically approved
3. Item Response Theory Observed-Score Kernel Equating
Open this publication in new window or tab >>Item Response Theory Observed-Score Kernel Equating
2017 (English)In: Psychometrika, ISSN 0033-3123, E-ISSN 1860-0980, Vol. 82, no 1, 46-66 p.Article in journal (Refereed) Published
Abstract [en]

Item response theory (IRT) observed-score kernel equating is introduced for the non-equivalent groups with anchor test equating design using either chain equating or post-stratification equating. The equating function is treated in a multivariate setting and the asymptotic covariance matrices of IRT observed-score kernel equating functions are derived. Equating is conducted using the two-parameter and three-parameter logistic models with simulated data and data from a standardized achievement test. The results show that IRT observed-score kernel equating offers small standard errors and low equating bias under most settings considered.

Keyword
observed-score equating, item response theory, equipercentile equating, standard errors, NEAT design
National Category
Probability Theory and Statistics
Identifiers
urn:nbn:se:uu:diva-233484 (URN)10.1007/s11336-016-9528-7 (DOI)000394985400003 ()27743280 (PubMedID)
Funder
Swedish Research Council, 2014-578
Available from: 2014-10-21 Created: 2014-10-06 Last updated: 2017-04-20Bibliographically approved
4. An Evaluation of Hypothesis Testing Methods for Equating Differences in Kernel Equating
Open this publication in new window or tab >>An Evaluation of Hypothesis Testing Methods for Equating Differences in Kernel Equating
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In observed-score equating, hypothesis tests of equating differences are helpful in deciding which equating function is suitable. Here, a hypothesis testing procedure for item response theory (IRT) observed-score kernel equating using a Wald test is introduced. Simulations evaluating the Wald test when using IRT and log-linear models are conducted. The test with either IRT or log-linear models is shown to have high power and greatly outperform the Hommel multiple hypothesis testing method. The Wald test is applied to two datasets in both an equivalent groups design and a non-equivalent groups design, showing that the Wald test can provide different conclusions to other hypothesis testing methods in practice.

National Category
Probability Theory and Statistics
Identifiers
urn:nbn:se:uu:diva-233486 (URN)
Available from: 2014-10-21 Created: 2014-10-06 Last updated: 2015-02-03

Open Access in DiVA

fulltext(366 kB)389 downloads
File information
File name FULLTEXT01.pdfFile size 366 kBChecksum SHA-512
4dc9534fea5edac046151ec921bf5215eb421e542dc8aba03ed15fce57351f3e7209d99f7140ae3d628294b428290a538e3ace130452c630f2d9a0dc3c695ae8
Type fulltextMimetype application/pdf
Buy this publication >>

Authority records BETA

Andersson, Björn

Search in DiVA

By author/editor
Andersson, Björn
By organisation
Department of Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 389 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 958 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf