Evaluation of Abilities by Grouping for Small IRT Testing Systems


Y. Tokusada, H. Hirose


5th International Conference on Learning Technologies and Learning Environments (LTLE2016), pp.445-449, July 10-14, 2016, Kumamoto, Japan.

In applying the adaptive testing equipped with the item response theory to actual cases, it is crucial to know the appropriate number of question items suffice for the specified accuracy.
In almost all the systems we have developed so far, we adopt small number of items in a sequence of questions for adaptive testing because too many questions will bore examinees or force to give up completing the tests although estimation accuracy can be obtained. By experience, we set the number of questions to be five. However, too small number of questions will cause less accurate estimates for examinees' abilities. In this paper, we show how the number of questions influences the accuracy of the estimates.
For the sake of stable estimation, we made use of Bayes method, which may make shrinkage estimation. In order to make sense to use the estimates, we propose grouping, or classifying to examinees' abilities. The optimal number of groups will be shown as a result.
By using simulation studies, we have found that five question items are insufficient to estimate the examinee's true ability. At least ten items are required even if the testing is adaptive.

Key Words
number of sufficient items, misclassification rate, confusion matrix, item response theory, adaptive online testing.



Times Cited in Web of Science:

Cited in Books: