Comparison of Ability Evaluation in Testing between the Classical Test Theory and the Item Response Theory


H. Hirose


The 11th Annual Hawaii International Conference on Education, pp.2955-2967, January 6 - 9, 2013, Hawaii, USA


The superiority of the item response theory (IRT) over the classical test theory (CTT) is known to IRT researchers from the detailed test evaluation view-point. However, teachers are still reluctant to use the IRT as a daily testing tool. The primary objective of this paper is to find the difference between the CTT and the IRT. In particular, we focus on the difference of ability evaluation.

We compared the CTT abilities and the IRT abilities using the hypothetically assumed abilities which are mimicked to a real case. By using a simulation study, we have found that the IRT is superior to the CTT to some extent.
The CTT uses the pre-assigned allotments contrary to the IRT with no allotment concept.
However, if we regard the ability evaluation by the IRT as a standard, we can find the most appropriate allotments in the CTT so that the total scores of the CTT are adjusted as close as to the abilities obtained by the IRT. This is a kind of allotment optimization problem. We show this methodology in this paper.
By applying our methodology to some simulation cases which mimic the real data case, we have found an intriguing feature with respect to the pre-assigned allotments.

If teachers want to raise the examination pass rate, we guess that they give higher scores (than actual scores) to lower students; we call this jack-up.
Using the allotment optimization, we have found that the jack-up causes higher allotments to easier problems in the CTT.

Key Words
ability evaluation; classical test theory; item response theory; allotment optimization; jack-up; least square; gradient descent.



Times Cited in Web of Science:

Cited in Books: