The superiority of the item response theory (IRT) over the classical
test theory (CTT) is known to IRT researchers from the detailed
test evaluation view-point. However, teachers are still reluctant
to use the IRT as a daily testing tool. The primary objective of
this paper is to find the difference between the CTT and the IRT.
In particular, we focus on the difference of ability evaluation.
We compared the CTT abilities and the IRT abilities using the
hypothetically assumed abilities which are mimicked to a real case.
By using a simulation study, we have found that the IRT is superior
to the CTT to some extent.
The CTT uses the pre-assigned allotments contrary to the IRT with no allotment
concept.
However, if we regard the ability evaluation by the IRT as a standard, we can
find the most appropriate allotments in the CTT so that the total scores of
the CTT are adjusted as close as to the abilities obtained by the IRT. This
is a kind of allotment optimization problem. We show this methodology in this
paper.
By applying our methodology to some simulation cases which mimic the real data
case, we have found an intriguing feature with respect to the pre-assigned
allotments.
If teachers want to raise the examination pass rate, we guess
that they give higher scores (than actual scores) to lower students;
we call this jack-up.
Using the allotment optimization, we have found that the jack-up causes higher
allotments to easier problems in the CTT.
|