To enhance the chance of use of the item response theory (IRT) in universities, we developed a test evaluation system via the Web for university teachers, and we have been evaluating students' abilities by using the IRT system in midterm and final examinations for two years.
We show a surprising aspect regarding the adoption of the IRT system in university tests. That is, the IRT can not only give us the problem difficulty information but also can provide the accurate student ability evaluation, even if the number of problems is small. Therefore, we can include high and low level test items together so that we can assess a variety of students' abilities accurately and fairly; we do not worry about providing easier problems that will make the lecture level decline; in other words, we do not care about finding the most appropriate problem levels to each student. We can provide all level problems uniformly distributed to all students, and we can still assess the students' abilities accurately. Consequently, students do not raise claims about their scores; they seem to be satisfied with it.
We show these results, in this paper, by a theoretical background, a simulation study, and our empirical results.