運用試題反應理論計算多向度能力分配中量尺總分之探討

Abstract

本研究使用實徵資料，探討應用試題反應理論計分模式在多向度能力分配上，對於測驗量尺總分的計分效果。本研究的目的在使用Kolen、Wang 和Lee (2012) 的方法建立以試題反應理論模式為基礎的程式，在測驗組合、各學科所測量能力彼此互為相關的情境下，計算量尺總分，然後再對所得總分不同的特性進行分析與探討。本研究使用國中基測2008 年第一次測驗5,000筆考生的隨機資料；評鑑的標準包含考生在多向度能力分配裡所得量尺總分之描述統計值與分數分配圖形、總誤差量、信度值，以及測量標準誤。研究結果顯示國中基測各學科所測量的能力彼此存在高度的相關，Kolen 等人的試題反應理論計分程序有令人滿意的表現。對於合併使用兩種或兩種測驗以上作影響性決定的大型測驗而言，如何將不同的測驗分數結果整合是相當重要的議題；本研究建立多向度能力分配、使用實徵資料，探討以Kolen 等人的模式計算量尺總分的效果，相信研究的結果已提供了更多有關量尺總分的測量特性，對於日後使用試題反應理論模式，在多向度能力情境中計算考生能力的相關研究上也提供了更有力的基礎 。

This study was designed to explore the composite scores under the multivariate distribution of latent proficiencies using a procedure based on IRT for empirical data. The purpose was to employ the IRT models under the multivariate ability distribution in order to investigate the composite scores with the consideration that correlations existed among the examinees’ proficiencies. This research followed Kolen, Wang, and Lee’s (2012) method for the development of the IRT-based procedure. The five tests of the Basic Competence Test were used, with a random sample of 5,000 examinees obtained in 2008. The analyses included the descriptive statistics and frequency distributions of the composite scores, overall SEM and reliability values, as well as the CSEMs. The findings indicated that there existed strong relationships between pairs of the proficiencies of the five tests and the results of Kolen et al.’s modeling were satisfied. For large-scale testing programs that rely on two or more tests to make high-stakes decisions, the issue of combining individual scores into a single total score is critical. Through the employment of Kolen et al.’s model for deriving the composite scores under the multivariate distribution, and along with this research utilizing the real data, the results of this study have revealed more of the psychometric features of the composite scores and have also helped lay a more solid foundation for many studies to embark on the estimation of examinees’ ability levels in the multivariate proficiency setting via IRT.