Last modified: 2017-03-31
Abstract
The success with chance is one of the major factors causing the error in measurement results. Therefore, it is already one of the most research topic on the multiple choice test and true-false tests. The effects of guessing on differential item functioning, equating, internal consistency and criterion validity is among the most studied topics. There are few studies in the literature regarding the effect on test dimensionality. On the other hand, skewness can be a variable that effects both guessing and test dimensionality. Since guessing and skewness effects the variance of test scores, items variance, covariance and correlation between items, test' reliability and validity, effects of guessing and skewness on tests’ dimensionality can be expected. Thus, reliability of tests and size of the explained variance can also be expected to vary.
Aims of this study to determine to effects of guessing and skewness on test dimensionality, the variance explained by the dimensions, the root mean square error of approximation (RMSEA) and McDonald's omega reliability.
This study was conducted on simulation data. Research has been studied in one-dimensional and multi-dimensional simple structures. Test length, guessing level, the correlation between the dimensions, number of items in the dimensions, distribution of the student ability (positive skewed and negative skewed) and sample size are taken into account. Analysis of test dimensionality was done by Factor 10.3 software. In determine of factor number, Optimal Parallel Analysis were used. In factor analysis, tetrachoric correlations between items are taken into account.
According to the results, guessing increases number of factors and reduces the explained variance of the dominant factors. Interaction of guessing with skewness were found high. Also when guessing increased, reliability reduced and the root mean square error of approximation (RMSE) was increased.
When using multiple-choice or true-false items, guessing should be taken into consideration and statistical techniques should be utilized to fix it.