Tutorial 7: Differential Item Functioning(201381) 

Home 1. Install R 2. Install TAM 3. Rasch Model 4. CTT, Fit 5. Partial Credit Model 6. Population Model 7. DIF More resources 
Summary of TutorialThis tutorial shows how to carry out differential item functioning analysis.
R script fileThe R script file for this tutorial can be downloaded through the link Tutorial7.R .Data filesThe data file is from FIMS (The First International Mathematics Study, IEA), the same file as for the Exercises in Tutorial 4. The test questions and data file can be downloaded from the following links. FIMS Test A (part) QuestionsFIMS Test A Australian data Load R librariesBefore TAM functions can be called, the TAM library has to be loaded into the R workspace. Line 2 loads the TAM library. Read data file
Line 5 sets the working directory and line 6 reads the data file
"FIMS_AUS_TestA.csv"
into a variable called "FIMSdata". Line 7 extracts the item responses into
a variable called "raw_resp", as column 1 contains the gender
variable. Score the data fileLines 1011 specify the keys and score the item responses.key < c(1, 3, 1, 4, 1, 3, 3, 1, 5, 1, 1, 4, 1, 1) scored < sapply( seq(1,length(key)), FUN = function(ii){ 1*(raw_resp[,ii] == key[ii]) } ) Run IRT analysis
Line 14 runs an IRT analysis with MML estimation. Results of the IRT analysis can be show using R command "summary(mod1)". Line 17 extracts values of the gender variable into a variable called "gender". Line 20 computes the test score for each student by calculating the row sum of each student's scored responses. Line 23 computes the mean test score for each gender group: 1=male, and 2=female The mean test score is 6.12 for group 1 (males) and 6.27 for group 2 (females). That is, the two groups performed similarly, with girls having a slightly higher mean test score. The step of computing raw test scores is not necessary for the IRT analyses. But it's always a good practice to explore the data a little before delving into more complex analyses. Facets analysis
To conduct a DIF analysis, we set up the variable "gender" as a facet and rerun the IRT analysis. Lines 2628 show the R code to do this.
Line 31 shows the results.
The interaction term in the facet formula, item*gender, provides the magnitudes of item DIF. An excerpt of the output is shown below. A quick significance test can be conducted by dividing the interatcion estimate by its standard error. For example, for item 5, calculate 0.332/0.030= 11. Compare this with a zstatistic (within 2 and 2?) The second way is to consider the effect size of DIF. For item 5, the difference in item difficulty between the two gender groups is 2x0.332 = 0.66. How siginificant in real terms is this magnitude? Exercises
The following data set contains FIMS data for both Australia and Japan. The country code is in column 16. The gender code is in column 1.
Carry out a DIF analysis with respect to "country" (i.e., difference between Australia and Japan). 