Instructional Design Student Analysis and Item Analysis

Canvas quiz statistics provide two CSV (Excel) files for you to take a deeper look into student performance and item analysis.

Quiz Summary

Quiz Summary Page 3.png

1 - Section Filter: View quiz statistics for a section

Section Filter

If you teach the same course for multiple sections, you can look at quiz statistics based on a section or as a whole.

2 - Student Analysis: Student information including individual questions that were answered and points that were given

3 - Item Analysis: Analysis of each question in relation to student performance including specific statistical outcomes based on each question

Student Analysis

The following list provides a brief description of each column found within the CSV file:

Student Analysis Graph.png

1 - Student Information: Student name, Canvas ID, and UNC Bear number

Student Information

Column A: Full name

Column B: Unique student ID generated by Canvas

Column C: Unique student ID generated by UNC SIS

2 - Course and Section Information: Course number and section, Canvas-generated section ID, Canvas-generated course ID, and section ID based on year

3 - Submission Information: Date and time of submission and attempt number

Submission Information

Column G: Ordered by "Year-Month-Day Hour:minute:Second" in Universal Coordinated Time (UTC)

Column H: Each time a student submits a quiz, a new row will appear. If a student submits a quiz more than once, the attempt number will increase by one.

4 - Question Information: Question ID, text, and the number of points earned

Question Information

Column I & K: Each question has a back-end, Canvas-specific identification. The question text is also included.

Column H & L: Depending on your quiz question settings, students will earn a number of points. 0 = no points

Item Analysis

The Item Analysis report only displays statistics for Multiple Choice and True/False questions. The following list provides a brief description of each column within the CSV file:

Item Excel Graph 1 - 4.png 1 - Question Information: Question ID, text, and the count of students who were given the question

Question Information

Column A - Question ID: Each question has a back-end, Canvas-specific identification.
Column B - Question Title: The entire question is provided via text.
Column C - Answered Student Count: Helpful when using randomized question banks to see how many students received a particular question.

2 - Top / Middle / Bottom Student Count: Count of students who were given the question based on performance

Top / Middle / Bottom Student Count

Top Middle Bottom Graphic.png

Column D - F: Canvas automatically sorts the top / middle / bottom count for each question based on the student's total quiz grade. The top / middle / bottom performing students are automatically categorized based on their total quiz score: the top 27%, the middle 46%, and the bottom 27%.

Column G - Quiz Question Count: Total number of questions within the quiz.

3 - Correct / Wrong Student Count: Number of students who answered the question correct / wrong

Correct / Wrong Student Count

The sum of both columns (H & I) will equal the Answered Student Count (column C)

4 - Correct / Wrong Student Ratio: Proportion of students who answered correct / wrong

Correct / Wrong Student Ratio

Correct-wrong student count graphic-1.png

Column J - Correct Student Ratio: This number is also found in the Question Breakdown section for each question and is also calculated as the Difficulty Index (column Q). The sum of both proportions will always equal 1.

Item Excel Graph 5 - 9.png

5 - Correct Top / Middle / Bottom Count: Number of students who answered the question correctly based on overall performance

Correct Top / Middle / Bottom Count

Top Middle Bottom Correct Graph-1.png

The overall performance automatically sorts each student into one of the three tiers (top / middle / bottom).

6 - Variance / Standard Deviation: Spread of scores for each question

Variance / Standard Deviation

The greater the number, the greater the spread. Variance is the standard deviation squared.

7 - Difficulty Index: Correct to total number ratio of students who answered the question

Difficulty Index

Column Q - Correct Student Ratio: This number is found in the Question Breakdown section for each question (example below) and is also calculated as the Correct Student Ratio (column J).

Difficulty Index.png

8 - Alpha Score: Measures the entire quiz's internal consistency.

Alpha Score

Also considered reliability, the alpha score assesses if multiple questions are measuring the same information in which the student will answer in a similar way. The higher the alpha score, the more reliable the overall quiz. A good alpha score is anything above 0.7.

9 - Point Biserial: Measures how well a single question's possible answers are answered based on overall performance. The differences (discriminates) between students who did well and who did not. Only available for true/false and multiple choice questions.

Point Biserial (Discrimination Index)

Column S - Point Biserial Correct:The calculation is a correlation between the top / middle / bottom performing students who answered the question correctly. The top / middle / bottom performing students are automatically sorted based on their total quiz score: the top 27%, the middle 46%, and the bottom 27%.

Anything above +0.24 is considered to have a good discrimination index. A good discrimination index is a question that the top / middle performing students answered correctly while the low performing students did not answer correctly. A discrimination index of 0 is considered neutral because all students answered the question correctly. If your discrimination index is red (less than +0.24), consider revising the question and / or possible answers.

The image below is a representation of how the discrimination is calculated based on students either answering correctly or incorrectly.

Discrimination Index Represenation-1.png