Grading & Assessment

Overview

Although discussions take place in the face-to-face learning environment, it can sometimes be challenging to introduce a variety of assessment methods into online teaching and learning spaces. Online discussions are a great way to engage learners, build student community, and promote learning.

Beyond a tool for growing social presence and creating community within a course, online discussions can: measure active participation; gauge comprehension; encourage analysis; focus on evaluation; encourage reflection; and/or prompt synthesis.

How Discussions Work

MyCanvas includes an integrated system for class discussions, which allows educators and learners to start and contribute to discussion topics. Discussions can be created as a graded assignment because of the integration with the MyCanvas gradebook or can simply be used for class participation. Discussions can be organized as focused or threaded discussions:

Focused discussions allow for 2 levels of nesting, the original post and subsequent replies.

Threaded discussions allow for infinite levels of nesting.

Uses of Discussions

Focused Discussions

Threaded Discussions

Questions for Discussion posts can be created by the educator, or the learners can create discussion questions to be answered by their peers.

Functionality within Mohawk's MyCanvas LMS facilitates your creativity when crafting discussions. Options and restrictions allow you to:

From the students' perspective, Discussions in MyCanvas offer robust editing features - students can see the same WYSIWYG editor that they are used to using in Word and can enhance their contributions with images, hyperlinks, embedded videos, etc.

In-Person/Synchronous Discussions

Discussions can also take place in in-person or synchronous online classes. These discussions can be completed in small groups or breakout rooms, where individuals from each group are selected to report the discussion findings of their smaller group. The debriefing method is important to ensure that students are summarizing the most important aspects of their discussion. Some debriefing techniques include: verbally, newsprint/flipchart, whiteboard, screen sharing, etc. Large group discussions can also be used with some classes, but the educator should be mindful of ensuring that the group stays on task.

Assessing Discussions

Learners may have distinct ideas of what a quality discussion is, so it is important to clearly outline course expectations. You can attach a score to an individual thread, multiple contributions by a learner or associate a rubric with the topic. A discussion rubric can be used to ensure that all learners are on the same page and build learner accountability. Another method of assessing discussion participation is having the learner keep track of what they have learned in the class discussions, both in person and virtually, and submit a reflection on their learning. Either score can link directly to Grades.

Assessing in-person discussions can be challenging, especially with larger groups. In this case, assessment can be based on participation, contribution, peer grading, etc. Another way to assess participation in class discussions is to have learners submit a reflection on the discussion topic and the ideas presented in class.

Use at Mohawk

Discussions are a widely used assessment method at Mohawk College. Many educators use them in all modalities of learning; Hybrid, Online Asynchronous, Online Synchronous, in-person and HyFlex. When used appropriately discussions can be an effective assessment tool. Consider adding variety in your discussions, especially those posted in the LMS. For example:

Consistent use of the tool is wonderful for learner experience of online learning, but varying how the tool is used is equally important for engagement and ensuring that the task does not become monotonous.

Additional Resources

Brank, E., & Wylie, L. (2013). Let's discuss: Teaching students about discussions. Journal Of The Scholarship Of Teaching And Learning, 13(3), 23-32

Not specifically detailing online discussion, but an interesting article exploring how to approach the importance of discussions with students. Connects the engagement with discussions to deeper success in the course. The list of references are also a sound springboard to more information.

Lai, K. (2012). Assessing participation skills: online discussions with peers. Assessment & Evaluation In Higher Education, 37(8), 933-947. doi:10.1080/02602938.2011.590878

A comprehensive analysis of a specific example for an online discussion forum. The highlight on students' critical thinking skills and how to assess and evaluate them will be of particular interest. The author provides exemplars of the different levels of performance as well as various marking guidelines from checklist to rubric.

Defining Authentic Assessment

Authentic Assessment is important in higher education because it can help prepare students with the skills and tasks they will need in the workplace. As Authentic Assessments involved real-world tasks that are relevant to students.

Traditional Assessment vs Authentic Assessment

What is the difference between traditional assessments and authentic assessment?

"Authentic assessment drives the curriculum" (Meuller, 2018) versus traditional assessment where the curriculum drives the curriculum. In Traditional Assessment the assessment is completed to ensure learners have gained the required skills and knowledge. In Authentic Assessment the goal is to ensure that when learners graduate, they are able to perform in the real world, therefore Authentic Assessment tasks learners with real-world examples. In essence, assessments are created to ensure students can demonstrate mastery, then the curriculum is created to support the assessment. This is also called backwards design.

Authentic assessments are direct measures. We want students to be able to use their acquired knowledge and skills in the real world, after they graduate. Authentic Assessment can capture constructive nature of learning, by having them demonstrate capability instead of just answering questions about capability. The same task used to evaluate learning, is also used as a vehicle for the learning. Authentic assessment allows for multiple paths to the same result, other words students will demonstrate their learning of a concept in different ways.

Note: Authentic Assessment complements traditional assessment, a combination of both can be used to ensure students have achieved the learning outcomes of a course.

Benefits

Four Steps to Creating an Authentic Assessment

"Authentic tasks do not have to be large, complex projects. Most mental behaviors are small, brief "tasks" such as deciding between two choices, or interpreting a political cartoon, or finding a relationship between two or more concepts. Thus, many authentic tasks we give our students can and should be small and brief, whether they are for practicing some skill or assessing students on it. (Mueller, 2018)"

Example

Learning Objective: Write original code to demonstrate the use of a programming concept.

Task: Make a captioned video that uses code you've written to explain a programming concept with a real-world application. Post the video link in a MyCanvas discussion. Review and provide feedback on two peers' videos.

What the researchers say

Additional Resources

UDL and Assessment

References

Mueller, J. (2018). What is authentic assessment? (Authentic assessment toolbox). https://jonfmueller.com/toolbox/

Online Network of Educators. (2020, June 17). Authentic assessment guide. https://onlinenetworkofeducators.org/pocket-pd-guides/authentic-assessm…

Stiggins, R. J. (1987). The design and development of performance assessments. Educational Measurement: Issues and Practice, 6, 33-42.

Wiggins, G. P. (1993). Assessing student performance. San Francisco: Jossey-Bass Publishers.

When designing and creating quality multiple choice and true/false questions, educators need to be purposeful in all aspects of questions and answer design. Please use the guidelines below to assist in designing quality multiple choice and true/false questions:

1. Use clear and concise language to avoid any ambiguity or confusion. Make sure the questions are easily understandable and do not contain unnecessary jargon or complex sentence structures.
2. Ensure that the questions are directly related to the course learning materials and cover the key concepts. Avoid including irrelevant or trivial information that could distract or mislead the students.
3. When possible, frame the questions within authentic scenarios or contexts that reflect real-world situations. This helps students see the practical applications of the concepts they are learning and enhances engagement.
4. The questions should align closely with the course learning outcomes. Each question should assess specific knowledge or skills that students are expected to acquire within the modules.
5. For multiple choice questions, provide conceivable distractors that closely resemble the correct answer. This ensures that students must understand the content fully to select the correct option, making the question more challenging.
6. Randomize the order of the answer choices to avoid any unintentional patterns or biases. This ensures that students cannot rely solely on the position of the correct answer and encourages them to carefully evaluate each option. It is strongly recommended, when possible, to randomize question order as well.
7. Clearly state the instructions at the beginning of the assessment to guide students/learners on how to respond to the multiple choice and/or true/false questions. Specify whether the question is multiple choice (single correct answer) or multiple answer (multiple correct options) for multiple choice questions.
8. Include questions that assess different cognitive levels, such as recall, understanding, application, analysis, and synthesis. This allows students to demonstrate a deeper understanding of the content and promotes critical thinking skills.
9. Apply universal design principles by considering the needs of diverse learners. Ensure that the questions are accessible to students with disabilities and provide alternative formats if necessary, such as providing a text-to-speech option for visually impaired students.
10. If the question is a multiple answer question, ensure to clearly specify all learners should select all answers that apply to the question. For example, add "select all the apply" to the question.

Example Questions taken from Yale University Centre for Teaching & Learning.

In the following examples of effective and ineffective MC questions, students explore potential energy, or the energy that is stored by an object.

#1. Good Stem, Poor Distractors

Potential energy is:
a) the energy of motion of an object.
b) not the energy stored by an object.
c) the energy stored by an object.
d) not the energy of motion of an object.

In this question the good stem is clear, brief, and presents the central idea of the question through positive construction. However, the distractors are confusing: b) and d) are written in negative constructions that force students to reinterpret the stem, while c) and d) have overlapping, inconsistent content that confuses and tests reading comprehension over content recall. Finally, choices do not move logically by grouping content, failing to visualize and test larger concepts for students.

#2. Poor Stem, Good Distractors

Potential energy is not the energy:
a) of motion of a particular object.
b) stored by a particular object.
c) relative to the position of another object.
d) capable of being converted to kinetic energy.

In this question the poor stem contains the word "not," which fails to identify what potential energy is, and tests grammar over student understanding. However, the good distractors are written clearly, cover unique content, and follow a logical and consistent grammatical pattern.

#3. Good Stem, Good Distractors

Potential energy is:
a) the energy of motion of an object.
b) the energy stored by an object.
c) the energy emitted by an object.

In this example both the stem and the distractors are written well, remain consistent, and test a clear idea.

Additional Resources

The University of Waterloo provides additional information on designing Multiple Choice questions.

References

Yale Center for Teaching and Learning. (n.d.). Designing quality multiple choice questions. Retrieved from: https://ctl.yale.edu/MultipleChoiceQuestions

In 1933, American psychologist and educational reformer John Dewey noted that we do not learn from experience. We learn from reflecting on experience.

Having learners participate in reflection activities solidifies the connection between the experience and what they learned. If a student does not reflect on an experience, it is difficult to assess what they have learned. It helps develop critical thinking skills by allowing learners to examine their strengths and explore areas for improvement in their future learning.

Reflection is a key component of experiential learning. According to Ontario's Ministry of Colleges and Universities' Guiding Principles for Experiential Learning, all Experiential Learning must include student self-assessment - in other words, a graded reflection.

Adding reflection to student assessment

Reflective learning can either be added to existing assessments or be stand-alone assignments. It can include larger assignments such as reflective journals, portfolios, goal setting, and more.

Reflection can be done using a reflective framework, such as the Reflective Cycle (Gibbs, 1988) that has six stages:

1. Description: What happened?
2. Feelings: What were you thinking and feeling?
3. Evaluation: What was good and bad about the experience?
4. Analysis: What sense can you make of the situation?
5. Conclusion: What else could you have done?
6. Action Plan: What would you do next time?

Adapted from University of Waterloo.

Group work can be an effective method to motivate students, encourage active learning, and develop key critical-thinking, communication, and decision-making skills. But without careful planning and facilitation, group work can frustrate students and instructors, and feel like a waste of time. Use these suggestions to help implement group work successfully in your classroom.

Preparing for Group Work

Designing the Group Activity

Introducing the Group Activity

Monitoring the Group Task

Ending the Group Task

Additional Resources

Teaching Tips

Other Resources

References

Brookfield, S.D., & Preskill, S. (1999). Discussion as a Way of Teaching: Tools and Techniques for Democratic Classrooms. San Francisco: Jossey-Bass Publishers.

Gross Davis, B. (1993). Tools for Teaching. San Francisco: Jossey-Bass Publishers.

Jaques, D. (2000). Learning in Groups: A Handbook for Improving Group Work, 3rd ed. London: Kogan Page.

Johnson, D. W., Johnson, R. T., & Smith, K. A. (2014). Cooperative learning: Improving university instruction by basing practice on validated theory. Journal on Excellence in College Teaching, 25(3&4), 85-118.

Race, P. (2000). 500 Tips on Group Learning. London: Kogan Page.

Roberson, B., & Franchini, B. (2014). Effective task design for the TBL classroom. Journal on Excellence in College Teaching, 25(3&4), 275-302.

Defining Rubrics

A rubric is an assessment tool that explains marking criteria for course assessments. They divide assessments "into component parts" and describe the expected level of performance for each part (Stevens & Levi, 2013). Rubrics can be used for a variety of assessment types, such as essays, projects, presentations, discussions, and reflections.

One can define the term rubric as:

Relevant User-friendly Benchmarks Reinforcing Instruction and Cultivating Success

This concise definition was crafted as an acronym but it provides a concrete description of the purpose of a rubric.

A rubric is a table that provides performance criterion (left hand column) and levels of achievement (top row) to better communicate assignment expectations with learners for assessment and evaluation.

"At its most basic, a rubric is a scoring tool that lays out the specific expectations for an assignment" (Stevens & Levi, 2013, p 3).

How Rubrics Work

A rubric should accompany each assessment in a course. Rubrics can take on many forms; however, usually there are four levels of achievement (4, 3, 2, 1) that correspond to four grades A, B, C and D starting from the highest to the lowest level, reading left to right in the table. In some cases, there may only be 3 levels of achievement.

Criterion outline what part of the assessment the levels of achievement are referring to. For example, you may want to focus on Grammar and Spelling in one criterion and content development in another.

For each criterion, the educator crafts a description for each level of achievement, from acceptable to unacceptable (or level 4 to level 1). For example, each cell or box of the matrix has a unique entry that explains what is expected in order to achieve the specified level (i.e. level 4, 3, 2, or 1).

The educator uses the rubric to assess and evaluate learner work. Rather than having to write lengthy feedback/comments, the rubric speaks to areas of strength and weakness. A learner can easily identify assessment expectations in order to be successful. Overall, learners receive a more comprehensive assessment of their contribution/learning and recognize the alignment of the assessment to important outcomes, skills, and abilities.

Benefits

Faculty Benefits:

Student benefits:

Common Missteps

Educators tend to over-conceptualize their descriptions, causing difficulty for the learner to connect their levels of performance to learning outcomes and identify areas for improvement. Among the different levels of achievement, the following qualifiers are used in order to ensure that the outcome is clear:

Use of the terms above makes progression along the continuum easier to discern for both educators and learners.

Use at Mohawk

Rubric use is encouraged at Mohawk College and the Centre for Teaching & Learning Innovation is available to assist with the creation of your rubrics. A Curriculum and Program Quality Consolation (CPQC) can assist with the creation/wording of rubrics.

In addition, MyCanvas has a built-in Rubrics tool that allows educators to build digital rubrics directly into the Learning Management System. Any rubrics created in MyCanvas can be associated with assessments making online evaluation and feedback seamless. An Instructional Designer (ID) can support faculty with implementing rubrics in the LMS.

Additional Resources

Association of American Colleges & Universities VALUE Rubrics
Introduction to Rubrics - Stevens and Levi
Types of Rubrics - Queen's University
Online Discussion Rubric
 

References

Queens University. (n.d.). Rubrics and marking schemes. Queen's University Canada. https://www.queensu.ca/teachingandlearning/modules/assessments/34_s4_04…

Stevens, Levi, A., & Walvoord, B. E. (2013). Introduction to rubrics: an assessment tool to save grading time, convey effective feedback, and promote student learning (Second edition.). Stylus.

Western University. (n.d.). Grading with rubrics. Centre for Teaching and Learning - Western University. https://teaching.uwo.ca/teaching/assessing/grading-rubrics.html