The strengths and weaknesses of critiques as an assessment tool.
As design educators, we might have a slight love/hate relationships with critiques. They are a great way to assess learning and an opportunity to use design vocabulary in a natural way. On he other hand, they can be exasperating for both teacher and student and some days you just feel like there is no point. Students come unprepared or they say very little. It feels like they say a few words and then “check out” after.
Blythman, Orr and Bernadette found that there is an even bigger frustration for students who feel they are not getting clear direction out of critiques. They can often feel personally attacked or that the critique is a one-sided affair where you tell them what to do, rather than an exchange of ideas. There is also greater diversity in today’s classrooms where international or second language students have both a difficult time expressing ideas in a critique, but also understanding the information being covered though the fluid exchange. We often get stuck in a mode with critiques where everyone puts up the work and we lead the critique, often speaking more than the students. Often it is less organized and time runs out before we can get through everyone’s work or we rush through the last few at the end. It is important to remember how students can perceive this as a slight to their work or that lack of conversation around their work means it is negative.
Overall, the benefits of critiques far out way the negatives. Three key points from Blythman, Orr and Bernadette that I feel are the most beneficial in the classroom are: 1) The students get to see each others work at multiple stages and can learn from the discussions about not only their own work but the evaluation of other students’ work. 2) Critiques connect the learning, teaching and assessment in a tangible way. It is a time to see if you are on track with the class and effectively communicating expectations and using visual design language. 3) Students have a chance to see that there can be a variety of perspectives on design, showing that there are multiple way to arrive at a solution to a design problem. This is why it is best if you can team up with other educators when critiquing so they can see professional discord.
Throughout the research, they used interviews of students and staff as well as persona based case studies to work towards creating a roadmap for better, more effective critiques. Here are some key points from the research to help you improve engagement, outcomes and learning in your critiques:
USE A VARIETY OF CRIT METHODS
Not only does mixing things up bring variety to critiques, but you can employee different techniques for different student experience levels. If you have students that are relatively new to critiques, start with small group peer critiques where they may feel more comfortable talking about their work. This is especially useful in early stages of the design process. As students become more experienced in the later years, invite them to facilitate the critique, allowing them to improve their own presentation, time management, visual language and evaluation skills. Bring in industry professionals or cross-functional teams to participate, allowing for greater exchange of ideas.
RECORD THE LEARNING IN SOME WAY
Students often walk away from critiques without a written record of their feedback. Research has shown that they often remember the negative over the positive and forget constructive feedback shortly after the critique. Pair students up and have them take notes for each other. This way they can stay actively engaged in presenting their own work while someone else records the feedback. Another technique is to use post-it notes and have students spend some time reflecting on each others work before the verbal critique.
PAY ATTENTION TO THE ENVIRONMENT
Make sure that your space helps set everyone up for a successful critique. Gather students around the work, making sure everyone can see. Avoid the disruption by having to move around the space too much. What are the distractions to engagement? It seems like common sense, but reinforce laptops and phones being put away.
USE VISUAL LANGUAGE
Everyone should avoid the trap of the phrases “I like that”, “that just doesn’t work” or other phrases that become more about personal judgement and ego over constructive feedback. If something isn’t working, explain using visual language that your students understand. Have a glossary of terms for your crits and review the meanings with students. Use lead in questions that support the language.
MAINTAIN STUDENT ENGAGEMENT
Blythman, Orr and Bernadette point out that it is important to respond to work thematically or in relationship to other work. This helps students stay engaged because they aren’t just zoning off while they are waiting for their work to be discussed. Also make sure students understand the purpose and expectations of the critique before it begins and make participation part of their overall assessment. Lastly, employ spontaneity and variable strategies to keep things interesting. One example might be having students pull a design term, out of a hat and have them speak to how their or another students work supports the design principle or idea.
Critiquing the crit
BLYTHMAN, MARGO; ORR, SUSAN; BLAIR BERNADETTE, 2007
​
Setting expectations with clear rubrics that guide student success.
Developing assessment rubric in graphic design studio-based learning
MERVAT MEDHAT ET AL, 2011
Grading student studio work without an assessment tool such as a rubric can create an ambiguous process that has been proven to be inconsistent, lack-transparency and cause confusion for students in assessing their learning. Rubrics can be used not only to help you set up your own consistent standards but it can also help students understand what you are expecting as outcomes for a given project. This reduces fully subjective assessment that students can often feel as unfair, and the time it takes to assess studio work, getting feedback to students faster.
Research has shown that student engage with an assignment or subject based on how they are going to be evaluated and what the expectations are. We all know, students put a lot of emphasis on their assessment or grade because they see it as a direct relation to their learning. They want to know what you expect and how those expectations will equal a good grade outcome for them. That being said, the traditional method of critique and summative assessment only of the final outcome, with undefined criteria can actual result in less learning. Therefore, the key advantages of rubrics according to Mervat Medhta et al., are:
-
It allows assessment to be objective and consistent
-
It allows the instructor to be able to define what they are using for criteria
in assessment -
The student gets a clear view of how they will be evaluated
-
The instructor gets useful feedback on how effective their instruction was
-
They can provide benchmarks for the process as well as the final outcome
Starting rubrics can feel like a daunting task, and it is important to evaluate your rubrics, cross-referencing with required course learnings and outcomes after each time you use them in order for them to be most effective. There are two types of rubrics you can use depending on what the learning is and where in the process you are evaluating the students’ work. Analytic rubrics are formative and evaluate each criterion separately, this way you can benchmark process along the way. They usually have less impact on a formal grade and are more for giving feedback for improvement. Holistic rubrics are summative and evaluate the criterion all together. This is best used in assessing the final outcome of a project or section.
In studio-based learning, the criteria set for a rubric can include the process, product, participation, critical-thinking skills and in the case of early learners some basic tasks like following direction clearly. The key is to follow three main steps in building your rubric grid:
-
Develop your criteria for the rubric. Make sure they are tied back to course learnings and outcomes.
-
Settings levels for performance in relationship to your criteria. You might start with something like mastered, competent or needs improvement if you are giving feedback throughout the process. If you are doing a summative assessment rubric, you may have actual grade or point levels assigned to each performance level.
-
Create descriptions of what parts of the criteria are being met at each performance level that can be understood by students and show improvement with each level.
The above steps are built into a grid that allows for easy assessment and the accumulation of a score, whether that be a grade or points. Here is an example of one row in a rubric
​
Keep in mind that when developing criteria that any more than 7 will be overwhelming for you or your students to keep track of, and any less than three are not specific enough to give solid feedback and guide students’ learning. These expectations should also be clearly tied back to the goals and objectives of the course, so evaluate each one in that way. Once completed, a good rubric will make it easier for you to grade your students performance in multiple areas, but it is also essential that you give verbal feedback along with their rubric score in order to guide learning further.
At the start of a lesson or project where you are instituting a rubric, it is important to review it with your students to make sure they understand the expectations. Once your rubrics are completed you can us them to evaluate your own instruction by comparing common missed criteria. This is one of the true benefits of using rubrics. The criteria give you a step-by-step path to assessing performance.
Assessing the process, product and person for a balanced approach
A model for holistic studio assessment in the creative disciplines
DE LA HARPE, BARBARA; PETERSON, FIONA, 2011
​
When deciding on what the focus should be for grading your students’ work, the process of creating the final piece or artifact should be part of your criteria. Too often we focus only on the final outcome, but so much of what happens a long the way, is large part of student learning. It helps to build critical design thinking skills that are important to future success and allows you to check for students understanding.
Historically, the studio model of assessment follows the master/apprentice model that is teacher centered and focused on the final artifact. This approach causes students to work passively and wait for your approval before moving forward. There is little exchange of open ideas, a lack of discord that is necessary in active learning and a creation of robot students that only strive to do what you think to get a good grade. This creates a dependence that extends beyond the classroom and into the early years of their professional career.
Using a holistic approach the includes the outcomes of product, process and person, supports student learning and well-being while creating an assessment that is fair and balanced, as well as engaging the teacher and the student. It presents a less threatening and more authentic real-world view of the design process and includes opportunities for more peer and student self-assessment. In doing so you create a more well-rounded graduate that can develop strong work, have the critical thinking skills to engage in the design process and the ability to reflect on their experiences and develop as a person.
The research and analysis of 118 journal articles on studio assessment allowed de la Harpe and Peterson’s to build a holistic model of assessment that focus on three areas: The outcome, which includes the product, process and person in the assessment and not just the final artifact. The knowledge and skills which including things like technology, hard and soft skills and content learning. And lastly reflective and professional practices which looks at the students ability to think like a graphic designer and actively reflect on their practice. Within these areas you can evaluate range of criteria and key indicators that encompass multiple factors beyond just the final artifact.
The key to this like any assessment model you use in your classroom is that it has flexibility where not all criteria are required for every assessment. As a matter of fact, over assessment can lead to more rote responses rather than active learning. Sometimes the focus is on the product and sometimes the focus is on the process etc. The broad view of your assessment in your classes should include all three, the process, the product and the person in order to create assessment that is engaging and more rewarding for both you and your students.