Khan Academy

In-Class

Review Game

An engineer, user researcher, product manager, and I had the opportunity to research the classroom space and prototype how Khan Academy might fit into a teacher's workflow! This project spanned around 5 months from user research to concept testing.

Research

Going into the user research study, our hypothesis was that teachers want a homework replacement. We spoke to 13 mainstream math teachers in the US about their day-to-day challenges. We sought out teachers who didn't routinely use technology in their classrooms.

We learned a lot! At a high level, we discovered that teachers didn't find assigning homework challenging, but that keeping student engaged and knowing where they are is tough!

We also created some design principles to guide our thinking for the rest of the project:

Design Principles

Brainstorming

Brainstorming
We nicknamed this the Foamboard Fort! These are all the ideas the team generated from the HMW questions.

We created How Might We (HMW) questions and invited the rest of the company to generate ideas with us! Some of these questions included:

At the same time, we looked at existing classroom products that teachers loved! We realized that in-class review games like Kahoot!, QuizletLive, and Socrative had high rates of growth.

Competitors
From left to right: Kahoot!, QuizletLive, Socrative

Through ideation & competitive analysis, we narrowed our question to: How might we create an engaging in-class math review game? For the following couple months, we used this question to guide our prototoype.

Prototyping & Playtesting

Each week, we created a prototoype based upon a hypothesis and brought it into a classroom. The teacher would guide the class in the review game and we would debrief with them afterwards!

Here are some illustrations that show the general flow of the game:

Flow
Mrs. Frizzle checks wants to spend class time reviewing for the unit test tomorrow!
Flow
She projects the review question on the board and students use a device to answer the question.
Flow
She notices that the class is mostly done with the question and decides to see how they did.
Flow
Since there were two very common answers, she asks two students to explain how they arrived at the solution.

If you're interested, we also drew out a detailed core game loop diagram! Before I get into each iteration, I want to mention that I can't show any of the videos of the in-class testing. Because of this, I only have low quality GIFs of each prototype.

Week 1: Do teachers even want this?

We created the prototype in Framer! The desktop view is what the teacher projects on the screen. The mobile screens are the student view.

Prototype
We wanted to make starting a game easy, so students just had to join with a class code! We allowed students to pick an avatar to engage them.
Prototype
When a student answers, they have to select how confident they are before they see the result. The teacher can monitor by looking at their avatar on the projected view.

Teachers were really excited about the idea, but they found it difficult to immediately engage with the students because the confidence and check were so small!

Week 2: How do we allow teachers to engage immediately?

We adjusted the layout this week, so that it was more easily scannable! We wanted to surface struggling students immediately to teachers.

Prototype
When a student answers correctly, a large green check appears next to their name.

Teachers loved how they could engage immediately, however, there was no way they could revisit student responses after the game.

Week 3: What does a post-game report look like?

This week, we focused on what happens after the game. We generated a PDF report that showcased student attempts and confidence! We also wanted to surface this information to the student, so that they could know what to review for the test.

Prototype
This is the teacher report!
Prototype
This is the student report!

Teachers loved how they could dive deeply into student attempts and responses. It helped them identify where certain students had gaps in learning!

Week 4: What would a game with points look like?

This week, we decided to focus on making the game more engaging. We wanted to test if a more "competitive" would be more engaging for students! When a student joined, we would put them on a random team. For every correct answer, a student adds up to 10 points to their team's score. If a student attempts incorrectly, they still get 1 point for effort!

Because we had a little more time, we also spent some time doing a visual refresh of the earlier prototypes! We wanted to bring less focus to the confidence ratings of the indvidiual students, so that students were more comfortable inputing how they actually felt.

Prototype
Team Green is on the left and Team Blue is on the right!
Prototype
Post-question screen. The teacher can quickly assess whether or not they should go over the question.
Prototype
End of game screen!

When we tested this in the classroom, the teacher would tell students to try again if they only got 1 point. It was so rewarding watching the class play because the students would celebrate when they answered a question correctly!

This was our last prototype! If you're interested in playing, you can play the Demo at www.khanacademy.org/r/classroom-teacher.

Exploratory Designs

Throughout the project, I also worked on exploratory designs for how the review game might flex in the future! In the iteration below, each team is trying to grow a plant. Instead of selecting an emoji, the student can decide if they're feeling rainy or feeling sunny, both of which will grow the plant!

Prototype
Prototype
Prototype
Prototype

Since Khan Academy has a partnership with Pixar, we thought it might be fun to have themed games! In the example below, a teacher could set a class "balloon" goal. For each correct answer, the student gets to add a balloon to the Up house. If the class reaches the goal, the Up house flies away at the end of class.

Prototype

In the following exploration, the goal is for students to defeat the monster! Similarly to the Up example, a teacher could set the HP (number of correct questions) beforehand.

Prototype

Teachers love to have students reason through their answers! In the following exploration, we would surface two popular answers and select two students to "debate" the solution. Other students can send hearts to the two debaters based upon who they think is correct!

Prototype
Prototype

Next steps

Although we received lots of teacher validation during the project, the leadership team decided to hold off on pursuing this idea until we had a more solid foundation for the teacher product!