Problems with "Process Over Product" (Part 1)

The top 4 mistakes I see teachers make when teaching process.

First of all, I want to express how thankful I am to my LinkedIn community. Over the past few months, I’ve been writing a book on teaching process in The Age of AI. As I write and share my ideas there, my community has really engaged with me. They’ve pushed against my assumptions, and show me (again and again) just how complex questions about AI and education are.

Secondly, for this newletter issue, I wanted to do something a little different. In addition to writing down my thoughts here, I want to share my ideas in the next installment of my EdUp AI podcast.

So, if you’d like to listen to me talk about this content, check out the video podcast (videocast?) at the very bottom.

All right, so let’s get into it…

What I’m Thinking About These Days

Ever since the release of ChatGPT (almost 3 years ago), “process over product” has become a rallying cry for many teachers and education reformers.

The idea: if a student could approximate a polished product (typically an essay) with a Gen-AI program, then educators need to shift to teaching process. They need to design learning experiences that focus more on the actions students take, and less on the finished product they hand in at the end of the day.

I see the cry for “process over product” as a compensatory pendulum swing. Educators are pushing in the other direction, because for a long time the education system has priviledged the product over the process. Eventually, I think we’ll find a more balanced way to incorporate process, product, and progress in our assessments — so that these all feed into one another.

I fear that the “process over product” adage has had a side effect.

It suggests that teaching process is a silver bullet, that moving from a product-oriented mindset to a process-oriented mindset is the transformation we need to finally adapt to The Age of AI.

In my experience, that is far from true. Teaching process brings up almost as many problems as it addresses.

I want to share the 4 most common mistakes I see, when teachers try to focus on process over product.

In doing so, my aim is to (1) help anyone who thinks this shift may work for their own courses and (2) push the conversation about process and product in a more nuanced direction.

These are the top 4 mistakes I see educators make:

1. Pitfall #1: Over-burdening students with products.

2. Pitfall #2: Over-scrutinizing creativity

3. Pitfall #3: Dichotomizing Product and Process

4. Pitfall #4: Flattening Out Process

Originally, my goal was to cover all 4 of these in a single newsletter. But as I wrote, I realized that the newsletter was getting far too long.

So, I’m going to do a series.

For this one, I’m going to focus on the first pitfall: over-burdening students with products.

Pitfall #1: Over-Burdening Students With Products

Imagine you’re a professor.

Currently, your assignment is something along the lines of “Write a 5-page essay in which you announce your argument about cell phone bans in U.S. schools. Support that argument with at least 5 reliable sources, which are cited at the end in MLA format.”

For a while, this works. But as more and more of your students are using LLMs to complete their work, you decide to revamp the assignment.

Instead of grading a single product at the end, you ask for proof of the student’s process along the way. In addition to the 5-page final product, you ask students to send in:

1. An outline at the very beginning, that captures the student’s vision for the paper.

2. A rough draft.

3. A mid-way reflection on how the paper is going so far.

4. A list of the changes they made in between the rough draft and the final version.

5. A final reflection.

6. An AI Disclosure Statement.

I share this list to demonstrate a harsh truth.

In many ways, the “process over product” adage is a lie. There is no way to get unmitigated, direct access to a student’s process. We’ll always rely on products as proxies of the processes that went into them.

Very quickly, process-oriented pedagogy balloons everything, so that students are simply sending in 6 products when they used to send in 1 or 2.

This was the gist of Owen Matson’s critique of process-based assessment, when he posted this on LinkedIn.

If you want to see the entire post (along with comments and responses), click here. I highly recommend it.

Process-oriented teaching can:

1. Shift the site of surveillance — so that we’re trying to detect AI misuse and over-use by surveilling the process. This is especially important when we think about third-party transparency programs, which often track students’ every use to see what they are copying and pasting, when they may be using GenAI programs, and so on.

2. Flatten out process — so that educators can evaluate the steps in that process, and ask students to comply with certain standards and expecatations. This goes against the organic, often non-linear process of creativity.

One of the dangers is that process-based pedagogy feels progressive, while it doubles down on the product-focused mindset.

For me, this isn’t an argument against teaching process. It’s an indictment against teaching-process-gone-wrong. So, for anyone trying to avoid these landmines while teaching process, I often ask them these questions:

1. Are you allowing students to create their own processes, and pivot as things come up? Does your process-focused assignment allow for the emergence that comes organically from creativity?

2. Are the steps in the process linked to a larger learning goal, or are they simply there to cut down on the misuse or over-use of AI? If some steps are just there to catch students without any other improvement, I often recommend deleting them.

3. What are the trade-offs of your approach, in terms of surveillance and keeping tabs on students’ work? Is the trade-off worth it? I come back to this question myself, when designing my own learning experiences. It’s easy to make process-based teaching a form of invasive surveillance. (This is one of the reasons why I’ve opposed third-party transparency programs, especially if they record the student as they work.)

In my classes, I take these issues into account.

I ask students to:

1. Design their own project/product.

2. Design their own process. (I call these SEWPS, or Self-Empowering Writing Processes). This is just a guess. Students can change them as they go along.

3. Create a rubric they will use to evaluate their own success on that product.

4. Create a first draft.

5. Stitch together different sets of feedback — from their own self-assessment, a fellow student, an AI program (I use the PAIRR system, designed by Anna Mills and her team at UCSD). Students can pick and choose the feedback they want to incorporate.

6. Share the actual process they ended up using to create their product (these can be very different than the ones they set out in Step 2).

7. Self-Assess their own work, using their own self-designed rubric.

8. Submit a Transparency Statement, in which they share if/how they used AI and whether that use or non-use empowered them.

I wrote about using Transparency Statements to boost metacognition for Faculty Focus, a little while ago. And if you want to see the basic format I use for these statements, you can click here.

There are some important differences between this list of steps and the list earlier.

For one, these steps are all linked to specific learning objectives. For each step, we talk about exactly what we’re practicing and why.

Secondly, they are ungraded. As long as students complete them and hit the learning objective, they get a Complete. If they miss the learning objective, they get an Incomplete and can keep trying.

Thirdly, they account for emergence. The second step, which asks students to set up a process, is provisional. They are welcome to depart from it completely, depending on where their creative process takes them.

Fourthly, I go out of my way to avoid policing. Even the Transparency Statement is designed to encourage self-awareness and to start conversations with students about how things went.

****

Now, no matter how we design things, we’ll fall victim to the lie behind “process over product.” Unless I’m willing to have students record themselves at the moment they create (which I’m not), I’ll always be left with a set of products.

But…

I do think we can carefully design those products, so that we get glimpses of our students’ creative processes and can start to piece together the narrative of their progress from small product to small product.

That’s it for my take on Pitfall #1.

For the next newsletter, I’m going to focus on the second pitfall for process-based assessment.

Over-scrutinizing creativity…

Below, as promised, I’m including the EdUp AI podcast episode, about this very topic.