Why we still can’t stop plagiarism in undergraduate computer science

Imagine that you’re hired to work at your local public library. As an eagle-eyed checkout clerk, you soon realize that half the patrons leave without actually checking out their books! This leaves everyone else scratching their heads when the catalog doesn’t match the shelves. But conveniently, the library has an unused anti-theft alarm sitting in the back room.

There’s just one problem: your supervisor, though sympathetic to your cause, doesn’t want you using the alarm. You see, people take books for many reasons, and not all of them are malicious. Besides, who cares if the shelves are in disarray? They’ve been like that for years, but everyone who works at the library still gets their paychecks the all the same. If you want to set up the alarm, you’ll have to do so on your own time, in addition to your other responsibilities.

In many undergraduate computer science programs, this is the absurd reality we face when trying to combat plagiarism. Everyone agrees plagiarism is wrong. Everyone wishes they could stop it. Everyone has access to the tools that find it. But no one seems willing to take any action.

Why bother?

The most important goal is to keep the course fair for students who do honest work. Instructors must assign grades that accurately reflect performance. A student who grapples with a problem — becoming a stronger programmer in the process — should never receive a lower grade than one who copies and pastes.

Finally, as educators, we also hope that the accused student can learn difficult lessons about ethical behavior in the classroom rather than the workplace.

Understanding the scope of plagiarism

I’ve been a teaching assistant in a lower-division computer science course for the past two years. Each semester, in a class of 200 to 300 students, we typically discover 20 to 40 blatant cases of plagiarism on homework. And because of the nature of our process, even more cases go undetected. Here’s how it works:

We begin by uploading our students’ code to an online plagiarism detection tool. The tool compares the submissions with each other and with our massive back catalog of previous solutions, flagging pairs of similar programs. We’ll manually review nearly 1,000 of these pairs over the course of the semester, throwing out all but the most suspicious cases. These cases end up representing about 100 students.

Then, we apply another filter, keeping only the cases that contain indisputable evidence — for example, hundreds of lines copied right down to the last whitespace error. We have virtually eliminated false positives at this point. (In the course’s entire history, only one such example exists.)

That means we aren’t catching cases where there’s plausible deniability. And we certainly can’t find students who transform someone else’s homework beyond recognition. (They’ve demonstrated a better grasp of programming than your average copy-paste-rename job, but still haven’t learned what the homework was trying to teach.)

Because our process heavily favors precision over recall, the 20 to 40 cases at the end represent a lower bound on plagiarism in the course. This is corroborated by the results from Lisa Yan et al, the authors of a new plagiarism tool named TMOSS. In a Stanford course, 43 percent of true positives detected by TMOSS wouldn’t have been discovered by a traditional tool similar to ours.¹

The large number of students suspected of plagiarism isn’t unique to our course. In fact, if a university or course doesn’t have comparable numbers, it most likely reflects their (lack of) plagiarism detection, rather than the actual rate of plagiarism.

According to a 2017 New York Times report,² a course at UC Berkeley found about 1 in 7 students in violation of policies on copying code. At Stanford, one course suspected up to 20 percent of its students. And in spring 2017, Harvard’s CS50 reported nearly 60 of 600 students for cheating.

The aftermath

What happens after an instructor confronts a student about plagiarism?

Some admit their mistake right away. Others deny it for awhile before coming clean. And every semester, there will be a few conversations that go something like this:

“The TAs think you copied a few homeworks. You need to tell me what’s going on.”

“I didn’t cheat. I don’t know what you’re talking about.”

“Look, Bob, you have three screens of code that are exactly the same as Alice’s, except she called the parameter searchQuery and you renamed it to info_strings in your final commit.”

“Those similarities are a coincidence that happens fairly often in programming. Stop harassing me!” *files complaint with department*

Trying to gaslight the computer science department about computer science makes no sense, but let’s move on.

The long tail of case resolution times. After confronting students, conversations like the above take up the vast majority of the teaching staff’s time. A small but vocal minority of students hope to avoid consequences by dragging out their cases.

They send endless emails to the teaching staff, sometimes even getting parents involved. They appeal to the relevant offices in the university, claiming that in computer programming, independently writing hundreds of identical lines of code is a common occurrence. To refute these claims, the teaching staff might then be asked to provide a nontechnical explanation of the similarities, sometimes months after the semester has ended.

Combined with the time it takes to find cases in the first place, the time sink alone is enough to discourage many instructors from looking. But don’t worry, because there’s more:

Lack of support from the university. When there’s enough noise, administrators in high places start getting spooked. No one wants to end up on the front page of the New York Times. So rather than offer their support and influence, they do nothing. Teaching staffs are left to deal with the issue using their own time and resources.

Becoming the bad guy. At the end of it all, students take out their anger in course evaluations. Instructors become “terrible human beings” and worse for daring to find out why so many programs written by different people have the exact same bugs.

These reviews matter: the hiring process for faculty positions often requires candidates to submit their past student evaluations. And like any other form of anonymous online abuse, repeatedly reading personal attacks and other vitriol about yourself as part of your job carries a large mental health cost. (Imagine if your boss outsourced a portion of your quarterly performance review to a panel of YouTube commenters and Twitter trolls.)

On a more personal level, it hurts to know which students are plagiarizing. We teach because we want to help people learn. It might be a little disappointing when, for instance, a student comes to office hours at the end of a semester not knowing how to compile programs. But it’s far worse to find out that they’d asked for compilation help because they were trying to submit a friend’s solution.

Competing against inaction

It’s clear that instructors who choose to pursue plagiarism cases encounter a wide variety of disincentives, ranging from slightly unpleasant to truly dreadful. But on an individual level, nothing bad really happens to those who don’t bother confronting plagiarism directly. Research-track faculty can still publish their papers. And with fewer negative reviews, teaching-track lecturers might even improve their chances of being hired in the future.

So it should come as no surprise that most professors avoid doing anything about plagiarism directly.

Many creative solutions

I attended a discussion on plagiarism at this year’s SIGCSE, a computer science education conference. It opened my eyes to all the ideas faculty have to avoid facing plagiarism directly.

Some of them simply don’t generalize, like creating new homework assignments from scratch each semester. People who bring this up have likely never asked an instructor of an introductory course whether they’d like to spend large chunks of time rushing out assignments in exchange for also reducing their quality. And it still wouldn’t address plagiarism among students in the same semester.

But what really bothered me were all the moralistic solutions.

Making students sign an honor pledge. Offering leniency to those who turn themselves in during the 72-hour “regret period” after the deadline. Or even inviting students to email the instructor at any time if they are about to cheat, so that the instructor can talk them out of it one-on-one.

These are great ideas — for supporting students in courses with large enrollments. But they cannot be the only solution to combating plagiarism, because they do nothing to address the underlying incentive structure:

• Benefit: Copying and pasting someone else’s code saves a ton of development time, and ensures a higher grade. Good grades might be tied to desirable outcomes like graduation, financial aid, and job prospects.
• Cost: Zero! If the teaching staff doesn’t look for plagiarism directly, students know they’ll never be caught.

Empirical evidence shows that this is true. Our students sign an honor pledge at the beginning of the semester, but we did not see any change in the rate of plagiarism. Even for Harvard’s CS50, the birthplace of the “regret period,” David Malan reports that it “has not materially impacted CS50’s number of cases.”³

Finding our way out

Student incentives come from faculty

Student incentives are set by the faculty: for example, if the instructor assigns a larger weight to some assignment, students usually spend more time working on it. This means instructors have a lot of influence here! They can combat plagiarism by making it costly.

If students knew that there would be a consistent, non-zero cost to plagiarizing, many of them wouldn’t do it. Copying homework would no longer be in their best interest. To have an effect, the policy would have to be implemented consistently across all courses, beginning with the introductory courses, which play the role of communicating the department’s standards to new students.

Achieving this level of uniformity means we need to provide a strong incentive every instructor — not just the ones who care the most — to enforce the rules.

Fixing faculty incentives

University administrators should communicate their support. Instructors should know that, not only will they suffer no retaliation, but that the university encourages them to enforce university policies. This might require administrators to acknowledge the inconvenient truth of widespread plagiarism.

Next, we need to reduce the cost of looking for plagiarism.

Efficiently deploying plagiarism detection software. Most instructors I’ve spoken with use MOSS (if they use automated detection at all). However, each teaching staff must learn to use the software on their own, sometimes writing additional code to format the inputs or aggregate the results. We’ve somehow managed to turn plagiarism software — which should be written once and deployed widely at zero marginal cost — into something that is very expensive to use!

Instead of deploying the software on a per-class basis, the university should pay to integrate it into their learning management systems. The software could even automatically run on all programming assignments by default.

Spreading the workload among more TAs. Universities can further decrease the burden by increasing the TA headcounts of courses that enforce plagiarism rules. This ensures that time spent combing through the results of the software doesn’t come at the cost of teaching.

Improving the quality of results from plagiarism detection software. The software returns more false positives as class sizes increase, since the number of pairs of students grows quadratically. But because MOSS is proprietary, the community’s open-source development revolves around creating tools that wrap MOSS, rather than improving the core algorithm. Anyone who has an idea for improving the algorithm must first reimplement all existing functionality starting from the pseudocode in the paper.⁴ There is an opportunity to have a big impact by creating a free and open source alternative.

We won’t be able to fix everything. For example, personal attacks in course evaluations may be interleaved with genuine criticisms, making them difficult for moderators to separate.

---

I’m not smart enough to have all the answers. These are just some of my ideas for moving incentives in the right direction. It’s not even clear which ideas will work. But one thing’s for certain: students will learn to care when their teachers start caring.

Thanks to Nelson Gomez, Joshua Zweig, John Hui, Vivian Shen, Edward Wang, and two anonymous readers for their feedback and discussion.

---