Results of the Plagiarism Detection System Test 2010
A test of 26 plagiarism detection systems has the top scorers detecting only 70% of the plagiarism for a grade of C-
„Plagiarists beware – we use software!“ Many universities attempt to coerce their students into compliance with good scientific practice with the use of plagiarism detection software. And there are many systems that offer such services. They suggest a fast, cheap, and easy way to winnow out the plagiarists from the honest authors.
Prof. Dr. Debora Weber-Wulff, professor for media and computing at the University of Applied Sciences HTW Berlin has tested plagiarism detection systems in 2004, 2007, and 2008 and has published widely on the topic. In this fourth test series that was just completed at the end of 2010, 26 systems out of 47 available systems were closely examined. Particular focus was given to seeing how well the systems detect a known amount of plagiarism, and how they react when offered original material.
In 2008 the systems fared slightly better on the test, but in 2010 many have lost the ability to detect plagiarisms that have been slightly edited – word orders switched, words dropped or added, or synonyms used. The best systems only reached a grade of C-, not quite reaching 70% of the possible points.
The 2010 test included not only short essays in German, but also ones in English and Japanese. Additionally, a usability metric was calculated that took into account aspects such as design, language consistency, navigation, print quality of the reports, and how well the system fits into the workflow of a university. A new professionalism metric includes giving a real street address and the name of a contact person; not advertising for paper mills or ghostwriting services; answering the phone during normal business hours and not installing malware on the computer under the guise of installing the detection software.
The systems were categorized as partially useful, barely useful, and useless for university purposes. The best systems with between 60 and 70% effectiveness are PlagAware, Turnitin, Ephorus, PlagScan and Urkund.
Our recommendation: Only use these systems when suspicions of plagiarism arise that cannot be found with 3-5 words in a search machine. The focus should be on teaching students about plagiarism and how to avoid it instead of investing time in using software. Most of the work involved later is in preparing a plagiarism case and dealing with the plagiator, and for this good documentation is needed. Very few systems provide good documentation.
Test Overview
The test overview chart shows that the results of the test in a large chart. A summary (pdf) of the test and the individual results (only in German) are available. We have described all of test cases and the Evaluation (in German) of the test cases.
Evaluation of plagiarism detection systems
Partially Useful Systems
Barely Useful Systems for Education
- 6: Plagiarism Finder
- 7: Docoloc
- 8: Copyscape Premium
- 9: Blackboard/SafeAssign
- 10: Plagiarisma
- 11: Compilatio
- 12: StrikePlagiarism
- 13: The Plagiarism Checker Free
- 14: The Plagairism Checker Premium
Useless Systems for Education
- 15: iPlagiarismCheck
- 16: Plagiarism Detector
- 16: UN.CO.VER
- 18: GenuineText
- 19: Catch It First
- 20: plagium
- 21: Viper
- 22: PlagiarismSearch
- 23: PlagiarismChecker
- 24: Grammarly
- 25: PercentDupe
- 26: ArticleChecker
DiscontinuedTests
- CopyScape free
- FairShare
- AntiCutAndPaste
- DocCop
