There are many software systems that suggest that they can reliably determine if a submitted text or an online document is plagiarized or not. This paper discusses the problems associated with such software and reports on a test of plagiarism detection and collusion detection systems conducted in 2007. There were no systems that achieved a mark of very good and there was only one system, Ephorus, which was just barely given a mark of good. Eight systems were graded acceptable, with the popular Turnitin only reaching eighth place, and four systems were found to be completely unacceptable. Details of the test methodology and the test cases are given, as well as results for each software system. The paper concludes that plagiarism detection systems cannot be relied on to find all instances of plagiarism, and many cannot even find simple plagiarisms.