6.37 When discussing the admission of evidence from devices controlled by software code, judges do not distinguish between a single, highly specialist device that is self- contained, and a linked network containing any number of devices each independently operating on its own set of software code. As noted above, when considering cases dealing with specialized devices such as breath testing machines and blood testing machines, judges have used nebulous terms in the absence of scientific analysis, using such terms as such as ‘notoriety’, ‘common knowledge’ and ‘properly constructed’.
There is little evidence to demonstrate that proper evidential foundations have been adduced to permit such presumptions to be admitted. In this regard, it is useful to consider, although not exclusively, the case law in Australia, where these devices have been subjected to greater judicial analysis.
6.38 The Southern Australian case of Mehesz v Redman1 was a case that concerned the method of analysing a blood sample. At trial, the Special Magistrate categorized the blood sample testing device as a scientific instrument with the presumption that it was in the category of a ‘notorious’ instrument whose accuracy is presumed. On appeal, Zelling J rejected this on the basis that the device was not a mere calculator, although it interpreted the results because of the software program, because there was
no evidence to demonstrate that the machine was accurate or reliable. The appellant was tried a second time, convicted again, and appealed to the Supreme Court once more. This appeal was referred to the full court.2 The main argument of counsel for the appellant related to the evidence tendered by the prosecution regarding the analysis of a blood sample, in that the evidence relied on the use of two instruments (a gas chromatograph and the ‘Auto-lab system 4B’ data analyser) whose accuracy had not been established. King CJ rejected the submission that the Auto-lab was an instrument that could not be relied upon because there was no evidence as to the ‘correctness’ of the software program. He said:
The courts do not require such evidence. If the instrument is so well known that its accuracy may be assumed as a matter of common experience, the Court is entitled to presume its accuracy without evidence.3
1 (1979) 21 SASR 569.
2 Mehesz v Redman (no 2) (1980) 26 SASR 244.
3 (1980) 26 SASR 244, 247.
6.39 Proof of the accuracy of a particular instrument will ‘ordinarily be proved by those who use and test it’, and the results obtained are acceptable in evidence ‘provided that the expert witness has himself formed an opinion that the methods used are apt to produce the correct result’.1 Notwithstanding the inability of the operator of a machine controlled by software code to demonstrate the accuracy or otherwise of the code that he does not control and has no ability to alter, this proviso is important. (White J also made a similar point.2) This means that the operator of such a machine ought to be able to assess when the machine produces results that are not expected, even if the operator is not able to establish why the results produced are wrong. If such a machine produces results that are not anticipated, the operator is put on notice that the machine (and the software code) might not be reliable. In such circumstances, it will be necessary to have the machine tested before being relied upon for future analysis.
1 (1980) 26 SASR 244, King CJ at 248.
2 (1980) 26 SASR 244, 254.
6.40 Dealing with the submission that the prosecution failed to provide proper foundations for the Auto-lab analyser, White J set out the conditions that must be fulfilled before evidence will be admitted regarding the measurements of scientific instruments:
1. If the instrument falls within the class of instrument known as notorious scientific instruments, the court will take judicial notice of its capacity for accuracy, so that the operator merely proves that he handled it properly on the particular occasion.
2. If the instrument is not a notorious scientific instrument, its accuracy can be established by evidence: (a) that the instrument is within a class of instrument generally accepted by experts as accurate for its particular purpose; (b) that the instrument, if handled properly, does produce accurate results: ((a) and (b) must be established by expert testimony, that is, by experts with sufficient knowledge of that kind of instrument; and upon proof of (a) and (b), a latent presumption of accuracy arises which allows the court to infer accuracy on the particular occasion if it is proved) – (c) that the particular instrument was handled properly and read accurately by the operator on the particular occasion; ((c) can be established by
a trained competent person familiar with the operation of the instrument, not necessarily the type of expert who proves (a) and (b)).
3. Where the actual accuracy of the measurement can be inferred from all of the proved circumstances, it is not necessary to rely upon the presumption arising from (a) and (b), proof of which is superfluous.1
1 (1980) 26 SASR 244, 251 – 252.
6.41 At the second trial, the prosecution called evidence from Professor Northcote, Chairman of the School of Mathematics and Computers at the Institute of Technology in South Australia, and an expert in mathematics, physics and computers. He gave evidence about the workings of the Auto-lab from his reading of the manufacturer’s manual and his understanding of the content of the manual. He was not able to read the software code, because the manufacturer had sealed the program against inspection, tampering and modification. Although Professor Northcote was not an expert in relation to the Auto-lab, the members of the Court of Appeal in the Supreme Court were of the option that both Professor Northcote and Mr Vozzo, who gave evidence at both trials, were sufficiently qualified to give evidence, even though neither witness had access to, nor any knowledge of, the software code. The Chief Justice also stated that ‘It is sufficient that the expert who uses it is able to say that it is an instrument which is accepted and used by competent persons as a reliable aid to the carrying out of the scientific procedures in question and that he so regards it.’1 He also prayed in aid the observations of Wigmore on Evidence to support this comment:
(2) Scientific instruments, formulas, etc. The use of scientific instruments, apparatus, formulas, and calculating-tables, involves to some extent a dependence on the statements of other persons, even of anonymous observers. Yet it is not feasible for the professional man to test every instrument himself; furthermore he finds that practically the standard methods are sufficiently to be trusted.
Thus, the use of a vacuum-ray machine may give correct knowledge, though the user may neither have seen the object with his own eyes nor have made the calculations and adjustments on which the machine’s trustworthiness depends.
The adequacy of knowledge thus gained is recognized for a variety of standard instruments.3
1 (1980) 26 SASR 244, 247.
2 (3rd ed), Volume 2, paragraph 665a.
3 (1980) 26 SASR 244, 247.
6.42 In this case, the court emphasized that there was evidence other than the trustworthiness of the software code that enabled the evidence from the machine to be admitted as being accurate. White J set out the following analysis of the problem:
The only defect in the expert evidence of Dr. Northcote and Mr. Vozzo, if defect it be, was their lack of direct knowledge of the internal operations of the sealed instrument. They relied upon what the manufacturer said about its operation.
The extreme position would be that only the expert actually supervising the manufacture of the instrument in the United States of America could prove (a) and (b). I do not think that the rules relating to expert evidence encourage that kind of extreme position. Quite apart from questions of expense and delay in the administration of justice, the Court is entitled to rely upon evidence of measurements made by instruments which reputable scientists accept as accurate, whether those scientists have direct knowledge of the reasons for the instrument’s accuracy or not, provided they have knowledge that the
instrument’s measurements are accurate according to a known standard, or are accepted as accurate by reputable scientists.1
1 (1980) 26 SASR 244, 253.
6.43 By implication, the court concluded that it would be extreme to establish the reliability of a software controlled device in a court of law by analysing the software code – the very software code that controlled the device and provided the evidence.
The court considered that evidence from the operator of the device was sufficient for the trial court to assess the accuracy of the evidence. The appeal was dismissed.
6.44 Given these comments, it is understandable that the court reached the conclusions it did in Mehesz v Redman (No 2). At issue was a self-contained device that was used by trained operators with suitable qualifications. On the basis that the readings from such devices were, at any time, not within the expected range, the suitably trained and qualified operators were expected to use their professional judgment to verify the reliability of the device before submitting the evidence for legal proceedings.
In such a case, the court would not require the software code to be challenged.
6.45 The case of Bevan v The State of Western Australia1 illustrates the approach taken when considering the admission of evidence from computers and computer- like devices. One of the grounds of appeal in this case was the admissibility of mobile telephone data in the form of text messages downloaded by a computer software program. An investigating police officer carried out two separate downloading operations using two separate tools, Cellebrite and XRY. At the beginning of the trial, counsel for the accused objected to the text messages being received into evidence. The trial judge held that the text messages were admissible. Questions were raised as to the reliability of the software and of the officer’s correct use of it. The Court of Appeal concluded that the trial judge erred in law in admitting the text messages into evidence.
This was because the officer did not explain the process of how he downloaded it in any detail at trial: it was the first time he had used the relevant software, and he did not have any formal training in its use. When considering the rebuttable presumption at common law as to the accuracy of ‘notorious’ scientific or technical instruments, Blaxell J said that ‘when evidence from a new type of scientific instrument or process is adduced for the first time, there must be proof of its reliability and accuracy’.2 He went on to say that:
When specific evidence of the accuracy of a new instrument is required, this need not come from the manufacturer. It is sufficient that the expert who uses it can say that it is an instrument which is accepted and used by competent persons as a reliable aid in the carrying out of the scientific procedure in question, and that he so regards it.3
1 [2010] WASCA 101.
2 [2010] WASCA 101, [30].
3 [2010] WASCA 101, [31].
6.46 Blaxell J approved of the observations by White J1 in Mehesz v Redman (No. 2) as noted above. He continued:
To the above principles I add the obvious comment that a court will not be satisfied that an instrument was ‘handled properly’ on a particular occasion, if it does not understand what was required of the operator for this to be so. Detailed
evidence as to the workings of the instrument need not be given … However, it is necessary that there be sufficient evidence for the court to apprehend what it was that the operator had to do in order to ensure an accurate result.2
1 Mehesz v Redman (no 2) (1980) 26 SASR 244 at [251]–[252].
2 [2010] WASCA 101, [33].
6.47 In essence, Blaxell J is saying that if the user of a smartphone can give evidence to demonstrate that he can use the smartphone, it follows that he is sufficiently knowledgeable to give evidence indirectly that the software code that controls the device is ‘working properly’, ‘reliable’ or ‘accurate’. It is as if the software programs that form the device are irrelevant. Additionally, no attempt was made to define how software code can be determined to be ‘working properly’, ‘reliable’ or ‘accurate’.
6.48 In Bevan v The State of Western Australia, the Court of Appeal heard a second appeal in the same case after a re-trial. The same argument arose regarding the method of downloading the data from the mobile telephone. There was a trial within a trial concerning the evidence of Detective Tomlinson. (Buss J referred to him as a First Class Constable, and set out his qualifications.2) Counsel for the appellant conceded that the witness was qualified to operate the equipment used to perform the download, but argued that he was not qualified to give evidence about the accuracy of the download material and the reliability of the material itself. In cross-examination, Detective Tomlinson explained he did not hold a certificate in relation to the Cellebrite and XRY software packages, but that he had been shown how to use them on about ten occasions. The following exchange took place regarding how the software worked:
Q. Can you tell me how the Cellebrite package actually works.
A. I don’t understand the question.
Q. How does it work? Explain to me, a layman, who knows nothing about Cellebrite, how it works.
A. It extracts data from a telephone.
Q. How? How does it do that?
A. It uses software.
Q. And how does that software work?
A. I couldn’t tell you.
Q. What about the XRY?
A. The same.
Q. If you don’t know how it works, how can you say its [sic] reliable?
A. You’d have to ask the manufacturer.
Q. Okay. I’m asking you. How can you say its [sic] reliable.
A. I can’t.
Q. You can’t. And, in fact, on one occasion that you used it in relation to the Nokia, it was unsuccessful.
A. Yes, that’s right.3 1 [2012] WASCA 153.
2 [2012] WASCA 153, [18]–[21], [105].
3 [2012] WASCA 153, [20], the last question and answer is at [106(g)].
6.49 In deciding to allow the evidence before the members of the jury, the trial judge said:
The workings of the instrument need not be given and it seems to me that in this case the notes of the experienced officer, the evidence that this software is regularly used by him establishes the level of accuracy and in his notes at the time that he was – successfully used the program seems to me to meet the tests ... He was a trained, experienced and competent operator and the software was operated properly and, in those circumstances, in this case I think this evidence is admissible and I will allow it to be given by the qualified expert.1
1 [2012] WASCA 153, [201].
6.50 Pullin and Mazza JJA agreed the trial judge did not err in overruling the objection to the tendering of the text messages. In essence, because Detective Tomlinson was qualified as an expert, he could testify about the performance of the machines and the software. It was inferred that as an expert, he considered the process to be accurate, and that because he had performed such actions previously, the actions undertaken on this particular occasion were properly performed – even though the user of the program will not know that it is giving inaccurate results. There was no requirement for the Detective to understand how the software worked, or whether there were any problems with the software he used.2 Pullin JA said: ‘His evidence provided sufficient assurance that the results produced by the machines were reliable and accurate, because he (a trained operator of the machines) observed them to be so.’3 But it does not follow that any operator of an electronic device will be able to detect if the device was malfunctioning in any way. As noted by Eric Van Buskirk and Vincent T. Liu:4
There is a general tendency among courts to presume that forensic software reliably yields accurate digital evidence. As a judicial construct, this presumption is unjustified in that it is not tailored to separate accurate results from inaccurate ones.
1 As in the case of the death of Casey Marie Anthony in 2011, for which see Craig Wilson, ‘Digital Evidence Discrepancies – Casey Anthony Trial’ (11 July 2011) <www.digital-detective.net/digital- evidence-discrepancies-casey-anthony-trial/>; Tony Pipitone, ‘Cops, prosecutors botched Casey Anthony evidence’ (Clickorlando.com, 28 November 2012) <www.clickorlando.com/news/cops- prosecutors-botched-casey-anthony-evidence>; Jose Baez and Peter Golenbock, Presumed Guilty:
Casey Anthony: The Inside Story (updated edn, BenBella Books 2013) 46, 180–183, 211, 346–348, 365, 368–371, 400, 426–428; Jess Ashton and Lisa Pulitzer, Imperfect Justice: Prosecuting Casey Anthony (William Morrow 2011) 105, 239, 277, 291–2, 298, 315.
2 [2012] WASCA 153, the rationale was set out at [66] and [67].
3 [2012] WASCA 153, [67].
4 Eric Van Buskirk and Vincent T Liu, ‘Digital evidence: challenging the presumption of reliability’
(2006) 1 Journal of Digital Forensic Practice 19.
6.51 They suggest there are two approaches to resolve the problem:
One is through the proper application of scientific jurisprudence to questions of digital evidence and the other is through some combination of certain broad market and social corrections.
6.52 The important question is: If the device was malfunctioning, how would the operator know? More significantly, the question should be: How would the malfunction manifest itself, if at all, and in a form evident to the operator?
6.53 In the minority, Buss J considered that none of the relevant basic facts and circumstances were proved. The judge considered the applicable legal principles in detail.1 He cited the relevant case law, and also extracts from The Science of Judicial Proof (3rd edn, 1937) by Professor Wigmore:
Professor Wigmore enunciated three fundamental propositions applicable to evidence based on the use of a mechanical or scientific instrument constructed on knowledge of scientific laws:
1. The type of apparatus purporting to be constructed on scientific principles must be accepted as dependable for the proposed purpose by the profession concerned in that branch of science or its related art. This can be evidenced by qualified expert testimony; or, if notorious, it will be judicially noticed by the judge without evidence.
2. The particular apparatus used by the witness must be one constructed according to an accepted type and must be in good condition for accurate work. This may be evidenced by a qualified expert.
3. The witness using the apparatus as the source of his testimony must be one qualified for its use by training and experience (§220).2 (original emphasis)
1 [2012] WASCA 153, [111]–[129].
2 Para 111.
6.54 The judge continued:
Wigmore on Evidence (Chadbourn Rev, Vol III, 1970) §795 states the requirements for the admissibility of evidence based on the use of scientific instruments, as follows:
What is needed, then, in order to justify testimony based on such instruments, is preliminary professional testimony: (1) to the trustworthiness of the process or instrument in general (when not otherwise settled by judicial notice); (2) to the correctness of the particular instrument; such testimony being usually available from one and the same qualified person. (original emphasis)1
1 [2012] WASCA 153, [112].
6.55 Buss J rejected the evidence of the Constable, partly because he was not qualified to comment of the software, and because the ‘machines/software’ were not so well-known that their accuracy may be assumed as a matter of common experience.1 Evidence was required to demonstrate their accuracy. It followed that the State had to produce evidence from a suitably qualified expert of the trustworthiness of the machines and software in general, and of the correctness of the particular instruments for the purposes of downloading of data from mobile telephones.2 Arguably, had the State produced sufficient evidence to convince a judge of the accuracy of the machines and software, it would not have been necessary to reply on the presumption.
Notwithstanding this observation, the approach by Buss J is to be preferred. His brother judges appear to accept the astonishing conclusion that not having any knowledge of how a device works is irrelevant to the results of the analysis. In their approach, the work of software programmers is immaterial. Software code is not germane when determining causation. If this approach were accepted, no longer would decisions in legal proceedings be based on knowledge and systematic and scientific judicial inquiry.