All authors contributed to the conceptualisation and writing of the article;
D.T. was responsible for project leadership and funding acquisition.
References
1. Baltimore D, Berg P, Botchan M, Carroll D, Charo RA, Church G, et al. A prudent path forward for genomic engineering and germ line gene modification.
Science. 2015;348(6230):36–38. https://doi.org/10.1126/science.aab1028 2. Normile D. CRISPR bombshell: Chinese researcher claims to have created
gene-edited twins. ScienceMag. 2018 November 26 [cited 2019 Jul 18].
https://doi.org/10.1126/science.aaw1839
3. Association for Responsible Research and Innovation in Genome Editing.
Common statement between the Association for Responsible Research and Innovation in Genome Editing (ARRIGE), the Genome Writers Guild (GWG) and the Japanese Society for Genome Editing (JSGE) [document on the Internet]. c2019 [cited 2019 Jul 18]. Available from: https://arrige.org/
Common_statement_Arrige_GWG_JSGE.pdf
4. World Health Organization. WHO expert panel paves way for strong international governance on human genome editing [webpage on the Internet]. c2019 [cited 2019 Jul 18]. Available from: https://www.who.int/
news-room/detail/19-03-2019-who-expert-panel-paves-way-for-strong- international-governance-on-human-genome-editing
5. Germline editing: time for discussion. Nat Med. 2015;21(4):295. https://doi.
org/10.1038/nm.3845
6. Lander E, Baylis F, Zhang F, Charpentier E, Berg P. Adopt a moratorium on heritable genome editing. Nature. 2019;567:165–168. https://doi.
org/10.1038/d41586-019-00726-5
7. Wolinetz CD, Collins FS. NIH supports call for moratorium on clinical uses of germline gene editing. Nature. 2019;567:175. https://doi.org/10.1038/
d41586-019-00814-6
8. National Academies of Sciences, Engineering, and Medicine. Human genome editing: Science, ethics, and governance. Washington DC: The National Academies Press; 2017. https://doi.org/10.17226/24623
9. Ormond KE, Mortlock DP, Scholes DT, Bombard Y, Brody LC, Faucett WA, et al. Human germline genome editing. Am J Hum Genet. 2017;101(2):167–
176. https://doi.org/10.1016/j.ajhg.2017.06.012
10. Nuffield Council on Bioethics. Genome editing – an ethical review [document on the Internet]. c2016 [cited 2019 Jul 18]. Available from: https://
nuffieldbioethics.org/wp-content/uploads/Genome-editing-an-ethical-review.
11. Nuffield Council on Bioethics. Genome editing and human reproduction:
Social and ethical issues [webpage on the Internet]. c2018 [cited 2019 Jul 18]. Available from: https://nuffieldbioethics.org/publications/genome- editing-and-human-reproduction
12. Lagay F. Gene therapy or genetic enhancement: Does it make a difference?
AMA J Ethics Virtual Mentor. 2001;3(2). https://doi.org/10.1001/virtualmen tor.2001.3.2.gnth1-0102
13. Nordgren A. Designing preclinical studies in germline gene editing: Scientific and ethical aspects. J Bioeth Inq. 2019;16:559–570. https://doi.org/10.1007/
s11673-019-09947-9
14. South African Department of Health (DoH). Ethics in health research:
Principles, processes and structures. Pretoria: DoH; 2015.
15. Health Professions Council of South Africa. Guidelines for good practice in the health care professions. Booklet 14: General ethical guidelines for biotechnology research in South Africa [document on the Internet]. c2008 [cited 2020 Sep 10]. Available from: https://www.sada.co.za/media/
documents/HPCSA_Booklet_14_Biotechnology_Research_in_SA.pdf 16. South African Medical Research Council. Guidelines on ethics in reproductive
biology and genetic research [document on the Internet]. [updated 2002–
2004; cited 2019 Jul 18]. Available from: http://www.mrc.ac.za/sites/default/
files/attachments/2016-06-29/ethicsbook2.pdf
17. Macintosh KL. Heritable genome editing and the downsides of a global moratorium. CRISPR J. 2019;2(5):272–279. https://doi.org/10.1089/
crispr.2019.0016
18. Haque IS, Lazarin GA, Kang HP, Evans EA, Goldberg JD, Wapner RJ. Modeled fetal risk of genetic diseases identified by expanded carrier screening. JAMA.
2016;316(7):734–742. https://doi.org/10.1001/jama.2016.11139 19. National Health Act 61 of 2003, South Africa.
20. The Republic of South Africa. Regulations Relating to Research with Human Participants GN R719 GG 38000 of 19 Sep 2014.
21. The Republic of South Africa. Regulations relating to the Use of Human Biological Material GN R177 GG 35099 of 2 Mar 2012.
22. Medicines and Related Substances Control Act 101 of 1965, South Africa.
23. Reitzer Pharmaceuticals (Pty) Ltd v Registrar of Medicines 1998 (4) SA 660 (T).
24. Bato Star Fishing (Pty) Ltd v Minister of Environmental Affairs and Tourism and Others 2004 (4) SA 490 (CC).
25. Natal Joint Municipal Pension Fund v Endumeni Municipality 2012 (4) SA 593 (SCA).
26. Minister of Safety and Security and Another v Xaba 2003 (2) SA 703 (D).
27. Rossouw v Sachs 1964 2 SA 551 (A).
28. Habermas S. The future of human nature. Cambridge: Polity; 2003.
29. Bosley KS, Botchan M, Bredenoord AL, Carroll D, Charo RA, Charpentier E, et al. CRISPR germline engineering – the community speaks. Nat Biotechnol.
2015;33:478–86. https://doi.org/10.1038/nbt.3227
30. Brokowski C. Do CRISPR germline ethics statements cut it? CRISPR J.
2018;1(2). https://doi.org/10.1089/crispr.2017.0024
31. AB v Minister of Social Development [2016] ZACC 43, 2017 (3) SA 570 (CC).
32. Jordaan DW. Stem cell research, morality, and law: An analysis of Brüstle v Greenpeace from a South African perspective. S Afr J Hum Rights.
2017;33(3):429–451. https://doi.org/10.1080/02587203.2017.1392839 33. The US National Academies of Sciences, Engineering and Medicine.
International summit on human gene editing: A global discussion [webpage on the Internet]. c2015 [cited 2019 Nov 26]. Available from: https://www.
nap.edu/catalog/21913/international-summit-on-human-gene-editing-a- global-discussion
34. US National Institute of Health. Statement on NIH funding of research using gene-editing technologies in human embryos [webpage on the Internet].
c2015 [cited 2019 Sep 24]. Available from: https://www.nih.gov/about-nih/
who-we-are/nih-director/statements/statement-nih-funding-research-using- gene-editing-technologies-human-embryos
35. Comfort N. Can we cure genetic diseases without slipping into eugenics?
The Nation. 2015 July 16 [cited 2019 Aug 24]. Available from: https://www.
thenation.com/article/can-we-cure-genetic-diseases-without-slipping-into- eugenics/
36. Association for Responsible Research and Innovation in Genome Editing.
Statement from ARRIGE steering committee on the possible first gene-edited babies [document on the Internet]. c2018 [cited 2019 Sep 01]. Available from: http://arrige.org/ARRIGE_statement_geneeditedbabies.pdf
37. Harris J. Enhancing evolution: The ethical case for making better people.
Princeton, NJ: Princeton University Press; 2010
38. Bailey R. Liberation biology: The scientific and moral case for the biotech revolution. Amherst, NY: Prometheus; 2005.
39. Greely HT. Human germline genome editing: An assessment. CRISPR J.
2019;2:253–265. https://doi.org/10.1089/crispr.2019.0038
40. The Republic of South Africa. Regulations Relating to Artificial Fertilisation of Persons GN R175 of 2012.
41. Savulescu J, Pugh J, Douglas T, Gyngell C. The moral imperative to continue gene editing research on human embryos. Protein Cell. 2015;6(7):476–479.
https://doi.org/10.1007/s13238-015-0184-y
42. Silver LM. Remaking Eden: How genetic engineering and cloning will transform the American family. New York/London: Ecco; 2007 .
43. Kotzè M. Human genetic engineering in the South African context with its inequalities: A discourse on human rights and human dignity. Scrip.
2014;113(1)1–11. https://doi.org/10.7833/113-0-722
44. Minister of Home Affairs v Fourie; Lesbian and Gay Equality Project v Minister of Home Affairs [2005] ZACC 19, 2006 (1) SA 524.
45. Hemel D. The case for universal health coverage for in vitro fertilization.
The Seattle Times. 2018 November 15 [cited 2020 Jan 02]. Available from: https://www.seattletimes.com/opinion/the-case-for-universal-health- coverage-for-in-vitro-fertilization
46. Vélez MP, Connolly MP, Kadoch IJ, Phillips S, Bissonnette S. Universal coverage of IVF pays off. Hum Reprod. 2014;29(6)1313–1319. https://doi.
org/10.1093/humrep/deu067
47. National Health Insurance Bill. B11-2019, South Africa.
© 2020. The Author(s). Published under a Creative Commons Attribution Licence.
Sisa Pazi1
Chantelle M. Clohessy1 Gary D. Sharp1 AFFILIATION:
1Department of Statistics, Nelson Mandela University, Port Elizabeth, South Africa
CORRESPONDENCE TO:
Gary Sharp EMAIL:
Gary.sharp@mandela.ac.za DATES:
Received: 20 Apr. 2020 Revised: 18 May 2020 Accepted: 13 June 2020 Published: 29 Sep. 2020 HOW TO CITE:
Pazi S, Clohessy CM, Sharp GD. A framework to select a classification algorithm in electricity fraud detection. S Afr J Sci.
2020;116(9/10), Art. #8189, 7 pages. https://doi.org/10.17159/
sajs.2020/8189 ARTICLE INCLUDES:
� Peer review
☐ Supplementary material DATA AVAILABILITY:
☐ Open data set
☐ All data included
� On request from author(s)
☐ Not available
☐ Not applicable EDITOR:
Amanda Weltman KEYWORDS:
electricity fraud detection, confusion matrix, classification algorithms FUNDING:
SASA/NRF Crisis in Academic Statistics (grant no. 94756); Nelson Mandela University.
In the electrical domain, a non-technical loss often refers to energy used but not paid for by a consumer.
The identification and detection of this loss is important as the financial loss by the electricity supplier has a negative impact on revenue. Several statistical and machine learning classification algorithms have been developed to identify customers who use energy without paying. These algorithms are generally assessed and compared using results from a confusion matrix. We propose that the data for the performance metrics from the confusion matrix be resampled to improve the comparison methods of the algorithms.
We use the results from three classification algorithms, namely a support vector machine, k-nearest neighbour and naïve Bayes procedure, to demonstrate how the methodology identifies the best classifier.
The case study is of electrical consumption data for a large municipality in South Africa.
Significance:
• The methodology provides data analysts with a procedure for analysing electricity consumption in an attempt to identify abnormal usage.
• The resampling procedure provides a method for assessing performance measures in fraud detection systems.
• The results show that no single metric is best, and that the selected metric is dependent on the objective of the analysis.
Introduction
Revenue that is lost due to the difference between electricity supplied and electricity purchased is partitioned into two classes. The first class resulting from transmission and other infrastructural limitations is labelled as technical losses, whilst the second class, the majority of which are a result of meter tampering or bypassing, is labelled as non-technical losses1-3 (NTL). Estimates of losses worldwide are in the billions4,5 of US dollars and suppliers of electricity have expressed concern over these losses and the sustainability of the supply6,7.
The literature related to fraudulent electricity losses is detailed with the first traceable case as early as the 19th century claiming, ‘unprincipled persons had availed themselves of the opportunity to steal electricity’8,9. More recent literature is a result of the computational hardware and software developments over the last two decades.
Galvan et al.10 brought to the fore statistical methods for identifying ‘abnormal’ consumer behaviour in the electrical domain. This has seen fraud detection systems from finance, banking and insurance being applied to the electricity domain. The review by Messinis and Hatziargyriou11 of the methods applied to detect electricity theft provides compelling evidence that the research domain is exciting and extensive. The review informs researchers of the types of data used in fraud detection, the algorithms that have been proposed and clarifies performance metrics for comparing the algorithms.
A comprehensive list of the classifiers used in electricity fraud detection systems can be found in Messinis and Hatziargyriou11. Notably these include support vector machines (SVM), naïve Bayesian (NB) methods and k-nearest neighbour (k-NN) classifiers – the algorithms used in this study. The field has not stagnated; recent computational methods include convolutional neural networks3 and ensemble-based classifiers7 whilst time series methods have been explored2. In many of these studies, the common methodological approach is to assess the classifier by considering results from a test data set captured in a confusion matrix summary and reported as a performance measure. Several performance measures are used in the literature and for the most part are defined to reflect accuracy and precision of the classifiers.
The confusion matrix is a 2 x 2 table summarising the predicted versus the actual frequency counts for a binary classification model. The table, used in the financial sector to identify fraudulent customers, is best suited to data sets for which the number of observations in the two classes are similar or moderately similar. Ideally for a model, the predicted results match the actual counts. In the confusion matrix, this implies that the frequencies of the true positives are the same as the actual positives and the frequencies of the true negatives are the same as the actual negatives. In addition, the false negatives and false positives should be zero. The confusion matrix, in Table 1, was used in studies by Messinis et al.12 and Li et al.3 to assess classification models, whilst Guo et al.13 addressed the problem for data sets which are highly imbalanced.
Our research study had a dual objective. The first objective was to use a resampling approach to assess the performance of the classifier using a data set from the Nelson Mandela Bay Municipality. The second objective was to add to the South African literature on fraud detection in the electrical domain.
Table 1: A 2 × 2 confusion matrix for two classes True class Predicted class
Total
Positive Negative
Positive True positive (TP) False negative (FN) Actual positive Negative False positive (FP) True negative (TN) Actual negative Total Predicted positive Predicted negative Total counts