Drones, A.I and the Future of Asset Data, from Zero(s) to Hero.
Shelley Burchett Proterra Group
Abstract:
Data - we use this term every day, but the big question is, are organisations leveraging data effectively to make informed decisions about their assets?
Collaboration between those organisations collecting data and those reviewing and analysing the data is key.
This presentation will discuss making data collection more accurate, cost effective and accessing up- to-date asset data faster through advances in technology and innovation.
So, back to data, the Merriam-Webster dictionary provides us with a simple, yet insightful definition of data, that being, “Data is factual information (such as measurements or statistics) used as a basis for reasoning, discussion or calculation”.
Data is collected constantly, according to Forbes, there are about 2.5 quintillion bytes of data (quintillion has 18 zeros folks) created each day at our current pace, with 90% of that being created over the past two years alone.
As Carla Fiorina (former executive, president, and chair of Hewlett-Packard Co) said, “The goal is to turn data into information, and information into insight”.
Now we are not all “data experts”, but to effectively utilise all this data we have collected, we need to analyse and understand it; essentially turning it from raw data to information.
Lucky technology can help us all out there. Technology is rapidly advancing, is increasingly affordable and is readily available (most things can be bought at a click of the mouse button and delivered to your door these days).
Over the past thirty years, leaps and bounds in technology have included: the internet, portable GPS, cloning, email and text messaging, hybrid cars, self-driving cars and drones – it boggles my mind to think what the next thirty years will bring.
Let us start exploring this together!
Keywords:
• Technology
• Asset Management
• GIS
• Data
• Artificial Intelligence
Introduction
Over the past thirty years, the technology we have access to has developed in leaps and bounds.
Some of these include: the internet, portable GPS, cloning, email and text messaging, hybrid cars, self-driving cars, and drones.
This paper will explore how technological advancements have made an impact within the Asset Management industry and the different hardware and software options available to organisations to achieve better practices and look to future possibilities.
A disclaimer, I am no way an expert on this subject matter, this paper (and accompanying presentation) is informative in nature only.
Body
Let us start at the beginning, a vital aspect of the asset management process is the data collection phase.
At this stage, the asset custodians come together and create a data attribute schema that can be used by the inspectors in the field. Asset custodians were once isolated within their organisations, relying upon their nown knowledge and experience (and that of those they were surrounded by) to develop their own best practice. Over the past thirty years, the internet has significantly changed the way we gain knowledge, communicate, and collaborate. It has allowed asset custodians to reach out on online forums (IPWEA Ask Your Mates Forum), join like-minded groups to share ideas (on
LinkedIn, Facebook), access practice notes and standards (Australian Standards, IPWEA, WSAA, NAMS), view other organisations asset data capture manuals (Google search) and collaborate on technical ideas to further better practices.
Data capture methods have also dramatically advanced with the help of technology - what was once a tedious paper checklist has now evolved into an app that can be used on a hand-held tablet.
Figure 1 Assetic Mobility
(courtesy of Assetic) Figure 2 Konect Manager Software (courtesy of Konect Manager)
Figure 3 Reflect Software (courtesy of Asset Edge)
There are a wide variety of apps currently on the market offering data collection options to suit a wide range of organisational sizes and needs. These apps all work and capture data slightly differently. For example, some may only import or export CSVs, while others are designed to work with spatial files. Some are form based, meaning that the field worker primarily works with a data entry form, while others are map based, and the field worker is primarily working with spatial details
& locations. All these apps have their strengths meaning it can be tricky to find the right app for your needs.
One of my favourites is the ESRI suite. In recent times, ESRI has released a number of different field collection apps, with another new app (an integration of multiple ESRI apps) currently in beta phase.
One of the greatest benefits of using the ESRI suite is the ability to seamlessly link the field collection with any pre and post data processing, with the ESRI suite offering full integration from field to office and beyond. This streamlines the entire process, provides consistency from start to finish, and enables real time updates.
The 3 main ESRI collection apps are ArcGIS Collector – a map-based application, ArcGIS Survey123 - a form-based application and ArcGIS QuickCapture – a big button application. All of which allow field workers to easily collect accurate data, even in offline settings, and enable everyone in the field and in the office to work from the same accurate data.
Figure 4 ArcGIS Collector Application (courtesy of ESRI)
Figure 5 ArcGIS Survey123 Application (courtesy of ESRI)
Figure 6 ArcGIS QuickCapture (courtesy of Asset ESRI)
We are currently working on trialling all of ESRI’s collection apps across our service offering at Proterra Group.
Once a data collection program has been agreed upon, we are ready to start inspecting, right? Not quite, there is one more consideration that should be discussed before project kick off, where will we store all this data?
Over the past fifty years considerable advancements in data storage have been reached. We have advanced through the following stages:
• Floppy disk to Compact Disc (CD),
• CD to Zip Drive,
• Zip Drive to Digital Video Disc (DVD),
• DVD to SD Card,
• SD Card to USB Flash Drive
• USB to Blu Ray Optical Disc,
• Blue Ray Optical Disc to Cloud Storage.
That is a lot of progression, and I can bet that many of you are still receiving data from your Contractor’s via hard drives. The hard drives are reviewed and then due to storage limitations, selected data is uploaded to your organisation’s server and then the hard drive placed in a box in a cupboard, never to be seen again.
Unfortunately, this method may create a silo of data within a department and may not allow all asset custodians / stakeholders access to their full data sets, therefore not utilising the data to its full potential.
However, moving towards a cloud storage system may help eliminate some of those data silos, advantages of a cloud storage system include:
• Reduction in data storages costs as cloud storage can be cheaper per GB compared to hard storage,
• Increase accessibility between teams and allows access from anywhere with an internet connection,
• Able to recover data more easily than a hard drive failure or other malfunction,
• Changes are easily synced, stored, and shared with all users, and
• Additional layers of security are often available for cloud storage providers.
So, before undertaking any project, I recommend having a conversation with all project stakeholders about the retention of your data.
Now, how are we inspecting our assets? Traditional methods of physical inspection visual
inspections are still most popular; however, technology moves so fast and the rise of drone usage within the realm of asset data collection has not gone unnoticed.
Drones can be used in a range of applications, some of these include:
• Helping farmers check their water troughs,
• Undertake aerial surveys of construction sites or to create 3D models or thermal maps, and
• Amazon Prime Air has recently received FAA approval, meaning that in getting your Amazon package via drone will be a reality (well in the USA anyways).
Whilst many people believe that drones are the silver bullet to solve all of their problems, I don’t think that is the case, however there is a huge case for them to be used in a range of public works situations. For example:
• Undertaking visual inspections of confined space areas or with limited access, e.g. dam walls, water tanks,
communication towers and buildings,
• Inspecting culverts that are too large for a QuickView or CCTV camera, but you do not want an inspector walking through.
• Conducting traffic network analysis with the goal to improve flow and safety conditions,
• Undertaking aerial topographic mapping for building, construction, and development projects,
• Create volume measurements and run cut fill analysis for example on small quarries or gravel pits, and
• Overlay and compare as designed with as-built construction.
Some popular drones being used for asset data capture and inspections include:
Figure 8 Elios Inspection Drone (courtesy of Nexxis)
Figure 9 DJI Matrice 200 Drone (courtesy of DJI)
Figure 10 DJI Mavic Pro Drone (courtesy of DJI)
Whilst drones have many advantages, do not forget to check the legal limitations in your area, remember the safety of the public and do not just let anyone operate it – training is required.
Another technology buzz word of the past couple of decades has been Artificial Intelligence or AI.
Back in the 1950s, the fathers of the field Minsky and McCarthy, described artificial intelligence as any task performed by a program or a machine that, if a human carried out the same activity, we would say the human had to apply intelligence to accomplish the task (Marr, 2019).
AI systems will typically demonstrate at least some of the following behaviours associated with human intelligence: planning, learning, reasoning, problem solving, knowledge representation,
Figure 7 A still from a culvert inspection undertaken by a drone (courtesy of Proterra Group)
perception, motion, manipulation and, to a lesser extent, social intelligence, and creativity (Marr, 2019).
AI is found everywhere today, it’s used to recommend what you should buy next online, to
understand what you say to virtual assistants such as Amazon's Alexa and Apple's Siri, to recognise who and what is in a photo, to spot spam, or detect credit card fraud (Marr, 2019).
Whilst these applications are outside the field of asset data collection and management, there are many exciting potential applications within our field. These include:
• The use of AI to auto recognise defects from CCTV inspections and standardise pipe condition grading,
• Organisations benefiting from improved search and discovery of data, allowing people to extract value from their data more efficiently, and
• Allowing AI software and machine learning to undertake time-consuming manual checks of data against data attribute schemas and field collected data.
Figure 11 AI identifying defects from a CCTV inspection
(courtesy of VAPAR) Figure 12 Machine learning classifying stormwater assets (courtesy of Proterra Group)
We are currently working on a project at Proterra Group which involves using AI and machine learning to validate data collected in the field by our crews.
Essentially the AI will review the images taken and classify the image into certain categories, for example asset type, chamber shape, grate shape and grate type. The AI will continue learning with the goal to teach it defect identification and condition grading.
Figure 13 AI during its training stages (courtesy of Proterra Group)
Whilst these technologies are remarkably interesting, they can seem a little daunting for many people. The good news is if you are not quite ready to make the leap, there have been plenty of advancements in current inspection equipment and software. Some of these include:
• Laser shape scanning (in conjunction with CCTV pipe inspections) allowing a cross section profile and a deformation measurement to be established for selected pipes,
• A wireless QuickView Camera that connects to a standard tablet for control, allows defect distance measurement and enables other onsite users to connect to the app to ‘view’ the inspections in real time,
• Crab crawler cameras which can be used for vertical pipe inspections and other crawlers designed for inspections of assets in extreme situations,
• The Cleverscan, designed for rapid, automated defect inspections and provides the user with a flat scan that captures image detail from every inch of manhole wall, and
• Prototype pipe crawling robots designed for high pressure pipelines.
Figure 14 Wireless Quickview airHD (courtesy of SECA)
Figure 15 Crab Crawler (courtesy of Nexxis)
Figure 16 Laser scanning profile (courtesy of iPEK)
Another great progress in the industry of pipe inspections is the release of WinCan Web.
WinCan Web allows the data inspected in the field to be viewed by the office teams in real-time.
Once an inspection is completed, the inspection data, images and video are uploaded (via an internet connection) to the cloud to be accessed by anyone with a secure log in.
The user selects the asset they would like to view from the section list, the defect data and video then appear.
If the user clicks on any of the defects from the list, it will take them to that defect in the video, they are also able to click on the camera icon to view the photo.
The software also allows users to select defects from the pipe diagram, with the same principle applying.
Figure 17 Still from WinCan Web VX (courtesy of Proterra Group)
Technology is also allowing the location of buried assets to become easier and more accurate. Gone are the days of taking a paper-based plan into the field and digging until you found your asset.
These newer tools helping asset custodians undertake non-invasive exploration whilst minimising service interruptions, risks, and costs. They can provide accurate locational information including GPS coordinates and depth to invert. Some of these tools include:
• A Sonde (or the magic wand) that is used in conjunction with a CCTV rover, essentially it is a self-contained battery operated transmitter used for tracing the paths of pipes, ducts, sewers and drains and in the precise location of blockages or collapses,
• Ground penetrating radar, which uses radar pulses to image the subsurface to investigate underground utilities such as concrete, asphalt, metals, pipes, cables, or masonry, and
• We also have a metal detector which is an electronic instrument that detects the presence of metal nearby. Metal detectors are useful for finding metal inclusions hidden within objects, or metal objects buried underground – particularly buried manhole lids.
Figure 18 Sonde Wand (courtesy of Rycom)
Figure 19 Ground Penetrating Radar
(courtesy of PTCE)
Figure 20 Metal Detector (courtesy of BCF)
So, now, you have collected your data, what do you do with it? How can you possibly validate analyse, visualise, and store all of this data?
Asset data has historically been used in a one-dimensional format but with the addition of spatial reference information, this is rapidly changing. By including spatial information within a dataset, we are able to map asset locations, and analyse or present them visually. Visualising data makes things much easier when it comes to analysis, querying, and manipulation. The data can also be simplified and presented for less experienced users to get value out of it.
During the project setup phase, the use of GIS software such as ArcPRO or ArcGIS online simplifies the querying of existing asset data and allows users to visually check and confirm the asset locations.
Data can also be manipulated if necessary, for use in the data collection app. This also assists with the streamlining of field worker movements and planning of works throughout the project.
The same applies in the post-processing of data with the additional benefit of real time data uploading and processing abilities. Users are able to monitor the project progress, adjust planning and workflows if necessary, and even relay this visually to others through maps or dashboards.
Additional data processing capabilities include the ability to run automated queries and checks, reality check assets visually, run dataset comparisons and simplify any processing on big datasets.
ESRI has also released additional review capabilities in the ArcPro data reviewer - which automates and simplifies data quality control using data constraints and queries.
Spatial information also creates the ability to present data to all users visually in map format or as dashboards showing final data statistics or progress updates. Because you do not fully understand your data until you see how it relates to other things in a geographic context.
Figure 21 Example ESRI Dashboard (courtesy of ESRI)
For those that may not have the resources to have a spatially integrated system to undertake your validation checks, analysing and visualisation of data, there are other options.
Now I know that some of you may find it hard to let go of your excel spreadsheets – I am one of those people, I’m very comfortable building them and using them to analyse data and create graphs and figures etc. – but there is a better way.
So, to help me analyse and visualise my data quicker and better, I have started using a Microsoft program called Power BI. Essentially it is designed to easily connect to, model, and visualise your data and it also helps you create reports quickly.
Some of the functions that I really love are:
• Undertaking data validity checks is simple and customisable depending on the data attribute schema we are working against,
• I can create rich and personalised dashboards that are easily shared with my team,
• It is super customisable, I add a little weather widget to each of my project dashboards,
• Report templates can be created, modifications to existing reports can be completed and published very quickly,
• As it is cloud-based software there does not seem to be any memory or speed constraints,
• Being a Microsoft program there are so many tutorials and how to videos available online,
• There is a drag and drop function when adding variables to your graphs and reports,
• And you can use it as a desktop program or web based.
Figure 22 Example Dashboard set up in Power BI (courtesy of Microsoft)
Conclusion
As you can see, technology has greatly affected the Asset Management Industry. There are so many options out there now, it can sometimes be difficult to decide whether to pursue any of them, so here are some tips I have picked up along the way:
• Build a business case, ensure you identify your objectives and calculate your return on investment,
• Do not get sucked in by the shiniest and brightest new toy, ensure you evaluate all your viable options,
• Ask around, there are many communities of people out there looking at using technology to better their practices, they often have some good advice, and
• Ensure you involve all stakeholders throughout the project, communicate well, create and easy transition and a support system.
And finally, a look to the future, as I am writing this paper there are a few far out, advancements in technology continually being developed, these include:
• Wireless energy transfer – imagine solar powered roads that can charge your car as you drive,
• Self-healing living concrete – scientist say this could help building heal their own cracks or suck up dangerous toxins from the air, and
• Microsatellites to provide internet access to remote areas – imagine internet for everyone.
Figure 23 Wireless energy transfer
(courtesy of Stanford University)
Figure 24 Self-healing living concrete (courtesy of Science Focus)
Figure 25 Microsatellites for internet coverage
(courtesy of Science Alert)
I cannot wait to see what the next thirty years hold!
Acknowledgements
To all the asset custodians out their trying new technology.
References
• AI: Pipe Inspection’s Next Big Thing - Trenchless Technology 2020, Trenchless Technology, viewed 26 October 2020, <https://trenchlesstechnology.com/artificial-intelligence-pipe- inspections-next-big-thing/>.
• BBC Science Focus Magazine 2020, Future technology: 22 ideas about to change our world, BBC Science Focus Magazine, BBC Science Focus Magazine, viewed 27 October 2020,
<https://www.sciencefocus.com/future-technology/future-technology-22-ideas-about-to- change-our-world/>.
• Heath, N 2018, What is AI? Everything you need to know about Artificial Intelligence, ZDNet, ZDNet, viewed 26 October 2020, <https://www.zdnet.com/article/what-is-ai-everything- you-need-to-know-about-artificial-intelligence/>.
• Hindy, J 2014, 15 New Technologies That Will Become Mainstream Soon, Lifehack, Lifehack, viewed 27 October 2020, <https://www.lifehack.org/articles/technology/15-new-
technologies-that-will-become-mainstream-soon.html>.
• HP Carly Fiorina Speech: Information: the currency of the digital age - Oracle OpenWorld 2012, Hp.com, viewed 26 October 2020,
<http://www.hp.com/hpinfo/execteam/speeches/fiorina/04openworld.html>.
• Marr, B 2019, ‘How Much Data Do We Create Every Day? The Mind-Blowing Stats Everyone Should Read’, Forbes, 5 September, viewed 26 October 2020,
<https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create- every-day-the-mind-blowing-stats-everyone-should-read/#618cd1d160ba>.
• VAPAR | Automate, Standardise & Centralise Pipe Condition Assessment 2020, VAPAR, viewed 26 October 2020, <https://vapar.co/>.
