Reducing Risk in the Petroleum
Industry
Machine Data and Human Intelligence
Reducing Risk in the Petroleum Industry
by Naveen Viswanath
Copyright © 2016 O’Reilly Media Inc. All rights reserved. Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or
corporate@oreilly.com. Editor: Tim McGovern
Production Editor: Shiny Kalapurakkel Copyeditor: Gillian McGarvey
Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Panzer
Revision History for the First Edition
2016-08-11: First Release
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Reducing Risk in the Petroleum Industry, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the
intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
Reducing Risk in the Petroleum
Industry: Machine Data and
Introduction
To the buzzword-weary, Big Data has become the latest in the infinite series of technologies that “change the world as we know it.” But amidst the hype, there is an epochal shift: the current exponential growth in data is
unprecedented and is not showing any signs of slowing down.
Compared to the short timelines of technology startups, the long history of the petroleum industry provides stark examples to illustrate this change. Seismic research happens early in the exploration and extraction stages. In 1990, one square kilometer yielded 300 megabytes of seismic data. In 2015, this was 10 petabytes — 33 million times more, according to Satyam
Priyadarshy, chief data scientist at Halliburton. First principles, intuition, and manual arts are overwhelmed by this volume and variety of data. Data-driven models, however, can derive immense value from this data flood. This report gathers highlights from Strata+Hadoop World conferences that showcase the use of data science to minimize risk in the petroleum industry.
In the short term, data can be used to mitigate operational risk. Given good data, machine learning can be used to optimize well completion parameters such as the amount and type of proppant used. Ben Hamner, chief technology officer at the data science startup, Kaggle, says these are the biggest drivers of well cost and the biggest expense when drilling the well. They also have a proportionate impact on how much a well can produce. Using completion parameters from machine learning on one well, the gain after costs was $700,000.
Operational Risk
The spectrum of available data can be used to answer a variety of questions. High-quality input data is required for most analyses, and the output data can address different realms, like current operational risk and longer-term
organizational challenges.
Exploration
Exploration is an exciting time during which there can be immense payback for making the correct choices. The right data and the information that results from this data processing can be valuable tools in the upstream arsenal.
Domain expertise on data sources
The oil and gas industry has been a prolific user of data for a long time, as Chevron’s Martin Waterhouse points out — and just as keeping oil flowing is a complex operation running across continents, keeping information flowing can be just as much of a challenge. Big oil are large companies, but they are not monoliths. They are conglomerations of organizations which can be considered large companies on their own. The culture of the people, the role data plays, and the time that the data is retained can be very different in each organization. It can take years to figure out whom to ask questions, where things are done, and how the company functions. Connecting domain
expertise with the latest in modeling and predictive analytics is as important as implementing those models, but the payoff is worth it.
In unconventional production (shale), well production is highly correlated with location. Machine learning can help determine where to acquire acreage. The input data can come from:
Geology
Core samples are rich and accurate, but also rare and very expensive
Drilling and completion
Amount of proppant and fluid, number of stages, and injection rate
Production
Publicly available in the US; varies by state
automated approaches is used to correct data when possible and reject otherwise. Hamner estimates, 95% of the effort in tackling predictive
problems in the industry lies in deeply understanding data sources and how they fit into the business use case. A related challenge is how to expose results to key decision-makers.
Integrating disparate data sources
A variety of sources can contribute to the data repository. This can range from automated high-sample-rate sensors to a human dropping a rope in a tank every six months. They can include audio, video, handwritten notes, and text reports. The challenge is to convert these different sample rates,
accuracies, accessibilities, costs, and difficulties into a validated, usable form. In a case that (like many others) cuts across both data varieties and domains, André Karpištšenko and his team at Marinexplore Inc. (now PlanetOS) have been working to ease the flow and increase the utility of ocean-related data. In many parts of the world, risk is synonymous with weather. The advent of inexpensive, robust drones powered by wave and solar energy has made available data that was once impossible to gather (in the eye of a storm) or too expensive (across the Pacific), which can keep us better informed of upcoming weather. This can directly impact planning locations for offshore drilling platforms and shipping routes for oil tankers.
Risk is also equated to uncertainty. In the ocean, no two days are the same and attributes like wind, waves, ocean currents, temperature, and pressure vary depending on location and time. A prompt, easily accessible system is more valuable than one with long data collection and processing times, when delays can render information useless.
When data is democratized, the experts are not isolated anymore. There are no long timelines to process and visualize data. Data streams from sensors, models, and simulations are available to everyone. This can even involve
Drilling and Production
Over the life of a well, the risk-return equation can be optimized with
predictive maintenance. Predictive maintenance, as understood by data folk, uses predictive analytics to understand causation and correlation with
millions or even billions of records as a matter of course, and formulates predictions about machine failure in order to proactively service devices instead of relying on isolated inspections. In a compressor, monitoring oil temperatures and vibrations in real time offers direct cost advantages by maximizing utility (service too soon) and minimizing downtime (service too late) by operating until the desired point on the PF curve (potential failure, functional failure). This, says Mainka, can result in big numbers. Even a 0.1% reduction in maintenance costs can translate into millions of dollars saved. For example, in Europe, maintenance cost is estimated to be 450 billion euros. Of this, 300 billion could be addressed by maintenance improvements and 70 billon is lost due to ineffective maintenance. The methods chosen for data processing should be able to handle the
characteristics of the incoming data. Priyadarshy highlights the characteristics of different types of upstream data. During seismic studies, the volume of data is very large, but the velocity is slow and the data does not have to be analyzed in real time. The value is significant because if you wrongly choose the drilling location of a well, it could cost you a few hundred million dollars. The complementary example is during drilling. The volume of data is much smaller compared to seismic studies, but the velocity is faster, and sometimes you have to analyze the data in real time. If predictive models fail, it can be expensive (when a drill bit gets stuck, for example). The value of real-time data in any particular case is significant but not as high as well location. Sensors in real time
sophisticated companies (Fortune 100, Fortune 500).
Sensor maturity translates to lower cost and improved robustness. Petabytes of data are now collected by millions of sensors. The challenge is how to use this fast enough so that value is not lost due to collection and processing. Karpištšenko shares an example from the early life of Marinexplore: once buoy data was collected and analyzed, it took a customer three months to make a decision. Given that the ocean is highly dynamic, this delay seems to negate the usefulness of the information. Marinexplore’s platform can show measurements from sensors and data from models and simulations (such as daily sea temperatures) in seconds instead of months or years.
Data methods
A few data science methods can be applied verbatim, whereas others require tailoring to suit the petroleum industry. While explaining use cases, the speakers offer a glimpse into their instantiation of this world.
Asset-intensive industries are especially interested in maximizing asset
productivity. Mainka describes how either the end user or the manufacturer is involved depending on whether the assets are owned or rented. By looking at billions of records, models can create rules and back-calculate possible root causes of failure. Anomalies can be either good or bad. If good, try to repeat it. If bad, try to avoid it. Multiple rules can be chained together to classify scenarios. In each case, by monitoring future performance, the system can be iteratively improved. When an impending failure is detected, from the
perspective of the manufacturer, the next step could be to offer preventative maintenance service for a positive customer experience. The risk of
unscheduled maintenance and associated costs can thus be reduced.
Organizations that generate the majority of maintenance work orders from preventative and predictive inspections and use sophisticated reliability-based maintenance procedures and tools to increase asset availability have a 27% lower unplanned downtime without any increase in service and maintenance cost.
following model evaluation strategies as being useful in picking parameters for deeper study or for selecting between models:
Random cross-validation
Test performance with randomly withheld wells. This could be biased when correlation exists between wells.
Time-based validation
Use results from existing wells to predict new well performance. This can correct for (1) but is harder in newer plays with not as many wells.
Spatial validation
Test performance with held-out geographic areas. This corrects for spatial biases and is applicable in newer plays. This helps quantify acreage evaluation models.
In oil and gas, drilling is based on physics and first principles, with data crunching to generate metrics and evaluate key performance indicators (KPIs). However, using the volumes of data already stored, the goal is to learn, innovate, and move to holistic data-driven analytics in real time.
Priyadarshy details the three aspects that make now seem like the right time:
Hardware
From a single processor to distributed grid processing
Data
From local files to flexible, nonrelational distributed file systems
Applications
Long-Term Risk
Practical
Institutional
It is not sufficient to pick the right problem and solve it using good data. It is equally important to share the results among the target population, ensure that the acquired knowledge does not perish, and future decisions are based on what was learned during a given study. Any of these can be more challenging than the others, for unexpected reasons.
As a model of the integration of machine learning with human expertise in materials research, Kai Trepte, lead engineer at Harvard’s clean energy project, explains how building blocks are mixed in computer models and their properties analyzed. The data from this analysis is mined for promising candidates, speeding up the discovery process. In addition, constraints for manufacturing and distribution are added to speed up real-world usage. Combinatorially, 26 promising fragments (from research at Stanford University) resulted in 10 million molecules. Help from human
experimentalists and theorists, and data mining and machine learning reduced this number to 2.3 million molecules that required further study. These 2.3 million molecules required 150 million calculations, generating 400 terabytes of data. From that, the yield was about 0.5%.
The compute time for such simulations is very large, so they used an existing open source framework IBM World Computing Grid and Berkeley Open Infrastructure for Network Computing (BOINC) where volunteers donate processing on their devices. With 600,000 volunteers donating 22,000 CPU years, it was equivalent to a 170,000-core supercomputer. This is orders of magnitude higher than what a single, well-funded research team could afford. It is difficult to fathom how long physically making these millions of
different molecules and testing them would take. Humans and machines together made this study possible.
But as a general statement in research, whether academic or industrial, there is little funding for data persistence (especially when there aren’t publishable results). In short, most of the data collected during research is lost. By
accessibility and the infrastructure for data sharing within this field. In addition to persistence of knowledge, a combination of automated and manual approaches must be used to correct data or, if uncorrectable, to remove them from consideration. Many techniques used by Hamner at
Kaggle are based on expertise in data-recording and data-reporting practices, as well as experience with the types of failures that occur in the field. A typical unconventional shale well may be online only for a short time during the first month. It quickly spikes to peak production and then declines with most of the oil extracted within 6–12 months.
In Texas, public reporting of production data is at lease-level, not at well-level. So there can be data corruption where the entire lease’s production is wrongly allocated to one well. This can show up as spikes in production as new wells come online. The risk in not correcting this is that we could wrongly deduce that this is an enormously productive well, which could throw off machine-learning models and related decisions. Another problem is if production was affected because of issues not related to well potential, such as well downtime, or choked production, allocation, or or transportation. The number of well data points can be small — 100 to 10,000 — but the cost per data point can reach $10–15 million. So, this requires different data
quality control than a Facebook news feed, for instance, where the number of data points is higher but the cost per data point is much lower. One method that has proven to work well in well-productivity prediction is Bayesian
additive regression trees (BART). This outputs not just a point estimate of the prediction but the full probability distribution that the model learned.
The challenges of personal behavior aren’t limited to customer behavior analytics: they point within the company as well. One challenge is to get executives to fund longer-term initiatives. For instance, it doesn’t help to only store a week’s worth of logs to study long-flow data anomalies. About a
year’s worth will be good. Even if the current constraint is cost,
understanding and solving this problem has the potential for significant future returns.
Mired in quarterly financial reports during tough economic times, it might be easier to quantify savings from continuous lowest cost outsourcing and
offshoring. But longer-term effects are much harder to quantify and may be unrecoverable. Organizations can lose future data experts due to fewer opportunities for peer expertise exchange because things are done in pieces, probably at different locations.
In lean times, Chevron’s Waterhouse has a few ideas for data specialists: Seek alternate areas to add value.
Practice internally and build communities. Encourage outreach.
Conclusion
Bibliography
Hamner, Ben. “Machine Learning for Oil Exploration.” Strata +
Hadoop World in San Jose 2015. February 17, 2015. Accessed August 4, 2016. http://oreil.ly/2aUHBmZ.
Karpištšenko, André. “The Ocean’s Big Data Platform.” Strata 2014. February 11, 2014. Accessed August 4, 2016. http://oreil.ly/2aphxeN. Mainka, Oliver. “Improving Business Operations with Predictive Maintenance and Service.” Strata + Hadoop World in San Jose 2015. February 17, 2015. Accessed August 4, 2016. http://oreil.ly/2aEc9oj. Priyadarshy, Satyam. “Leveraging Big Data and Data Science in Upstream Oil and Gas Industry.” Strata + Hadoop World in San Jose 2015. February 17, 2015. Accessed August 4, 2016.
http://oreil.ly/2aRBzCz.
Trepte, Kai. “Harvard’s Clean Energy Project: Big Data Maps to
Renewable Energy.” Strata 2014. February 11, 2014. Accessed August 4, 2016. http://oreil.ly/2aUxcqq.
About the Author
Reducing Risk in the Petroleum Industry: Machine Data and Human Intelligence
Introduction
Operational Risk Exploration
Drilling and Production
Long-Term Risk Practical
Institutional
Conclusion
