There are two main methods for project planning: waterfall and agile. Waterfall is a traditional approach in which the project is first planned in its entirety and then built from that planning. Agile is a more innovative approach in which small, multi-functional teams deliver incremental products, which eventually grow into the full solution.
The short delivery cycles in agile reduce the risk of misaligned delivery and force teams to build with an eye to modularity and flexibility. In addition, the focus within agile on cross-functional teams helps ensure that analytic initiatives are undergirded by necessary data, infrastructure and programming support, and are continuously re-aligning with business goals and insights.
Agile project management is gaining popularity, particularly in technology companies. It is especially helpful in big data analytics projects, where challenges and benefits are less clearly understood and more difficult to anticipate, and where underlying tools and technology are changing rapidly.
Agile is designed for innovation, and it pairs well with big data projects, which themselves focus on agility and innovation.
In IT and analytics, agile methodologies are most often carried out using the framework called scrum (think rugby), which is employed in one of its forms at least five times as often as other agile frameworks.69 Even departments outside of IT work with scrum, and it is not uncommon to see HR or marketing teams
standing around scrum planning boards.
Agile methodologies are being embraced even at the highest levels within corporations, with companies embracing the principles of ‘fail fast’ and ‘nail it, then scale it.’ In the context of their recent digital initiative, General Electric (GE) has been developing what they call a ‘culture of simplification’: fewer layers, fewer processes and fewer decision points. They’ve adapted lean principles in what they call ‘Fast Works.’ They have broken away from many traditional annual operating cycles. As their (former) CEO Jeff Immelt said, ‘in the digital age, sitting down once a year to do anything is weird; it’s just
bizarre.’70
Keep in mind
Business feedback is key to making agile work. Work in short delivery cycles and solicit frequent feedback from your stakeholders.
It’s important to emphasize this once more. Don’t try to solve your full problem at once. Don’t try to assemble a complete, cleaned data set before starting your analysis. Spend two weeks building a 60 per cent solution using 10 per cent of the data, then get feedback on the results. Spend the next two weeks making a few improvements and then collect more feedback.
There are several advantages to such a short-cycled approach over trying to build the solution in one shot. First, you’ll demonstrate to your stakeholders after just a few days that you have indeed been working and that you are still alive. Second, if you happen to be headed down the wrong path with your analysis, either because the data doesn’t mean what you thought or because the problem wasn’t communicated clearly, then you can correct the misunderstanding before wasting more time. Third, it’s all too likely that the business priorities will change before you’ve completed the full project. Your short delivery cycles will allow you to cash in on the deliverable while it is still appreciated, before you start work on a now more relevant project.
Keep your analytics agile by following the following basic principles:
Start with a minimum viable product (MVP). Make it cheap and quick, because once you get feedback from your initial results, it will almost certainly need to change.
Learn and change quickly. Get feedback from end users as often as you can.
Gain their trust and support by listening closely to their input.
Build modular components that are fault tolerant. Consider a microservice architecture, where components are built independently and communicate through a well-defined, lightweight process. This architecture will have some cost in speed and efficiency but will improve fault tolerance and usability.
There are many books, certifications and trainings on the topics of lean, agile and scrum, as well as at least one book written entirely about lean analytics. I touch on the topics here only briefly, to emphasize the importance of working in an agile manner to effectively derive business value from big data.
Takeaways
Analytics can be divided into four levels of increasing complexity, but even
basic analytics can be extremely valuable. Start by getting your data in order and doing some spreadsheet analysis.
A well-designed graph can give insights you won’t get from a table.
When you have a choice of analytic models, use the simplest and most intuitive.
AI and machine learning have promises and pitfalls. Weigh the value, the risks, the costs and the alternatives.
Analytics projects are best carried out using agile approaches.
Leverage existing tools and technologies as far as possible, but consider the factors discussed above before making your choices.
Ask yourself
Which of the four types of analytics does your organization utilize
effectively? For those you are not already utilizing, are you hindered by lack of skills, use-cases or priority?
Think of times when an insight jumped out at you from a graph. What data are you regularly reviewing in tables that might benefit from a graphical representation?
Where in your organization are you using analytic models but not yet incorporating business intuition within the modelling process? Are you satisfied with the output of those models? You may need to push to bring more business insight into those models.
How frequently do you review deliverables from your analytics projects?
Which end users are testing out the intermediate deliverables for those projects?