The technological aspect of the BPI reporting project should include a standard set of front-end tools that can combine quantitative, graphical, and narrative informa- tion. These should permit flexible, ad hoc analyses that reduce the time the finance staff spends on low value-added work. You may find that every time the CFO has to make a presentation to the board, half the finance department winds up scram- bling around trying to collect and rekey data from P&L statements into PowerPoint slides, or obsessing over font colors and sizes. A common reporting platform will deliver the basic data sets in a readable format that can, in turn, be embedded di- rectly into popular office applications such as spreadsheets or presentation slides.
However, the technical end of your BPI project is likely to concentrate on the behind- the-scenes applications, such as relational and OLAP databases, as well as the appli- cations and routines these databases will use to receive and summarize data from transactional systems.
An assessment of the BPI technology enablers begins with an inventory of operational and financial systems, such as subledgers and CRM systems, which will serve as key data sources. This will require that you find and map relevant data from these systems to a common reporting structure, using a common terminology. There are four technology enablers that are essential for reporting BPI, which are described in the following subsections.
ANALYTICS APPLICATIONS AND THE FINANCIAL DATA WAREHOUSE
As of this writing, the market for financial analytics software is highly competitive, so you are likely to find a number of viable alternatives in the marketplace. The system you choose should allow integrations to multiple general ledgers, as well as sources of operational data such as payroll, CRM, and subledgers. In many organizations that lack a centralized reporting application, the logic and layout behind financial reports resides on separate files, such as spreadsheets and personal database applications (such as Microsoft Access or FoxPro), and can be scattered throughout the organi- zation. You can improve the reporting process by establishing a standard rollup struc- ture and naming conventions for accounts, product, and cost centers that facilitate a rapid consolidation of financial data. Whether you buy an off-the-shelf application or develop one internally, establishing an analytics platform where core financial
reporting tasks can be carried out centrally will enable you to make dramatic im- provements in reporting efficiency and accomplish several process improvements:
• Do away with spreadsheets and flat-file databases (such as Access or FoxPro) except as presentation and analysis tools.
• Carry out consolidation tasks for the whole organization.
• Calculate and store key figures, such as KPIs or Balanced Scorecard.
• Compare financial and statistical results across business units or divisions.
• Reduce disagreement on performance measures, as well as the manual effort involved in calculating and recording them.
• Enforce standardization of data and the use of a common language throughout the organization. For example, make sure that the identifier used for Federal Express in one division (whether it is a customer, a vendor, or both) is the same used for Federal Express in a different division.
• Give users a single source that makes the task of finding, combining, and slic- ing and dicing financial or statistical data less cumbersome.
Analytics applications allow users to summarize, analyze, and present data from a shared database (or collection of databases) which is, essentially, a financial data warehouse. You (or your application vendor) will probably decide to use a combi- nation of relational and OLAP databases as the physical structures of the data ware- house. The relational and OLAP models represent very different approaches to database design, and the nontechnical reader may find a brief overview of differ- ences between these two models worthwhile.
The relational database model seeks to reduce data redundancy as much as pos- sible. This is why online transaction processing (OLTP) applications such as CRM, general ledgers, and subledgers use the relational model. Less redundancy within database tables means greater speed when writing new records to the database. If you plan to use an analytics application for data entry (in order to collect and consol- idate budgets), it will probably conform to the relational model to some extent.
The task of organizing the data in the relational database model, also known as normalization, seeks to eliminate redundant data by separating related sets of attrib- utesinto separate tables (where an attribute corresponds to a column in a table). Thus, in designing a relational sales order database, you would separate customer records and sales order records into distinct tables to avoid repeating customer attributes such as customer ID, name, address, and so on for each sales order.
Of course, normalization isn’t without its consequences. The term performance hitis common in database parlance, and refers to the increase in access times that ac- companies any splitting of data into different logical tables. This happens because the task of joining data from two separate tables adds to the complexity of a query. In other words, when you run reports on a relational database, your query against that database has to undo the work of normalization. While normalization is good for storing data to a database, it has an adverse impact on query performance.
The online analytical processing (OLAP) model is geared toward summarizing and reporting transactional data as fast as possible. Since they are intended only for
reporting purposes, OLAP databases are deliberately denormalizedso that there is less need to join multiple tables in order to assemble a meaningful view of the data. Another essential feature of OLAP databases is aggregation. This is a means of precalculating results for more commonly used queries—especially high-level summaries—in order to improve access times. OLAP databases and OLAP browsers are designed to facilitate the rapid development of ad hoc reports that show a quan- titative measure (e.g., revenue or unit sales) along two or more dimensions (e.g., time, business unit, product, or geography).
Corporate environments where reporting needs constantly change can benefit from OLAP, to help meet the growing demand for self-service among the data consumers within these organizations. Again, we emphasize that your consolidation system may employ both relational and OLAP database platforms to accomplish dif- ferent tasks, according to your budgeting, reporting, and analysis requirements. While a few OLAP database platforms have write-backcapabilities that allow users to up- date numbers manually in an OLAP cube, they generally aren’t adequate for all but the most basic budgeting requirements.
As of this writing, several of the leading database brands, such as Microsoft SQL Server and Oracle, offer OLAP integrations to their relational database management system (RDBMS) applications. A relational database therefore can act as a staging area for a series of OLAP cubes that serve specific reporting needs. For example, your consolidation database may store all the rollup structures and elimination entries needed for balance sheets, P&Ls, and cash flow statements. This database might in turn feed data to a balance sheet cube, an income statement cube, and a cash flow cube.
Adding OLAP capabilities enables the following process improvements:
• Allows direct integrations to popular spreadsheet applications, which means less rekeying
• Enables users to generate detailed views on their own, rather than waiting on IT or finance departments to spec out and design more detailed reports.
• Serves as an excellent tool for understanding variances, especially if drill-through to transaction detail is permitted (see Chapter 18).
• Helps users to better understand the relationship of organizational structures such as divisions and product lines to the numbers, and to see the evolution of these relationships over time.
Most reporting needs that aren’t part of the half-dozen or so standard financial re- ports (such as the balance sheet, income statement, and various operating statements) can be answered with an OLAP solution. OLAP relieves the burden on overtaxed IT departments and gives data consumers more freedom of choice. It also lightens the burden on finance and IT departments by reducing the number of financial reports those staffs need to maintain and check for accuracy—they may still have to do reg- ular quality assurance on, say, one or two OLAP cubes versus 100 or 200 reports, but this is progress! Finally, it gives greater freedom to end users by letting them set up specialized views that accommodate their departments reporting needs, save these views, and use them again in the future until circumstances change (see Exhibit 19.1).
To summarize, the rule of thumb is to start with a single, somewhat normalized data warehouse where detailed and integrated (i.e., from multiple sources) corporate financial data will be stored. This data warehouse will serve a broad range of finan- cial reporting needs, will have no bias toward any one report or user, and will rep- resent the ultimate authority on financial results. The data warehouse can in turn be used to feed any number of data marts that are optimized for specific views of that data, such as balance sheet and P&L reports. Data marts are more likely to conform to the OLAP model and should be tailored to the reporting needs of specific de- partments, product lines, or divisions.
EXTRACT, TRANSFORM, AND LOAD: COMBINING DATA FROM DIVERSE SYSTEMS
Automating the task of extracting, formatting, and cleansing data from diverse sys- tems throughout the company is essential to improving the reporting process, espe- cially if process improvement for you entails the addition of new nonfinancial metrics into the reporting mix. Even if your reporting requirements are entirely financial, the prospect of a merger or acquisition with another organization that uses different financial systems may make this a crucial consideration as you seek to consolidate financial data from different accounting packages.
Extraction, transformation, and loading (ETL) is a generic term in the software industry that refers to any of a variety of products or methods that enable the trans- portation of data from one system or database to another. The terms extractandload are straightforward, and refer to the tasks of copying data from a source system into a destination system. However, since the systems or databases in question are likely EXHIBIT 19.1 Sample OLAP Report
Sales ($) by Region and Time
2000 2001 2002
Canada 6,778,956 7,635,678 8,399,246
Central 21,869,884 31,757,325 34,933,058
East 2,001,388 7,991,848 8,791,033
Northeast 8,938,288 9,832,116 8,647,771
New York 3,298,261 3,628,087 3,990,896 New Jersey 2,968,435 3,265,279 3,591,806 Connecticut 2,671,592 2,938,751 3,232,626 Southeast 10,770,441 30,805,863 33,886,449 Southwest 5,040,712 18,877,927 20,765,720
West 21,209,899 55,676,723 61,244,396
Total 76,609,568 162,577,480 176,667,671
User clicks on column and row headers to drill down to a more detailed level Users can select dimensions that appear on columns and rows, allowing them to build their own ad hoc reports
to use differing coding and formatting conventions for their data, an intermediate step between the extractionand the loadis normally required. Thus, the term transforma- tionin this context refers to the series of logical rules or tests to which data are sub- jected as they are moved from source to destination. These rules will ensure that the data meet your reporting requirements. During transformation, all necessary cor- rections to data that don’t meet these requirements are applied.
More than any other aspect of process improvement, ETL will require close and continual cooperation between the finance and IT departments. The IT department will of course have the necessary expertise to write the routines and develop applica- tions to extract, transform, and load data from source systems into your financial con- solidation system. Since integration efforts are probably underway for applications being used elsewhere in the company, your IT department probably already has one or more tools at its disposal that can accommodate your integration requirements.
You will want to take an inventory of available ETL tools prior to starting this project.
As of this writing, most commercial database vendors provide methods of reading and importing data from external sources.
It is useful to subdivide the task of implementing ETL into three phases:
1. Evaluating your data.
2. Defining data transformation rules.
3. Select software.
Evaluating Your Data
Start by assessing the condition and accessibility of your source data. At this stage, you pose three questions: What data do I need? Where (e.g., in which systems or files) are these data located? How good are these data? The first two questions should be understood from your meetings with your metrics team, as well as interviews with those who will be the consumers of information from your reporting processes.
Hence, your evaluation will concentrate primarily on the third question.
The data evaluation will yield a collection of information about source systems and their content that will facilitate the correct use of data, in the proper context.
Targets of data evaluation may include internal systems, such as the general ledger, CRM and manufacturing, as well as external files and databases that may contain market research, customer satisfaction data, or industry-specific statistics. This sys- tematic analysis should be done before attempting to extract data from any source system, files, or database. These sources will comprise data structures such as files, tables, and fields. Once the data structures within a source system have been identi- fied, your data evaluation should cover all of the following topics for each data struc- ture that you plan to use:
• Content. What the data actually mean from a business point of view (e.g., prod- uct codes, customer codes, demographic codes, etc.)
• Format. Examples of format are text versus numeric, size limitations, special characters allowed, and so on.
• Quality. Consistency, completeness and correctness
• Structure. Relationships to other data tables and fields
• BPI-specific uses. Mapping of values from the source system to corresponding values in the analytics/reporting system
Start the evaluation with whatever information is provided about the data in ex- ternal sources. These will include documentation that may have been provided by application vendors, as well as manuals written by employees or external consultants who helped implement the systems from which you want to gather data. You may also need to follow up by interviewing those most familiar with the design and main- tenance of your source systems. This collection of external sources about a system’s contents is typically known as metadata, or data about data.
Be aware, however, these external sources may prove to be of limited value, since ultimately the real data may have deteriorated from the specifications set forth months or years ago. Minor system fixes and modifications may have gone undoc- umented, and information about relationships between values held in different tables or fields are often not well explained. And, over time, users may have begun to use free text fields for different purposes. So the ultimate authority on the condition of your data is the data itself. Unlocking this information may require hours or even days of querying database tables in a source system, using the metadata you have already gathered as a guide, to look for undocumented relationships between database tables and fields, as well as inconsistencies.
Defining Data Transformation Rules
The transformation step in the ETL process does the job of quality assurance, mak- ing sure that the data going into your reporting/analytics system are accurate and fit for use. The evaluation stage will no doubt leave you with some hints as to the nature and extent of quality problems in the source data. If problems are widespread, you should probably opt at this point to clean and correct the data in your source systems before going any further.
Even after defects are dealt with in the historical data, errors may continue to creep in, so you’ll need to monitor the process for bad data. Errors that go undetected may skew aggregate values when the data are summed for reporting purposes. Thus, you will probably want to include a reconciliation step after each significant trans- formation task (such as code conversions and summation of numeric values). In BPI dogma, normally we caution against an excess of checking and reconciliation steps, but when automated and placed at strategic checkpoints throughout the process such reconciliation steps can save time and effort over leaving reconciliations for the end. The task of “tying out” becomes more time-consuming after the data have been summed, when it becomes trickier to trace mismatches between source and des- tination systems back to their origin. The transformation step should monitor data as it passes through specific checkpoints in the process, filter out invalid data, and for- ward nonconforming data through the proper channels for corrections.
The definition of transformation rules starts with an evaluation of users’ needs, then proceeds to the translation of the needs into logical rules and procedures that can be put in place at the system level:
1. Define standards of content, format, and completeness according to user needs.
In the evaluation stage, you assessed the current quality of transactional data.
Now you are in a position to set expectations for future data quality, as well as establish the rules that will be followed in the transformation of data. The team should agree on minimum standards that ensure that the data are complete, ac- curate, and useful to those who will base management decisions on these data.
These standards should specify essential content, optional content, tolerance levels for errors and inconsistencies, and formatting.
2. Translate standards into logical transformation rules. Here are some examples of logical rules you might use to implement the content and formatting standards you’ve set for your data. Your ETL application will apply one or more of these rules to the data that pass from your source system(s) to your reporting/analytics system:
• Value limitations. Some fields should only contain values from a specific range or list. This is usually necessary for fields that contain dates, account codes, currency codes, or product codes.
• Consistency. Which fields in each record must be filled out in order for records to be consistent and comparable? For example, whenever field A is filled out, must field B also be filled out?
• Completeness. Which fields in a record must be filled out in order for that record to provide adequate information to the user?
• Formatting. Should a field contain text, or numeric value? Do you need to add special characters such as quotes, commas, or dashes to make the data more readable to the user?
• Reconciliation. Do aggregate balances in the destination system agree with aggregate balances in the source system?
3. Establish procedures to handle errors and exceptions. Once transformation rules have been defined, the financial BPI team should agree on and document procedures for testing compliance to these rules and fixing errors. The team should also distinguish the types of errors that can be resolved at the system level from those that may require human intervention, and determine methods of routing different types of errors to the appropriate destination.
Choosing the Right Software
Probably you already do a number of ETL tasks on your own. These tasks may in- clude: copying data from a Microsoft Excel spreadsheet to a Microsoft Access data- base; running a series of queries in the Access database to clean the data and make