Introduction to Data Warehousing

(1)

Introduction to Data Warehousing

(2)

Why Data Warehouse?

(3)

Scenario 1

(4)

Scenario 1 : ABC Pvt Ltd.

Mumbai

Delhi

Chennai

Banglore

Sales Manager Sales per item type per branch

(5)

Solution 1:ABC Pvt Ltd.

 Extract sales information from each database.

(6)

Solution 1:ABC Pvt Ltd.

Mumbai

Delhi

Chennai

Banglore

Data Warehouse

Sales Manager Query &

Analysis tools

(7)

Scenario 2

One Stop Shopping Super Market has huge

operational database.Whenever Executives wants some report the OLTP system becomes

(8)

Scenario 2 : One Stop Shopping

Operational Database Data Entry Operator

Data Entry Operator

Management Wait

(9)

Solution 2

 Extract data needed for analysis from operational database.

 Store it in warehouse.

 Refresh warehouse at regular interval so that it contains up to date information for analysis.

(10)

Solution 2

Operational database

Data Warehouse Extract

data

Data Entry Operator Data Entry Operator

Manager Report

(11)

Scenario 3

Cakes & Cookies is a small,new company.President of the company wants his company should grow.He needs information so that he can make correct

(12)

Solution 3



_{Improve the quality of data before loading it}

into the warehouse.



_{Perform data cleaning and transformation}

before loading the data.



_{Use query analysis tools to support adhoc}

(13)

Solution 3

Query and Analysis

tool President

Expansio n

Improvemen t

sales

time Data

(14)

What is Data Warehouse??

(15)

Inmons’s definition

A data warehouse is -subject-oriented, -integrated,

-time-variant, -nonvolatile

(16)

Subject-oriented

 Data warehouse is organized around subjects such as sales,product,customer.

 _{It focuses on modeling and analysis of data for}

decision makers.

 _{Excludes data not useful in decision support}

(17)

Integration

 Data Warehouse is constructed by integrating multiple heterogeneous sources.

 _{Data Preprocessing are applied to ensure}

consistency.

RDBMS

Legacy System

Data Warehouse

(18)

Integration



_{In terms of data.}

– _{encoding structures.}

– _{Measurement of}

attributes.

– _{physical attribute.}

of data

– _{naming conventions.}

– _{Data type format}

(19)

Time-variant

 Provides information from historical perspective e.g. past 5-10 years

 _{Every key structure contains either implicitly or}

(20)

Nonvolatile

 Data once recorded cannot be updated.

 Data warehouse requires two operations in data accessing

– _{Initial loading of data} – _{Access of data}

load

(21)

Operational v/s Information System

Complex query Short ,simple transaction

Unit of work

Summarized, multidimensional Detailed,flat relational

View

Subject oriented Application oriented

DB design

Mostly read Read/write

Access

Historical Current

Data

Decision support Day to day operation

Function

Knowledge workers Clerk,DBA,database

professional User

Analysis Transaction

Orientation

Informational processing Operational processing

Characteristics

Information Operational

(22)

Operational v/s Information System

High flexibility,end-user autonomy

High performance,high availability

Priority

Query througput Transaction throughput

Metric

Information Operational

Features

100 GB to TB 100MB to GB

DB size

hundreds thousands

Number of users

millions tens

Number of records accessed

Information out Data in

(23)

Data Warehousing Architecture

Extract Transform

Load Refresh

Serve

External Sources

Operational Dbs

Analysis

Query/Reporting

Data Mining

Monitoring & Administration

Metadata Repository

DATA SOURCES TOOLS

DATA MARTS

OLAP Servers

(24)

Data Warehouse Architecture

 Data Warehouse server

– _{almost always a relational DBMS,rarely flat}

files

 OLAP servers

– _{to support and operate on multi-dimensional}

data structures

 Clients

– _{Query and reporting tools} – _{Analysis tools}

(25)

Data Warehouse Schema



_{Star Schema}

(26)

Star Schema



_{A single,large and central fact table and one}

table for each dimension.



_{Every fact points to one tuple in each of the}

dimensions and has additional attributes.

(27)

Star Schema (contd..)‏

Price Units

Period Key Product Key Store Key

Store Dimension _{Time Dimension}

Product Dimension

Fact Table

Benefits: Easy to understand, easy to define hierarchies, reduces

no. of physical joins.

Store Key

Region State City

Store Name

Month Quarter Year

Period Key

(28)

SnowFlake Schema



_{Variant of star schema model.}



_{A single,large and central fact table and one}

or more tables for each dimension.



_{Dimension tables are normalized i.e. split}

(29)

SnowFlake Schema (contd..)‏

Price Units

Time Dimension

Product Dimension

Fact Table

Store Key

City Key Store Name

Month Quarter Year

Period Key

Product Desc Product Key State

City Key

Region City

City Dimension Store Dimension

(30)

Fact Constellation



_{Multiple fact tables share dimension tables.}



_{This schema is viewed as collection of stars}

hence called galaxy schema or fact

constellation.



_{Sophisticated application requires such}

(31)

Fact Constellation (contd..)‏

Price Units

Store Dimension Product

Dimension Sales

Fact Table

Store Key

Region State City

Store Name Product Desc Product Key

Shipper Key

Price Units

(32)

Building Data Warehouse



_{Data Selection}



_{Data Preprocessing}

– _{Fill missing values}

– _{Remove inconsistency}



_{Data Transformation & Integration}



_{Data Loading}

(33)

Case Study

 Afco Foods & Beverages is a new company which produces dairy,bread and meat products with

production unit located at Baroda.

 _{There products are sold in North,North West and}

Western region of India.

 _{They have sales units at Mumbai, Pune ,}

Ahemdabad ,Delhi and Baroda.

(34)

Sales Information

Report: The number of units sold. 113

Report: The number of units sold over time

25 33

41 14

April March

(35)

Sales Information

Report : The number of items sold for each product with time

21 25

8 Swiss Rolls

8

Wheat Bread

Apr Mar

Feb Jan

Product

T

im

(36)

Sales Information

Report: The number of items sold in each City for each product with time

15 9

4 Swiss Rolls

8 3

Cheese

7 3

Wheat Bread Pune

6 16

4 Swiss Rolls

6

(37)

Sales Information

Report: The number of items sold and income in each region for each product with time.

Apr Swiss Rolls

8 3

Cheese

7 3

Wheat Bread Pune

6 16

4 Swiss Rolls

6

Wheat Bread Mumbai

U U

(38)

Sales Measures & Dimensions



_{Measure – Units sold, Amount.}

(39)

Sales Data Warehouse Model

42.40 16

February Swiss Rolls

Mumbai Wheat Bread

Pune Wheat Bread

Mumbai

(40)

Sales Data Warehouse Model

(41)

Sales Data Warehouse Model

Product Dimension Tables

2 Coconut Cookies

288

1 Swiss Rolls

590

1 Wheat Bread

(42)

Sales Data Warehouse Model

Region Dimension Table

India NorthWest

Pune 2

India West

Mumbai 1

Country Region

(43)

Sales Data Warehouse Model

Sales Fact

Region

(44)

Online Analysis Processing(OLAP)‏

 _{It enables analysts, managers and executives to gain}

insight into data through fast, consistent, interactive access to a wide variety of possible views of information that has been transformed from raw data to reflect the real

dimensionality of the enterprise as understood by the user.

Data Warehouse

Time Product

R

egio

(45)

OLAP Cube

7.44 3

March Wheat Bread

Mumbai

7.44 3

Qtr1 Wheat Bread

Mumbai

32.24 13

All Wheat Bread

Mumbai

98.49 38

All White Bread

(46)

OLAP Operations

Drill Down

Time

R

egi

on

Product

Category e.g Electrical Appliance Sub Category e.g Kitchen

(47)

OLAP Operations

Drill Up

Time

R

egi

on

Product

Category e.g Electrical Appliance Sub Category e.g Kitchen

(48)

OLAP Operations

Slice and Dice

Time

R

egi

on

Product

Product=Toaster

Time

R

egi

(49)

OLAP Operations

Pivot

Time

R

egi

on

Product

Region

T

im

e

(50)

OLAP Server

 An OLAP Server is a high capacity,multi user data manipulation engine specifically designed to

support and operate on multi-dimensional data structure.

 OLAP server available are

(51)

Presentation

Time

R

egi

o

n

Product

Report

(52)

Data Warehousing includes

 Build Data Warehouse

 Online analysis processing(OLAP).

 Presentation.

RDBMS

Flat File

Presentation Cleaning ,Selection &

Integration

(53)

Need for Data Warehousing

 Industry has huge amount of operational data

 Knowledge worker wants to turn this data into useful information.

(54)

Need for Data Warehousing (contd..)‏

 It is a platform for consolidated historical data for analysis.

 _{It stores data of good quality so that knowledge}

(55)

Need for Data Warehousing (contd..)‏

 From business perspective

-it is latest marketing weapon

-helps to keep customers by learning more about their needs .

(56)

Data Warehousing Tools

 Data Warehouse

– _{SQL Server 2000 DTS}

– _{Oracle 8i Warehouse Builder}  _{OLAP tools}

– _{SQL Server Analysis Services} – _{Oracle Express Server}

 Reporting tools

(57)

References

 _{Building Data Warehouse by Inmon}

 _{Data Mining:Concepts and Techniques by Han,Kamber.}  www.dwinfocenter.org

(58)