DATABASE DAN DATA WAREHOUSE. Pertemuan 06 2 SKS

(1)

Materi

1. Era Informasi

2. Strategi dan Peluang Yang Kompetitif 3. Database dan Database Warehouse 4. Desain Database

5. Sistem Pendukung Keputusan dan Sistem Cerdas 6. E-Commerce

(2)

DATABASE DAN DATA WAREHOUSE

Pertemuan 06

(3)

Data dalam SPK

• _{Data merupakan elemen penting}

dalam menentukan kualitas suatu

SPK.

• Data yang buruk atau tidak lengkap

menyebabkan SPK tidak mencapai

hasil yang optimal/bagus.

(4)

Preprocessing Data

• Data Warehouse :

–

_{Tempat kumpulan data yang digunakan untuk}

pengambilan keputusan, dikumpulkan dari

berbagai sumber dan biasanya terpisah dari

database organisasi/perusahaan.

• Data Mining :

–

_{Memilih data berdasarkan pola tertentu}

sehingga diperoleh relasi antar variabel dan

memiliki tingkat informasi yang lebih tinggi.

(5)

Tingkatan Data

• _Data:

– Kumpulan sesuatu, kejadian, aktivitas, transaksi yang direkam, diklasifikasikan dan disimpan namun tidak diorganisasikan untuk memberikan arti tertentu.

• Informasi:

– Data yang telah diorganisasikan sedemikian sehingga memberikan arti bagi penerimanya.

• _Knowledge:

– Data/informasi yang memberikan pemahaman,

pengalaman, pelajaran, keahlian yang berguna untuk pemecahan masalah.

(6)

Macam-macam Sumber Data

• _{Internal (dari dalam organisasi)}

• Eksternal (dari luar organisasi)

• Personal (dari tenaga ahli yang

(7)

Data Warehouse

• Definisi :

– “A data warehouse is a copy of transaction data

specifically structured for querying and reporting” (Ralph Kimball)

• Suatu database untuk pendukung keputusan yang disimpan terpisah dari database operasional suatu organisasi

• Mendukung pemrosesan informasi dengan

menyediakan platform data yang historical dan

(8)

Data Warehousing

• _{Data warehousing:}

Proses konstruksi dan penggunaan data

warehouses

• Data Warehousing berupaya mengumpulkan

data-data dari berbagai sumber data

sehingga mempunyai kualitas data yang

bagus.

• Kualitas data yang bagus sangat

mempengaruhi hasil keputusan.

(9)

Kualitas Data (Data Quality)

• Kualitas data (DQ) dapat dilihat dari 4 katergori:

– Contextual DQ: Relevansi, nilai tambah, timeliness,

kelengkapan dan jumlah data.

– Intrinsic DQ: akurasi, objektivitas, keterpercayan,

reputasi.

– Accessibility DQ: aksesibilitas, keamanan akses.

– Representation DQ: interpretabilitas, kemudahan

untuk dimengerti, representasi yang ringkas dan konsisten.

(10)

Arsitektur Data Warehouse

Arsitektur Data Warehouse 3-tier

Application Server Client Database Server Application & Database Server Client

(11)

Organisasi dan Struktur Database dalam

Data Warehouse

• Relational Databases. Berbentuk tabel.

• Hierarchical Databases. Berbentuk

pohon atau bagan organisasi.

• Network Databases. Berbentuk jaringan

kompleks.

• Struktur Lain: objec-oriented,

multimedia-based, documen-based,

intelligent databases.

(12)

Karakteristik Data Warehousing (1)

• Subject-oriented. Data diorganisasi

berdasarkan subyeknya. Mis: pelanggan

• Integrated. Data dari berbagai sumber

disimpan dalam format yang sama. Mis:

jenis kelamin : ‘L’ dan ‘P’. Maka data

yang masuk mengalami konversi.

• Time-variant.Menyediakan data dari

(13)

Karakteristik Data Warehousing (2)

• _{Nonvolatile. Tidak berubah/hilang. Data}

dalam data warehouse tidak boleh

diupdate.

• Summarized. Data operasional dapat

digabungkan ke dalam ringkasan.

• _{Not normalized. Tidak ternormalisasi.}

• Metadata. Metadata (data tentang data)

disertakan antara lain deskripsi struktur,

istilah dan definisi, kepemilikan data, dsb.

(14)

Data Warehouse vs. Operational DBMS

• OLTP (on-line transaction processing)

– Major task of traditional relational DBMS

– Day-to-day operations: purchasing, inventory, banking, manufacturing, payroll, registration, accounting, etc.

• OLAP (on-line analytical processing)

– Major task of data warehouse system – Data analysis and decision making

• Distinct features (OLTP vs. OLAP):

– User and system orientation: customer vs. market

– Data contents: current, detailed vs. historical, consolidated – Database design: ER + application vs. star + subject

– View: current, local vs. evolutionary, integrated

(15)

OLTP vs. OLAP

OLTP OLAP

users clerk, IT professional knowledge worker

function day to day operations decision support

DB design application-oriented subject-oriented

data current, up-to-date detailed, flat relational isolated

historical,

summarized, multidimensional integrated, consolidated

usage repetitive ad-hoc

access read/write

index/hash on prim. key

lots of scans

unit of work short, simple transaction complex query

# records accessed tens millions

#users thousands hundreds

DB size 100MB-GB 100GB-TB

(16)

Mengapa Memisahkan Datawarehouse

• High performance for both systems

– DBMS— tuned for OLTP: access methods, indexing, concurrency control, recovery

– Warehouse—tuned for OLAP: complex OLAP queries, multidimensional view, consolidation.

• _{Different functions and different data:}

– missing data: Decision support requires historical

data which operational DBs do not typically maintain

– data consolidation: DS requires consolidation (aggregation, summarization) of data from

heterogeneous sources

– data quality: different sources typically use inconsistent data representations, codes and formats which have to be reconciled

(17)

Model Konseptual dari Datawarehouse

• Modeling data warehouses: dimensions &

measures

– Star schema: A fact table in the middle connected to a set of dimension tables

– Snowflake schema: A refinement of star schema where some dimensional hierarchy is normalized into a set of smaller

dimension tables, forming a shape similar to snowflake – Fact constellations: Multiple fact tables share dimension

tables, viewed as a collection of stars, therefore called galaxy schema or fact constellation

(18)

Contoh dari Star Schema

time_key day day_of_the_week month quarter year time location_key street city province_or_street country location Sales Fact Table

time_key item_key branch_key location_key units_sold dollars_sold avg_sales Measures item_key item_name brand type supplier_type item branch_key branch_name branch_type branch

(19)

Contoh Snow Flake Schema

time_key day day_of_the_week month quarter year time location_key street city_key location Sales Fact Table

time_key item_key branch_key location_key units_sold dollars_sold avg_sales Measures item_key item_name brand type supplier_key item branch_key branch_name branch_type branch supplier_key supplier_type supplier city_key city province_or_street country city

(20)

Contoh Fact Constellation

time_key day day_of_the_week month quarter year time location_key street city province_or_street country location Sales Fact Table

time_key item_key branch_key location_key units_sold dollars_sold avg_sales Measures item_key item_name brand type supplier_type item branch_key branch_name branch_type branch

Shipping Fact Table time_key item_key shipper_key from_location to_location dollars_cost units_shipped shipper_key shipper_name location_key shipper_type shipper

(21)

Tiga Model Data Warehouse

• Enterprise warehouse

– Mengumpulkan semua informasi tentang subjek-subjek yang menjangkau seluruh organisasi

• Data Mart

– Sebuah subset dari corporate-wide data yang berguna untuk kelompok pengguna tertentu. Ruang lingkupnya lebih spesifik seperti marketing data mart

• Independent vs. dependent (directly from warehouse) data mart

• Virtual warehouse

– Sekumpulan view atas database-databases operational – Hanya beberapa dari view yang mungkin yang dapat

(22)

Penggunaan Data Warehouse

• Pemrosesan informasi

– supports querying, basic statistical analysis, and reporting using crosstabs, tables, charts and graphs

• Analytical processing

– multidimensional analysis of data warehouse data

– supports basic OLAP operations, slice-dice, drilling, pivoting • Data mining

– knowledge discovery dari pola-pola tersembunyi

– supports associations, constructing analytical models,

performing classification and prediction, and presenting the mining results using visualization tools.

(23)

Dari OLAP ke OLAM

(OnLine Analytical Mining)

• _{Why online analytical mining?}

– High quality of data in data warehouses

• DW contains integrated, consistent, cleaned data – Available information processing structure

surrounding data warehouses

• ODBC, OLEDB, Web accessing, service facilities, reporting and OLAP tools

– OLAP-based exploratory data analysis

• mining with drilling, dicing, pivoting, etc. – On-line selection of data mining functions

• integration and swapping of multiple mining functions, algorithms, and tasks.

(24)

Contoh Arsitektur OLAM

Data Warehouse Meta Data MDDB OLAM Engine OLAP Engine

User GUI API

Data Cube API

Database API Data cleaning Data integration Layer3 OLAP/OLAM Layer2 MDDB Layer1 Data Repository Layer4 User Interface Filtering&Integration Filtering Databases

(25)

Data Mining

• _{Istilah Data mining digunakan untuk}

mendeskripsikan penemuan pengetahuan

(knowledge) dalam database.

• _{Data mining merupakan proses yang}

menggunakan teknik, statistik, matematik,

kecerdasan buatan dan machine-learning untuk

mengekstrak dan mengidentifikasi informasi

yang berguna dan pengetahuan dari database

yang besar.

(26)

(27)

Metode Data Mining (1)

• Data mining mencoba menemukan pola

dalam data.

• _{Ada tiga jenis metode yang digunakan}

untuk indentifikasi pola tersebut:

–

_{Simple models (SQL, OLAP, keputusan}

manusia).

–

_{Intermediate models (regresi, decision}

trees, clustering).

(28)

Metode Data Mining (2),

Complex Model

• _{Text Mining:}

– Library database, e-mails, book stores, Web pages.

• _{Spatial Data Mining:}

– Geographic information systems, medical image database.

• _{Multimedia Mining:}

– Image and video/audio databases.

• Web Mining:

– Unstructured and semi-structured data – Web access pattern analysis

(29)

Metode Data Mining (3)

• Metode data mining dapat pula

dikategorikan ke dalam 2 kategori:

–

_{Hypotesis-driven. Data mining dimulai dari}

pernyataan yang kemudian diuji. Mis:

“Apakah penjualan DVD player berkaitan

dengan penjualan televisi?”

–

_{Discovery-driven. Data mining mencari pola,}

asosiasi, dan hubungan antar data yang

(30)

Tingkatan Model

• _{Beberapa model lebih baik dari model lainnya}

– Accuracy

– Understandability

• _{Model-model tersebut bervarias dari “easy to}

understand” ke tidak dapat dipahami

– Decision trees – Rule induction – Regression models – Neural Networks Lebih mudah Lebih sulit

(31)

Langkah-langkah Data Mining

• Seleksi. Memilih data.

• Preprocessing. Mengatasi masalah data

rusak atau hilang.

• Transformasi. Menyeragamkan format

data.

• Data mining. Menerapkan algoritma data

mining.

(32)

Fungsionalitas Data Mining (1)

• Karakterisasi (Characterization):

Summarization of general features of objects in

a target class. ( Concept description) Ex:

Characterize grad students in Science

• _{Diskriminasi (Discrimination):}

Comparison of general features of objects

between a target class and a contrasting class.

(Concept comparison)

Ex: Compare students in Science and students in

Arts

(33)

Fungsionalitas Data Mining (2)

• Asosiasi (Association):

Studies the frequency of items occurring

together in transactional databases.

Ex: buys(x, bread)



buys(x, milk).

• Prediksi (Prediction):

Predicts some unknown or missing attribute

values based on other information.

Ex: Forecast the sale value for next week

based on available data.

(34)

Fungsionalitas Data Mining (3)

• Klasifikasi:

– Organizes data in given classes based on attribute values. (supervised classification)

– Ex: Labeling celestial objects, medical diagnostic, …

• Clustering:

– Organizes data in classes based on attribute values. (unsupervised classification)

– Ex: group crime locations to find distribution patterns.

– Minimize inter-class similarity and maximize intra-class similarity  Similarity or dissimilarity-function ( distance)

• Outlier analysis:

– Identifies and explains exceptions (surprises) – Ex: fraud detection, rare event analysis