**ON MODELLING THE AVERAGE SCORES OF **

**NATIONAL EXAMINATION FOR PUBLIC **

**SENIOR HIGH SCHOOL IN WEST JAVA **

**KARIN AMELIA SAFITRI **

**GRADUATE SCHOOL **

**BOGOR AGRICULTURAL UNIVERSITY **
**BOGOR **

**STATEMENT OF ORIGINALITY**

The author hereby states that this thesis which is entitled On Modelling the Average Scores of National Examination for Public Senior High School in West Java is based on author’s own research with from supervisors comission and has not been published before. Information source which was either derived or cited from either published or unpublished work from another writer has been mentioned in the text and listed in the bibliography which is the final part of this thesis.

The writer hereby bestows this copyright from this research to Bogor Agricultural University.

Bogor, September 2015

*Karin Amelia Safitri *

NIM G152130511

**RINGKASAN**

KARIN AMELIA SAFITRI. Pemodelan Data Rata-Rata Ujian Nasional pada Sekolah Menengah Negeri di Jawa Barat. Dibawah bimbingan oleh ANANG KURNIA dan KHAIRIL ANWAR NOTODIPUTRO.

Pendidikan formal di Indonesia terdiri dari pedidikan pra-sekolah, Sekolah Dasar (SD), Sekolah Menengah (SMP-SMA), dan perguruan tinggi. Pemerintah Indonesia sudah berusaha keras untuk meningkatkan kualitas pendidikan di Indonesia. Peningkatan kualitas pendidikan dapat direpresentasikan dalam bentuk Ujian Nasional (UN) yang diujikan pada siswa.

Penelitian ini dilakukan dengan tujuan untuk melihat faktor apa saja yang mempengaruhi UN dan mendapatkan model yang terbaik. Data yang digunakan pada penelitian ini adalah data panel yang berupa data UN siswa SMA Negeri dari tahun 2011-2014 dan peubah bebas yang digunakan adalah Indeks Pmbangunan Manusia (IPM), Pendapatan Dometik Regional Bruto (PDRB), bahan pembelajaran, proses pembelajaran, tenaga pendidik, fasilitas sekolah, total akreditasi dan rata-rata Ujian Sekolah (US).

Skor rata-rata UN untuk kasus ini tidak bebas dan sangat bervariasi untuk setiap SMA Negeri dan setiap tahun sehingga SMA Negeri dianggap unit individu dan tahun bisa diasumsikan pengaruh tetap dan acak. Sehingga, data pada penelitian ini dimodelkan dengan Linear Mixed Model (LMM), Generalized Estimating Equation (GEE), dan model tetap. Dengan membandingkan nilai R-square, regresi dengan menambahkan pengaruh tetap sekolah dan pengaruh tetap waktu menghasilkan model terbaik dalam menjelaskan keragaman data UN dibandingkan jika dengan GEE dan LMM.

Model dengan efek tetap sekolah dan waktu dengan menambahkan peubah dummy dapat menangkap keragaman dari setiap subjek baik sekolah maupun waktu. Jika sekolah dan waktu diasumsikan sebagai pengaruh acak tidak menghasilkan model yang baik. Begitupula jika tahun mengikuti autoregressive atau AR(1) model yang dihasilkan menghasilkan R-square yang cukup tinggi tetap ada kemungkinan bahwa tahun tidak mengikuti AR(1) tetapi lebih tepatnya sekolah dan waktu diasumsikan sebagai pengaruh tetap. Semua peubah ebas memberikan kontribusinya masing-masing dalam menaikkan skor UN dimana peubah bebas tersebut adalah IPM, PDRB, bahan pembelajaran, proses pembelajaran, tenaga pendidik, fasilitas sekolah, total akreditasi dan rata-rata US.

**SUMMARY**

KARIN AMELIA SAFITRI. On Modelling the Average Scores of National Examination for Public Senior High School in West Java. Supervised by ANANG KURNIA and KHAIRIL ANWAR NOTODIPUTRO.

Formal education in Indonesia is commonly divided into stages such as
preschool, primary school (*SD*), Secondary School (*SMP-SMA*), and
universities/colleges. Indonesian government has been taking serious efforts on
how to improve the quality of education in Indonesia. The roadmap for continous
improvement of education quality can be designed based on the results of
National Examination (*UN*) taken regularly by high school students.

This research was aimed at exploring informations on how the scores of

*UN* can be linked with other explanatory variables and to find the best model. A
panel data which consists of average scores of *UN* for all public senior high
schools (*SMA Negeri*) in West Java Provinces during 2011-2014 and other
related variables such as total scores of accreditation, regional domestic product,
human development index, scores of school’s facilities and its infrastructure,
scores of school’s educators, average scores of final school exams, were used in
this research.

The average scores of *UN* in this case were dependent on variations
between high schools and time periods as well as other explanatory variables in
which the effects were either fixed or random. The data of this research was
modelled with linear mixed models, the Generalized Estimating Equation (GEE),
two fixed effects approach. This research showed that by comparing the
coefficient of determination, the regression with adding two fixed effects of time
and school provided a model of better performance than the GEE and linear
mixed model in explaining the variability of the response variable which was the
average scores of *UN*.

A model with fixed effect which used dummy variables had an ability to catch the variability of the subjects in this case the subjects were schools and time, had the highest R-squared value was because time effect is suitable if being assumed to be fixed, not fitted if assumed to be either AR(1) or random effect. All explanatory variables gave its own conribution at increasing the average scores of

*UN *which is human development index*, *regional domestic product, the average
scores of *US *and five components of accreditation (the scores of educators,
facilities, content, process, and total accreditation.

**Key words**: fixed effects, GEE, linear mixed model, national examination, *random effects. *

## © IPB Copyright, 2015

## Copyright Reserved

*Prohibited to quote some or all of this paper without including or *
*mentioning the source. Quoting is only for educational purposes, research, *
*scientific thesis, report writing, criticism, or reviewing an issue; and citations are *
*not detrimental to the interests of IPB *

*Prohibited to announce and reproduce part or all of this paper in any form *
*without permission IPB *

A Thesis submitted to the Graduate School of Bogor Agricultural University

as a partial fullfilment of the requirement for the degree of Master of Science

at

Department of Statistics

**ON MODELLING THE AVERAGE SCORES OF **

**NATIONAL EXAMINATION FOR PUBLIC **

**SENIOR HIGH SCHOOL IN WEST JAVA **

**GRADUATE SCHOOL **

**BOGOR AGRICULTURAL UNIVERSITY **
**BOGOR **

**2015 **

**ACKNOWLEDGEMENT **

All praises and gratitude is only to Allah *Subhanahu WaTa’ala *for the grace
and the blessings He never stops giving so as the writer can accomplish the
writing of this thesis which has a title “On Modelling the Average Scores of
National Examination for Public Senior High School in West Java”.

The writer also gives best thank to the most awesome advisors, Dr. Anang Kurnia, M.Si and Prof. Dr. Khairil Anwar Notodiputro, M.S who have been helping, critisizing, teaching the writer about everything related to her thesis. Big thank you to the best parents in the world for the prayers, support, encouragement and everything they have been giving to the writer. Thank you too for all the staffs of Applied Statistics Study Program, all of her friends in Statistics for their helping hands and advices.

The writer realizes that the thesis still need critics and opinion for the implementation of the next research in the future. The writer hopes this research will be useful of the scientific work.

Bogor, September 2015

**CONTENTS **

LIST OF TABLES vi

LIST OF FIGURES vi

LIST OF APPENDICES vi

1 INTRODUCTION 1

Background 1

Research Questions 2

Objectives 3

Research Benefit 3

2 LITERATURE REVIEWS 3

Panel Data 3

Fixed Effect Model 4

Linear Mixed Model 4

Estimation Method 6

Generalized Estimation Equation 7

Sampling Technique 7

3 MATERIALS AND METHODS 9

Data 9

Variables 9

Methodology 10

4 RESULT AND DISCUSSION 12

Principal Component Analysis 14

The Estimated Model 15

Overview of the Best Model 17

Prelimary Analysis of Required Sampling Technique 20

5 CONCLUSION AND RECOMMENDATION 22

Conclusion 22

Recommendation 23

REFERENCES 23

APPENDICES 25

**LIST OF TABLES **

1 Structures of working correlation matrix 7

2 The descriptive statistics of natinal examination 13 3 Coefficient correlation between response and explanatory variables

for natural science 13

4 Coefficient correlation between response and explanatory variables

for social science 13

10 Parameter estimates for the three selected components of natural

science 17

11 Parameter estimates for the first five components of social science 18 12 Parameter estimates for the two selected components of natural

science 18

13 Number of sampled cluster with different bound of errors 21

14 Variance of average of *UN* 22

**LIST OF FIGURES **

1. Scatterplot of the average score of *UN* versus years for a) natural

science b) social science in West Java 12

2. Normality plots for a) natural science b) social science 19 3. Residual versus predicted value a) natural science b) social science 20

4. Sampling scheme 20

3 Analysis of variance of the five principal component of social science 25 4 Analysis of variance of the two selected principal component of

social science 25

5 Eigen vectors for natural science 26

6 Eigen vectors for social science 26

7 Statistics descriptive of natural science 26

8 Statistics descriptive of social science 26

9 Statistics descriptive of each municipality/regency for natural science

10 Statistics descriptive of each municipality/regency for social science

in 2014 28

11 The formula of Estimated mean of three stage sampling ( stratified sampling – cluster sampling – simple random sampling) 29 12 The formula of Estimated variance of three stage sampling ( stratified

**INTRODUCTION **

**Background **

Education plays important roles for the national development. Quality
of human resources can be improved through better quality of education
continuously. In Indonesia, there are several types of education namely
formal, non-formal and informal education which are complemented each
other. The level of formal education comprises of elementary education,
high school or intermediate education, and higher education. A good quality
school must provide quality teaching, curriculum, management, and
facilities. The quality of a school can be reflected by the national
examination scores and the school examination scores, in Indonesia namely
as *UN* and *US*, respectively.
represent the quality of education mapping itself (Permendikbud No.5). *UN *

case has been urgently required for this research due to the importance of respresenting the education in Indonesia. Some previous research related to

*UN *was carried out by Sinaga (2011) analyzed correlation between the
scores of *UN, US* and school grades using canonical analysis. Mongi (2014)
also did research about relation between scores of *UN *and accreditation
standards in West Java.

Senior High School which is commonly abbreviated as *SMA* in
Indonesia acknowledged its role in preparing students to continue to the
college-level studies. Therefore, This research has analyzed the data of *UN *

scores all public *SMAs *in West Java. These *SMA*s have been referred as
observation units. This kind of data is panel data, since the data is a
combination of cross sectional data and time series data. Data was taken in
four years sequently from 2011-2014.

A linear mixed model is an appropriate approach which has beenused in this research since response variable is normally distributed. Linear mixed model is also well suited for the analysis of panel data, where each time series constitues an individual curve as a cluster. The model accommodates both fixed effects and random effect. The linear mixed model is an extention of linear model for data that are collected in groups. linear mixed model was useful in the analysis of panel data because it has accommodated the correlation of repeated measurements which are made on the same unit, the heterogeneity among subjects, and inference toward certain subjects.

2

Another approach that is used for analysis in this research is Generalized Estimating Equation (GEE). GEE is used to model the correlated data from repeated measure studies (panel data) and basically, it uses quasi-score likelihood for estimating the model parameters. GEE covers the extentions of Generalized Linear Model (GLM) to panel data. Specifically, GEE is a population averaging method that models panel data in which the response is a member of the exponential family of distribution, for example continous, binary, grouped and count data.

It is important to mention this research considered that *UN* scores can
be affected by internal factors as well as external factors. The external
factors are the factors within school, education system, and the effect of
school policies. Economics condition of each regency/municipality in West
Java where the schools are located may affect the quality of education.
Hence, it will affect the *UN* scores of the schools. This economic condition
can be measured by the gross regional income per capita (commonly
mentioned as *PDRB *in Indonesia) just like what Ahmad (2011) has stated
that the government funds allocated for education sector can affect
economic growth or *PDRB *and vice versa. According to Statistics Indonesia
(2008), the growth rate of *PDRB* also influences human development index
which is abbreviated as *IPM* in Indonesia through household expenditures
for daily primary needs including foods, medicines, and school stuffs. The
growth rate of *PDRB *may also influence *IPM *through domestic expenditure
policies made by government including priority funds for social aspects. In
contrast, *IPM *may influence *PDRB* through qualified human resources in
terms of good health and education services.

Another reason why *UN* is highlighted in this research because of the
students pass rate is no longer measured by *UN *since 2015, due to the
change of education system. This is not wise way to spend much money for
performing *UN *every year for every school. On the other hand *UN* is solely
used for measuring things that do not maximally improve the education in
Indonesia. Therefore, *UN *is better to be conducted for some schools as a
focused sample to mindmap the education instead of using the population
of all schools in Indonesia as measured target. The critical analysis is
required as preliminary task to do due to evaluate how the sampling can be
applied for *UN* implementation used for the consideration.

**Research Questions **

1. What is the best model in modelling between average scores of *UN* all
public *SMA*s for both natural science major and social science major
and other explanatory variables such as *IPM*, *PDRB*, average scores of

*US*, total accreditation scores, educators scores, school facilities scores,
teaching content scores, and teaching process scores?

3

**Objectives **

The main objectives of this study are

1. To develop the best model for modelling between average scores of *UN*

all public *SMA*s for both natural science major and social science major
and another explanatory variables mentioned above.

2. To evaluate how much do the explanatory variables contribute at
affecting the average scores of *UN*.

Apart from the main objectives mentioned above, this study also has a secondary objective which is to preliminary analyze the possibility of designing samples for evaluating student achievement.

**Research Benefit **

The best models which is obtained in this research will lead to what
influence *UN* the most. As the first preliminary step to evaluate the process
of conducting the *UN* since there has been provision that *UN* is no longer
used to measure student pass rate.

**LITERATURE REVIEWS**

**Panel Data **

A panel data set is one that follows a given sample of individuals over time, and thus provides multiple observations on each individual on the sample (Hsiao 2003). Observing a panel data set is basically observing a broad section of subjects over time, and thus allows us to study dynamic, as well as cross sectional, aspects of a problem (Frees 2004).

Baltagi (2005) mentioned the main advantages of panel data analysis such as.

1. Solves individual heterogeneity.

2. Provides more informative dataset, handles collinearity of variables, controls some types of ommited variables without observing them, obtains degree of freedom in a bigger amount.

3. Studies comprehensively the dynamics of adjustments.

4. Identifies and estimates effects which cannot be taken into traditional technique (e.g. time series analysis and usual linear regression).

5. Reduces bias when estimating because of plenty number observations.
Define following general panel data model for observation _{ } of cross
section data *i* at time *t*

4

where is a vector of regression coefficient with K x 1 where K is number
of explanatory variables, _{ } is a response variable observed for individual
quantities were non random. Fixed and random effects to respectively refer
to the population average and subject specific effects. Fixed effect model
which has form as.

(2) where denoted subject dimension which equals 1, 2, ... N. denoted time dimension which equals 1, 2, ..., T and with specification of residual

where is the number of explanatory variables.

, with is an identity matrix, is a vector. is
matrix of dummies which is included in the regression to estimate * α* if the
parameters are assumed to be fixed (Baltagi,2008).

**Linear Mixed Model **

According to Jiang (2007), linear mixed model takes general form that can be written as

5

Jiang (2007) also stated that estimation in linear mixed model consists of estimation toward both the fixed effect and random effect that can be shown below. Estimation of fixed effect is carried out using Best Linear Unbiased Estimator (BLUE)

̂ (5) and the random effect prediction which is Best Linear Unbiased Prediction (BLUP) can be written as

̃ _{( ̂ }_{ (6) }
In fact, variance component **V **is unknown therefore, in estimating fixed
effect and random effect, **V **is replaced by the estimator ̂ and the estimation
result is empirical best linear unbiased predictor (EBLUP)

̂ ̂ ̂ ** **(7)
where ̂ and ̂ are estimator using Maximum Likelihood Estimation (MLE)
or Restricted Maximum Likelihood (REML).

Jiang (2007) mentioned that the estimation result using REML for variance componen is a consistent estimator without assuming normality from random variable and error. REML is also an estimation method that is more robust and practical.

6

Setting these results to zero will obtain the following equations

{ _{ }_{ } _{ }_{ } _{ }_{ } _{ }
mixed model are the traditional approach, Maximum Likelihood Estimation
(MLE), and Restricted Maximum Likelihood (REML). One of those
approaches used for this study is REML.

REML can produce the unbiased or nearly estimator for variance and it does not require more complex computation just like the MLE estimators. Searle (1992) stated that the REML approach is now more preferable to estimate the variance parameters in mixed model.

**y** is a response vector that normally distributed which is transformed
for **z** = **A**’**y **which has distribution** z ~ N**(**0, A’VA**). Define a non-zero *n x *
then we come to the final expression of the desity of **z**

_{ }_{ }_{ }_{ }_{ { }

( ̂ ( ̂ }

the final density of **z** leads to the restricted or residual log likelihood which
is written below.

_{ } _{ }

_{ }_{ }

7 usually known as generalized estimating equation (GEE). Basically, GEE uses quasi score likelihood theory.

GEE approach in _{ } _{ } _{ } for estimating by using
*quasi-score likelihood *equationwhich is shown as follow

∑ [ ] with [ ] ⁄ dan ⁄ ⁄

is diagonal matrix with element var( _{ }), is *working *
*correlation matrix* for , and is constant (Dobson 2002). Some general
forms of structure of working correlation matrix presented in the Table 1 as
follow

Table 1. Structures of working correlation matrix Stuctures obtained by separating the population elements into nonoverlapping groups, called strata, then selecting a simple random sample from each stratum. The principal reasons for using stratified random sampling areas follows:

8

1. Stratification may produce a small bound on the error of estimation. This result is particularly true if measurements within strata are homogenous

2. The cost per observation in the survey may be reduced by stratification of the population elements into convenient groupings 3. Estimates of population parameters may be desired for subgroups of

the population. These subgroups should then be identifiable strata. The first step in the selection of a stratified random sample is to clearly specify the strata then each sampling unit of the population is placed into its appropriate stratum. After the sampling units have been divided into strata, select a simple random sample from each stratum.

Estimator of the population mean is

̅ _{ ∑ }

̅

where is number of sampling units in stratum *i*, *i* = 1, 2, .. *L*, ̅ denotes
the sample mean for the simple random sample selected from stratum *i. *

Estimator variance of ̅_{ }is

A two stage cluster sample is obtained by first selecting a probability sample of clusters and then selecting a probability sample of elements from each sampled cluster (Scheaffer et al. 2006). Cluster must be as heterogeneous (different) as possible within, and one cluster should look very much like another in order for the economic advantages of cluster sampling to pay off. If a population being divided into nonoverlapping groups of elements. These groups are considered to be clusters, then a simple random sample of groups is selected, and the sampled groups are then subsampled.

The advantages of two stage cluster sampling are

1. A frame listing of all elements in the population may be impossible or costly to obtain, whereas obtaining a list of all clusters may be easy

2. The cost of obtaining the data may be inflated by travel costs if the sampled elements are spread over a large geographic area. Thus, sampling clusters of elements that are physically close together is often economical.

9

population, is the number of clusters selected in a simple random sample,
is the number of elements in cluster . is the number of elements
selected in a simple random sample from cluster equals ∑_{ } ̅ is
the sample mean for the *i*th cluster ∑_{ } _{ } and _{ } is the th
observation in the sample from the th cluster, is

̂ ( ∑ ̅ ̅

∑ ̅

The estimated variance of is

̂ ̂ ( (

̅ ̅ ∑ _{ } (

where

∑ _{ } ̅ ̅

and

∑ ( ̅

is simply the sample variance among the terms ̅ and that is the sample variance for the sample selected from cluster .

**MATERIALS AND METHODS **

**Data **

1. The average scores of *UN* and *US* for both natural science diciplines and
social science diciplines of all *SMAN*s in West Java 2011-2014. Data was
accessed from Educational Research Center, Ministry of Education and
Culture with the lowest score is zero and the highest one is 10 then it is
multipled with 10 therefore the score range becomes between 10 and
100.

2. Data of total accreditation scores, educators scores, school facilities scores, teaching content scores, teaching process scores, school management scores, school finance scores, and graduates competence scores were accessed in National Accreditation Institution.

3. Data of human development index and gross regional product in West Java were accessed through Statistics Indonesia website.

**Variables **

10

Y : The average scores of *UN* each *SMAN* for both natural science and
social science

X1 : *IPM* scores each regency/municipality in West Java
X2 : *PDRB* each regency/municipality in West Java
X3 : Teaching content scores each *SMAN* in West Java
X4 : Teaching process scores each *SMAN* in West Java
X5 : Educators scores each *SMAN* in West Java

X6 : School facilities scores each *SMAN* in West Java
X7 : Total accreditation scores each *SMAN* in West Java
X8 : The average scores of *US* each *SMAN* in West Java

**Research Methodology **

This research is divided into two to reach the goals, first is modelling, and the second is sampling technique. Some steps in data analysis process in this research are

**Modelling The Data **

1. Exploratory data analysis

In this step, the variability and mean of response variable in each regency/municipality in West Java was investigated variable to visualize the information was provided.

2. Identification of the multicollinearity among explanatory variables The multicollinearity was identified by calculating the coefficient of correlation among the explanatory variables. The data was then analyzed using principal component analysis to remove the multicollinearity.

3. Modelling the data with the linear mixed model (LMM)

Models were fitted by incorporating fixed and random effects
which accomodated that every school *i *at time *t *had their own
intercept. The model which contained mixed effect took form as:

̂ ̂ ̂ ̂ , where ̂ is a vector of estimated beginning. In contrast, the time effect is assumed to be random because year can not be limited and there will always be next year therefore the data includes 2011, 2012, 2013 and 2014.

11

dummy variables respectively. The model parameters was estimated using restricted maximum likelihood (REML).

4. Modelling the data with generalized estimating equation (GEE).

̂ ̂ ̂ , where ̂ is the estimated school effect approached to be fixed and this model was estimated using GEE which accomodates the correlated response variable using the working correlation matrix : AR(1) in condition of the time series data which was taken solely covers 4 years. This model is considered because it is able to accomodate the heterogeneity among school effect using dummy variables. GEE model with response variable is normally distributed using identity link function. The working correlation matrix has the following form as shown below assumed to be fixed into the equation took form as

̂

The model parameters were estimated using quasi-score likelihood where the variance of as a function of the mean, contained the working correlation matrix autoregressive.

5. Modelling the data with fixed effects

This model was built with consideration that the numbers of both of schools and time were defined from the beginning. Because of the effects of schools and time were decided to be fixed, it has been added dummy variables into the model which could be written as

̂ ̂ ̂ ̂

where ̂ denoted the estimated school effect and ̂ is the estimated time effect. The model parameters were estimated by using analysis of variance.

6. Evaluation of the models by comparing the R-squared value and mean square error of each model. The model with the largest value is assumed as the best model.

7. Interpretation of the data analysis results.

**Sampling Technique **

1. Divided population in this case covers West Java into two strata which are city and regency.

2. West Java has 9 cities and 19 regencies in which is considered as population clusterings.

12

B on error of estimation which are in this research 0.05, 0.07, and 0.09, respectively.

4. *n *was defined, chose element of sampled cluster by using simple
random sampling.

5. Making an experiment by taking schools in each cluster with simple random sampling in terms of the characteristics how many schools each municipality/regency has.

6. Calculated the variance of the average scores of *UN* for each school
in each municipality/regency in which there had 6 variances found
out with three stages sampling formula which include stratified
sampling, cluster sampling, and simple random sampling.

13

**RESULT AND DISCUSSION **

As it has been commonly acknowledged in Indonesia, there are two programs available in many senior high schools namely natural science and social science. This research has both utilized and analyzed the data of those two different programs separately due to not every senior high schools in West Java has both natural science and social science and it then can not be concluded that students learning ability in natural science is as equal as in social science.

There were two initial models established. Those two models were built based on the data of natural science and social science. Evaluating the model and selecting the best model was done by comparing the R-squared value and mean square error of each model.

The averages scores of *UN *every year have been presented in Figure
1.

Figure 1. Scatterplot of the average score of *UN* versus years for a) natural
science b) social science in West Java

The Figure 1 showed that there was such a decline on the slope between
year 2012 and 2013. The most current years which are 2013 and 2014, the
average scores of *UN *for natural science has decreased. Sharp slope also
happened between year 2012 and 2013 on the plot of social science major.

Another exploration obtained was the mean and variability of the
average scores of *UN* in the very curren year, 2014 for each
municipality/regency were also provided in Appendix 9 and Appendix 10.
Both of natural and social science, Bandung City has the higest mean of the
average scores of *UN. *Bandung City is one of big cities in Indonesia in
which its goverment pays much attention for education there.

The descriptive statistics of the average scores of *UN *of all public
senior high schools in West Java also shown in the Table 2.

14

Table 2. The descriptive statistics of national examination

Term 2011 2012 2013 2014
system of *UN*, enforced that each task paper has been made exactly different
for each student which was implemented since 2013 in order to dismiss
every kind of cheating students possibly could do in *UN* performance.

Another descriptive statistics measured the relationship between response variable and explanatory variable has been statistically illustrated by coefficient correlation shown in the following table.

Table 3. Coefficient correlation between response and explanatory variables for natural science Table 4. Coefficient correlation between response and explanatory variables

for social science

regency/municipality in West Java, X3 : Teaching content scores each *SMAN* in West Java

X4 : Teaching process scores each *SMAN* in West Java, X5 : Educators scores each

*SMAN* in West Java, X6 : School facilities scores each *SMAN* in West Java, X7 : Total

accreditation scores each *SMAN* in West Java, X8 : The average scores of *US* each *SMAN* in

West Java

15

natural and social science the correlation between Y and X is not very strong.

Another exploration to make sure by calculating variance inflation factor (VIF) to quantify the severity of multicollinearity in an ordinary least squares regression analysis. The following table showed VIF of all X.

Table 5. Variance inflation factor of explanatory variables Predictor VIF there was multicollinearity which could cause a serious problem which make the standard error get higher if it was not removed and would lead to the wrong conclusion for this study.

**Principal Component Analysis **

From the result shown Table 5, it is clear that multicollinearity among explanatory variables has been existed. Principal component analysis has been widely used to eliminate the correlation between explanatory variables through transformation which has been presented in Table 5. It then can be ensured that all explanatory variables is independent. Many methods to solve multicollinearity yet principal component analysis has been chosen in order to include all informations in X without eliminating an X that has high VIF value. Table 6 of eigen analysis for both natural and social science is shown below and eigen vectors which was generated shown by Appendix 5 and Appendix 6.

16

Table 6 showed that the cumulative proportions of variance explained by the first five components have shown 0.91 both for natural and social science

The response (the average scores of *UN*) is continous variable which
was assumed normally distributed.This assumption was justified by the
central limit theorem. Here the response variable (the average scores of *UN*)
has been modelled with those five components for each program
respectively. Every modelling which has been done in order to find best
model gave different results by comparing the value of R-square, R-square
adjusted, and mean square error. The Table 7 shows the result of 3 different
modellings in the simple version.

Table 7 showed that it was directly identified that the best model was model 3 in which both school effect and time effect were assumed to be fixed with the highest R-squared value, 91,89 for natural science and 89,23 for social science, and with the lowest MSE 15,03.

The data also has been modelled with linear mixed model in Model 1.
This model contained two effects, fixed and random effects outright. The
response (average scores of *UN*) is a normally distributed outcome which is
allowed to use the linear mixed model. School effectwas assumed to be
fixed effect and time effect was assumed to be random effect. This model
was not the best model because time should not be random effect. In fact,
time was fixed effect because the sequence of year of observed data was
defined by the researcher from the beginning.

17

sequence of time which only included 4 years observed data. Therefore it could mean that possibly data of 2014 was not affected by the previous year, data of 2013. It also occured in data of 2013 was not affected by data of 2012 and so forth.

Another plot to see that the best model was a model which was good
at predicting. The accuracy at predicting can be presented in a plot between
the average scores of *UN* and its predicted values shown in the following
table.

Table 8. Plots of the actual and predicted values in each model

Natural Science Social Science

Modelling with Generalized Estimating Equation

Modelling with Linear Mixed Model

Modelling with Fixed School Effect and Fixed Time Effect

18

around the diagonal line. Both in model 3 for natural science and social science, standardized residual was normally distributed.

**Overview of Model 3 (Fixed Effect Model) **

Model 3 which had form as ̂_{ } _{ } ̂ ̂ ̂ where ̂ is the
estimated model parameters, ̂_{ } is the estimated response variable observed
for school *i *at time *t*, _{ }is the vector of principal components for school *i *

at time *t*, ̂ is the estimated schools effect, ̂ is the estimated time effect
which were assumed to be fixed was ensured because of the numbers of
schools and time (the observed years) were both defined and chosen from
the beginning by researcher.

Model 3 is a model with fixed effect which used dummy variables had
an ability to catch the variability of the subjects in this case the subjects
were schools and time, had the highest R-squared value was because time
effect more fits if it was assumed to be fixed rather than to be random or
AR(1). Beside that, the variance of time ( 2011 – 2014 ) was quite high
because there was always modification of system of implementation of *UN *

globally every year. The model parameters were resulted in the Table 9. Table 9. Parameter estimates for the first five components of natural science

Term Coef SE coef T-Value P-Value which indicated that those were statistically significant. After that, response variable Y was remodelled with and to generate the coefficient regression before transforming back to x.

Table 10. Parameter estimates for the three selected components of natural science

Term Coef SE coef T-Value P-Value

w1 1.79 0.67 2.66 0.01

w3 2.85 0.37 7.74 0.00

w5 -8.25 0.99 -8.31 0.00

Table 10 showed that linier combination of all explanatory variables which
were *IPM*, *PDRB*, the scores of educators, facilities, content, process, total
accreditation and the average scores of *US * took part at significantly
affecting the response variable (the average scores of *UN*). The resulting
equation for natural science took form as

19 explanatory variables is fixed, it could be interpreted that the coefficient of

*IPM* score (X1) is 2.30 which indicated that the average score of *UN* is

each change of 1 unit in educators scores’s (X5), Y changes 0.12 unit. The
definition of regression coefficient in the school facilities scores (X6) which
is 0.15 is the average scores of *UN* is predicted to increase 0.15 when the
school facilities score goes up by one. Of teaching process variable, for each
change of 1 unit in total accreditation scores (X7), Y changes 0.20 unit. The
average score of *UN* is also predicted to increase 0.09 when the average
score of *US* (X8) score goes up by one. These two models both for natural
science and social science have different intercept for every school *i* at time

*t* which was considered as the variability and it were denoted by and .
The analysis of variance for natural science is represented in the Appendix 1
and Appendix 2.

Another program the public senior high schools have is social science and resulting the model parameter estimation is shown in Table 11.

Table 11. Parameter estimates for the first five components of social science

Term Coef SE coef T-Value P-Value indicated that those were statistically significant.

Table 12. Parameter estimates for the two selected components of social science

Term Coef SE coef T-Value P-Value

w1 0.90 0.13 7.01 0.00

20

From table 12 which generated equation using regression with fixed effects was

̂

The equation above was tranformed into the original form by using the formula of mean and standard deviation of each explanatory variable which were reresented in Appendix 8 as

̂

From the resulted equation of social science, the interpretation of
regression coefficients in terms of all explanatory variables is fixed is the
coefficient of *IPM* score (X1) is 0.25 which indicated that the average score
of *UN* is predicted to increase 0.25 when *IPM* score goes up by one. The
average score of *UN* is also predicted to increase 0.05 when *PDRB* (X2)
score goes up by one. The average score of *UN* is predicted to increase 0.05
when teaching content (X3) goes up by one. Of teaching process variable,
for each change of 1 unit in teaching process (X4), Y changes 0.03 unit. For
each change of 1 unit in educators scores (X5), Y changes 0.05 unit. The
definition of regression coefficient in the school facilities scores (X6) which
is 0.03 is the average scores of *UN* is predicted to increase 0.03 when the
school facilities score goes up by one. Of teaching process variable, for each
change of 1 unit in total accreditation scores (X7), Y changes 0.07 unit. The
average score of *UN* is also predicted to increase 0.10 when the average
score of *US* (X8) score goes up by one. The analysis of variance for social
science can be seen in the Appendix 3 and Appendix 4.

Assesing the residual plots of the best models are needed. The following plot showed the normality of the residuals.

Figure 2. Normality plots for a) natural science b) social science Figure 2 showed that residuals of fixed model for natural science and social science are both normally distributed. The homoscedacity assumption for fixed model is also not violated shown by the Figure 3.

21

Figure 3. Residual versus predicted value for a) natural science b) social science

Since the plot on the Figure 3 can be categorized as well-behaved plot because of the standardized residual points do bounce randomly around the zero line. This suggests that the linearity assumption of relationship between response and principal component scores is not violated.

**Preliminary Analysis Of The Required Sampling Method For National **
**Examination **

In West Java there are 8 municipalities and 19 regencies in which
being consideration for making strata because it is possible there have a
significance difference between schools in the municipalities and other
schools that are located around regencies. Locations may have probability to
differ the output of the schools, in this context, the average scores of *UN*.
Almost all schools in the cities tends to have more sophisticated facilities
than schools in regencies which means the higher scores for *UN* must be
reached easily by the schools in the municipalities.

In this sampling technique, the next steps after making two stratas are clustering. To see more about the technique sampling that is used in this research, it is represented by the sampling scheme Figure 4.

Figure 4. Sampling Scheme

Regency/City

City Regency

R1 R2 C

11 C2 C9

S1 Sn

StRS CR

SR

S1 Sn

22

To determine the number of cluster to be sampled is by using the formula (Scheaffer et al. 2006) which is

∑
∑_{ }

where is the fraction of clusters allocated to stratum *i*, is the
population variance for stratum *i*, and , where B is bound of error.

The writer did an experiment used data of *UN* 2014 for natural
science for example. It has done for thrice replications with three sample
sizes of cluster which are obtained by the formula above with three different
bound of errors.

The consideration of choosing bound of errors 0.05, 0.07, and 0.09 because if it was taken less than 0.05 the number of cluster would be out of maximum number cluster (27 clusters). If it was taken more than 0.09 the number of cluster would be close to zero. The table 13 showed the number of sampled cluster with different bound of errors.

Table 15. Number of sampled clusters with different bound of errors Bound of errors The number of sampled clusters

0.05

and

0.07

and

0.09

and Clusters are sampled with simple random sampling for clusters from municipality stratum and regency stratum, respectively.

23 number of schools which is sampled with simple random sampling in cluster

*j*, is number population of clusters (municipality/regency) in stratum *k, *k
is stratum (municipality/regency) =1,2, is number of sampled clusters in
stratum *k*, _{ } is the *i*th observation of schools in the sample from the *j*th
cluster and the *k*th stratum, *i*= 1, 2, . . , _{ }. The formula derivation of mean
and variance of three stages sampling are shown both in Appendix 11 and
Appendix 12.

Table 16. Variance of average scores of *UN*

Number of

24

**CONCLUSION AND RECOMMENDATION **

**CONCLUSION **

It was concluded from this research that the *UN* scores could be
modelled using multiple regression approach with fixed effects in which
school and time effects that can catch the variability of every subject. This
model was used only for analyzing the four years observed data.

For natural science, the scores of *PDRB*, teaching process, educators,
school facilities, total accreditation and the average score of *US* gave
positive contributions when its score goes up by one unit to the average
scores of *UN*. Otherwise, for its change of one unit of the scores of *IPM* and
teaching content gave negative contributions to the average scores of *UN* .
For social science, all explanatory variables gave positive contributions
when its score goes up by one unit to the average scores of *UN.*

**RECOMMENDATION **

According to the fact that the average scores of *UN* is getting lower
from time to time, the West Java government must pay serious attention to
maintain and improve the quality of education. From the best model
obtained that the average scores of *UN* increased along with the increasing
scores of teaching content, teaching process, educators and facilities in
every school. Moreover, related to human development index (*IPM*),
government should put attention to improve the society life expectancy,
mean years of schooling, expected years of schooling, and gross national
income and through *PDRB* government can dispose the fund for education.

For the next implementation of National Examination can be more effective if not all schools in West Java included. Use a three stage sampling technique (Stratified Sampling – Cluster Sampling – Simple Random Sampling) more fits with the condition of all schools in West Java.

**REFERENCES **

Ahmad A. 2011. Analisis hubungan pengeluaran pendidikan dan
pertumbuhan ekonomi dengan menggunakan pendekatan kausalitas
Granger*.* *Jurnal Ekonomi & Pendidikan, Volume 8 Nomor 2, *
*November 2011*

Baltagi BH. 2005. *Econometrics Analysis of Panel Data Ed ke-3*. England:
John Wiley & Sons Ltd.

[BPS] Badan Pusat Statistik. 2008. *Index Pembangunan Manusia *
*2007-2008*. Jakarta : BPS.

25

*Aliyah/Sekolah Menengah Atas Luar Biasa dan Sekolah Menengah *
*Kejuruan Tahun Pelajaran 2007/2008*. Jakarta : Depdiknas.

Dobson AJ. 2002. *An Introduction to Generalized Linear Models*. Chapman
Hall, London, UK.

Frees EW. 2004. *Longitudinal and Panel Data: Analysis and Applications *

*in the Social Sciences*. Cambridge University Press, New York.

Hsiao C. 2003*. Analysis of Panel Data (2nd edition).* Cambridge University

Press, New York.

Henderson CR, Kempthorne O, Searle SR, Von KCN. 1959. Estimation of environmental and genetic trends from records subject to culling.

*Biometrics 15 (2) : 192-218*.

Jiang J. 2007. *Linear, Generalized Linear Mixed Models and Their *
*Applications. *New York : Springer.

Longford NT. 1993. *Random Coefficient Models*. Oxford: Oxford
University Press.

Mongi CE. 2014*. * Penggerombolan dan pemetaan kabupaten/kota di
Provinsi Jawa Barat berdasarkan nilai ujian nasional SMA dan
akreditasi sekolah [Tesis]. Bogor (ID): Institut Pertanian Bogor.
Rao CR. 1973*. Linear Statistical Inference and its Applications*. New York:

Wiley.

Searle SR, Casella G, McCulloch CE. 1992. *Variance Components*. New
York: Wiley.

Scheaffer RL, Mendenhall W, Ott RL, Gerow KG. 2006. *Elementary Survey *
*Sampling 6th Edition*. USA: Thomson Brooks/Cole.

Sinaga SS. 2011. The uses of canonical analysis to know the pattern of relationship among scores of national exam, scores of school exam, and progress report (case study at SMA Budhi Warman II Jakarta) [Tesis]. Bogor (ID): Institut Pertanian Bogor.

26

Appendix 1 Analysis of variance for the five components of natural science

Source DF Adj SS Adj MS F-value P-Value

Appendix 3 Analysis of variance for the five components of social science

Source DF Adj SS Adj MS F-value P-Value

27

Appendix 5 Eigen vectors for natural science

Variables Natural Science
Standardized Accreditation 0.52 -0.05 0.01 -0.12 -0.04
Standardized *US* scores 0.09 0.37 -0.83 0.23 -0.30
Appendix 6 Eigen vectors for social science

Variables Social Science
Standardized Accreditation 0.51 -0.06 0.06 -0.11 0.04
Standardized *US* scores 0.11 0.28 -0.91 0.08 0.28
Appendix 7 Statistic descriptive for natural science

Explanatory Variable Mean Standard Deviation

Total accreditation 89.54 6.39 59.16 98.88

The average scoresof *US* 83.01 3.02 73.63 94.23
Appendix 8 Statistics descriptive for social science

Explanatory Variable Mean Standard Deviation

Total accreditation 89.57 6.39 59.16 98.88

28

Appendix 9 Statistics descriptive of each municipality/regency for natural science in 2014

Municipality / Regency School Status Mean Min Max Stdev

M. BANDUNG N 41.35 23.65 57.65 4.95

M. BANJAR N 35.51 25.60 50.55 4.68

M. BEKASI N 39.48 25.60 54.65 5.10

M. BOGOR N 42.07 27.65 56.05 4.99

M. CIMAHI N 38.37 25.70 52.60 4.64

M. CIREBON N 36.42 23.65 54.55 4.84

M. DEPOK N 39.43 24.75 55.20 5.53

M. SUKABUMI N 35.93 23.30 51.80 5.01

M. TASIKMALAYA N 36.93 25.60 52.30 5.10

R. BANDUNG N 34.87 24.05 50.80 4.90

R. BANDUNG BARAT N 33.37 22.70 49.95 4.27

R. BEKASI N 33.31 21.55 55.65 4.89

R. BOGOR N 33.87 21.90 51.20 5.22

R. CIAMIS N 33.82 24.65 51.60 4.25

R. CIANJUR N 34.37 24.90 53.20 4.40

R. CIREBON N 32.67 22.80 50.05 4.39

R. GARUT N 32.35 18.55 50.05 4.24

R. INDRAMAYU N 31.59 23.65 50.20 3.44

R. KARAWANG N 30.79 22.75 52.05 3.28

R. KUNINGAN N 33.99 25.00 51.65 4.19

R. MAJALENGKA N 33.29 24.30 51.70 3.86

R. PURWAKARTA N 34.00 23.80 52.15 5.13

R. SUBANG N 30.72 21.50 49.55 4.76

R. SUKABUMI N 35.06 24.60 54.05 5.41

R. SUMEDANG N 33.16 22.60 53.75 4.12

R. TASIKMALAYA N 33.55 23.65 47.45 4.62

R. PANGANDARAN N 31.31 25.80 43.00 2.94

29

Appendix 10 Statistics descriptive of each municipality/regency for social science in 2014

Municipality / Regency School Status Mean Min Max Stdev

M. BANDUNG N 35.99 21.80 52.70 4.58

M. BANJAR N 33.02 23.95 49.20 4.10

M. BEKASI N 33.82 21.95 52.05 4.73

M. BOGOR N 39.33 25.40 54.30 5.23

M. CIMAHI N 33.43 23.85 50.65 4.53

M. CIREBON N 32.07 19.90 49.95 4.39

M. DEPOK N 35.74 21.20 54.95 6.60

M. SUKABUMI N 31.35 21.75 48.60 4.45

M. TASIKMALAYA N 32.51 22.25 49.45 4.41

R. BANDUNG N 32.26 20.60 49.15 4.17

R. BANDUNG BARAT N 30.89 22.80 51.25 3.86

R. BEKASI N 30.02 21.80 52.05 4.22

R. BOGOR N 31.27 19.65 49.55 5.09

R. CIAMIS N 31.62 21.30 48.60 3.94

R. CIANJUR N 32.23 23.20 48.45 4.55

R. CIREBON N 30.20 20.65 50.00 3.84

R. GARUT N 29.94 22.20 49.80 3.80

R. INDRAMAYU N 30.08 19.15 48.80 3.74

R. KARAWANG N 29.05 22.00 47.95 3.34

R. KUNINGAN N 32.30 22.85 52.15 4.19

R. MAJALENGKA N 31.42 22.50 46.70 4.03

R. PURWAKARTA N 31.27 23.15 49.45 4.31

R. SUBANG N 28.38 21.30 50.15 4.05

R. SUKABUMI N 33.01 22.35 51.55 5.52

R. SUMEDANG N 31.01 22.40 47.35 3.79

R. TASIKMALAYA N 31.28 21.95 44.70 4.42

R. PANGANDARAN N 29.22 22.50 42.90 2.88

31

Appendix 12 The formula of estimated variance of three stage sampling (stratified sampling – cluster sampling – simple random sampling)

̂ ̅ [ ̂ ̅ ̂ ̅ ]

Estimated variance of ̅ from two stages cluster sampling in strata 1

32

**BIOGRAPHY **