In this section a brief list of advantages of Bayesian inference over frequentist inference approach is introduced.
(1) Bayesian inference allows informative priors so that prior knowledge or results of a previous model can be used to inform the current model.
(2) Bayesian inference can avoid troubles with model identification by manipulating prior distributions. Frequentist inference with any numerical approximation al- gorithm does not have prior distributions, and could be stuck in regions of flat density, which is considered a problem with model identification.
(3) the Bayesian approach treats the data as fixed, and the parameters as random because they are unknowns. The frequentist approach considers the unknown parameters as fixed, and the data as random, which is indicates that the esti- mation does not rely on the data it self, it also relies on hypothetical repeated
sampling in the future with similar data [128].
(4) the Bayesian approach estimates the full probability model. While the frequen- tist does not do so.
(5) the Bayesian approach estimates p(hypothesis|data). In contrast, the frequen- tist approach estimatesp(data|hypothesis). Even though, the term(hypothesis testing) suggest it must be the hypothesis that is tested, given the data, not the opposite.
(6) the Bayesian approach has an axiomatic foundation [38] that is uncontested by the frequentist approach. Furthermore, the Bayesian approach is coherent to a frequentist, but a frequentist approach is incoherent to a Bayesian.
(7) the Bayesian approach has a strong decision theoretic foundation [14,125]. The aims of the statistical inference are to facilitate decision making [125]. The most optimal decision is the Bayesian decision.
(8) the Bayesian approach includes uncertainty in the probability model, producing more realistic prediction. The frequentist approach does not includes the uncer- tainty of the parameters estimates, so it produces less realistic predications the Bayesian approach.
(9) the Bayesian approach is consistent with many of philosophies of science re- garding to the epistemology, where knowledge can not be built entirely through experimentation, but requires prior knowledge [125].
(10) the Bayesian approach has the ability to compare different models with different methods using Deviance Informatin Criteria (DIC) including hierarchical models, but the the frequentist approach cannot.
(11) the Bayesian approach obeys the likelihood principle, Whereas the frequentist approach including MLE and the the General Method of Moments (GMM) or
the Generalised Estimating Equations (GEE), violates the likelihood principle [128].
(12) the Bayesian approach is protected against over fitting by integrating over model parameters whereas over fitting occurs in frequentist approach and is a serious problem in it.
(13) the Bayesian approach uses observed data only. But the frequentist approach uses both observed data and future data that are unobserved and hypothetical.
(14) the Bayesian approach uses the prior distribution which means that, more infor- mation is used and the 95% confidence interval of the posterior will be narrower than the 95% confidence intervals of the frequentist approach.
(15) the Bayesian approach uses probability interval to state the probability that the θ is between two points. The frequentist approach uses confidence intervals, which should be interpreted with a probability of zero or one that θ is in the region, so the frequentist never knows whether it is or is not, but can only say that if 100 repeated samples were drawn in the future, that it would be in the region for 95 samples.
(16) the Bayesian inference via using MCMC has a theoretical guarantee than the MCMC algorithm will converge if we run long enough but in frequentist infer- ence we do not have guarantees for the convergence of the MLE.
(17) the Bayesian inference via MCMC or PMC is unbiased with respect to the sample size whether it is small or large. The frequentist becomes more biased when the sample size is small.
(18) the Bayesian inference via MCMC or PMC uses exact estimation with respect to sample size. When the frequentist uses approximate estimation it depends on asymptotic theory.
(19) the Bayesian inference with correlated predictors sometimes allows the hyperpa- rameters to be distributed multivariate normal, therefore, including such correla- tion into the MCMC or PMC algorithm to enhance estimation. The frequentist inference does not use prior distribution, therefore, the confidence intervals will be wider and less certain with correlated predictors.
(20) The Bayesian inference with perfect priors is immune to singularities with matrix inversions, unlike frequentist inference.
4.8.1 Advantages of the frequentist approach over the Bayesian approach(disadvantages)
In this subsection the advantages of the frequentist approach over the Bayesian ap- proach are listed
(1) frequentist models are good in handling large data sets, while Bayesian models (MCMC,PMC) have a problem handling large data set.
(2) frequentist models are always much easier to prepare, because many things do not need to be specified, such as prior distribution, initial values for numerical approximation, and the likelihood function. The Bayesian approach specify the prior distribution, initial values and so on. The frequentist approach is also well developed so it easy for anyone to apply it.
(3) frequentist models have a much short time to run compared with the Bayesian method. Simple frequentist models may be run in minutes, while the same model in Bayesian approach may take a week to run.