COMPARATIVE STUDY OF BAYESIAN AND ORDINARY LEAST SQUARESAPPROACHES

Abstract

Frequentist (Classical) and Bayesian Statistics are two major approaches to data analysis in statistics; however, the difference is how both see a parameter. Frequentists see a parameter as constant value while the Bayesians see it as random variable. Resent research has witnessed increase in the application of Bayesian methods to statistical problems and in other fields. For linear regression modelling, frequentists use more often the Ordinary Least Squares (OLS) method despite violation of some
assumptions. Bayesian approach can be used when assumptions in linear regression model using OLS are not met. The study objective is to ascertain if datasets provided meet the classical OLS assumptions or not, and demonstrate the prominence of Bayesian methods where OLS assumptions are violated. Two different data sets were adopted in this paper for a comparative study using both OLS and Bayesian approaches to linear regression modelling. The analysis showed that the resulting linear regression model using OLS does not meet all required assumptions for a good model. The Bayesian approach as an alternative to regression
modelling was further established based on results of smaller standard errors. The results showed that linear regression modelling using Bayesian approach is better than Frequentist method using OLS regression modelling.