多元线性应用回归.ppt

《多元线性应用回归.ppt》由会员分享，可在线阅读，更多相关《多元线性应用回归.ppt（112页珍藏版）》请在三一办公上搜索。

1、浙江财经学院 倪伟才,1,第三章 多元线性回归,3.1 多元线性回归模型3.2 回归参数的估计3.3 参数估计量的性质3.4 回归方程的显著性检验3.5 中心化和标准化3.6 相关阵与偏相关系数3.7 本章小结与评注,浙江财经学院 倪伟才,2,第三章多元线性回归模型,例子：工资收入y，教育x1、工资经验x2;产品的销售量y，自身的价格x1、替代品的价格x2、互补品的价格x3;某品牌手机的销售额y，广告费x1、价格x2、可支配的收入x3、研发的投入x4;汽车的速度y，动力x1、重量x2;血糖y，胰岛素x1、生长素x2。以上例子的特点：被解释变量只有一个，解释变量有2个或2个以上，这样的模型称为多

2、元线性回归模型。本章的主要内容：多元线性回归模型、基本假设、未知参数的估计及性质、回归方程的系数和回归方程的检验、预测等。本章特点：利用矩阵进行计算。,浙江财经学院 倪伟才,3,3.1多元线性回归模型,一 多元线性回归模型的一般形式：y=0+1 x1+2 x2+p x p+y为被解释变量，是随机变量；x1,x2,x p 为解释变量，确定性变量，可以控制和测量；0,1,p 是（p+1）个未知参数；0回归常数，1,p 回归系数。n组样本观测值（xi 1,xi 2,xi 3,x i p;y i）i=1,2,n.(每一组样本观测值为一个向量，前面一个下标 i 表示第 i 组样本观测值，后面一个下标表示

3、解释变量),浙江财经学院 倪伟才,4,多元线性回归模型的矩阵形式,浙江财经学院 倪伟才,5,矩阵形式,其中Y,均为n维列向量；X为n*(p+1)的矩阵；为p维列向量,浙江财经学院 倪伟才,6,二.多元线性回归模型的基本假设,1.x1,x2,xp是确定性变量.2.xi(i=1,2,p.)之间无线性关系(即无共线性).3.高斯马尔科夫条件:E(i)=0,i=1,2,n.;Cov(i,j)=0,ij;Cov(i,j)=2,i=j.4.i的正态性假设：i N（0，2）。,浙江财经学院 倪伟才,7,三、回归系数的解释 例题,多元线性回归方程的解释,y表示空调机的销售量,x1表示空调机的价格,x2表示消费

4、者可用于支配的收入。,y=0+1x1+2x2+E(y)=0+1x1+2x2,在x2保持不变时,有,在x1保持不变时,有,浙江财经学院 倪伟才,8,三、多元线性回归方程的系数解释,浙江财经学院 倪伟才,9,四.完全共线性 例子,假设我们想估计竞选支出对竞选结果的影响.假定每次选举都有两位侯选人.令vote A为侯选人A的得票率,expend A为侯选人A的竞选支出;expend B为侯选人B的竞选支出;tot expend为竞选总支出.为了将每个侯选人竞选支出与竞选总支出的影响隔离开,考虑如下模型:vote A=0+1 expend A+2 expend B+3 totexpend+由于expe

5、nd A+expend B=tot expend,因此这3个自变量存在完全共线性.只要解释1 的意义就会揭示出问题.参数1 被认为是在保持侯选人B的竞选支出和竞选总支出不变的情况下,度量了侯选人A的竞选支出对其得票率的影响.因为如果expend B和tot expend都保持不变,我们就不可能增加expend A,所以这就毫无意义.解决完全共性方法:将3个自变量中去掉1个.,浙江财经学院 倪伟才,10,利用矩阵形式求回归参数的估计,关于向量求导,浙江财经学院 倪伟才,11,浙江财经学院 倪伟才,12,关于矩阵求导,浙江财经学院 倪伟才,13,浙江财经学院 倪伟才,14,浙江财经学院 倪伟才,1

6、5,浙江财经学院 倪伟才,16,残差性质,浙江财经学院 倪伟才,17,rank(X)=p+1 rank(XX)=p+1,浙江财经学院 倪伟才,18,例 题（用Stata!）,数据：ch05pr04.dta请用矩阵求线性回归模型的系数估计值 1：计算矩阵形式XX 2：计算矩阵形式（XX）-1 3：计算矩阵形式XY 4：计算矩阵形式系数的估计值（XX）-1 XY 5：将用矩阵运算得到的系数估计值和软件的直接回归得到的结果比较！,浙江财经学院 倪伟才,19,Stata命令,数据：ch05pr04.dtagen one=1 mkmat y,mat(y)mkmat(one x),mat(x)mat li

7、st x mat list ymat b=inv(x*x)*x*ymat list breg y x,浙江财经学院 倪伟才,20,最大似然估计,yN(X,2In),等价于使(y-X)(y-X)达到最小,这又完全与OLSE一样,浙江财经学院 倪伟才,21,例 3.1（数据：12元回归.sav）,例3.1国际旅游外汇收入是国民经济发展的重要组成部分，影响一个国家或地区旅游收入的因素包括自然、文化、社会、经济、交通等多方面的因素，本例研究第三产业对旅游外汇收入的影响。中国统计年鉴把第三产业划分为12个组成部分，分别为x1农林牧渔服务业,x2地质勘查水利管理业,x3交通运输仓储和邮电通信业,x4批发零

8、售贸易和餐饮业,x5金融保险业,x6房地产业,x7社会服务业,x8卫生体育和社会福利业，x9教育文化艺术和广播,x10科学研究和综合艺术,x11党政机关，x12其他行业。采用1998年我国31个省、市、自治区的数据，以国际旅游外汇收入（百万美元）为因变量y，以如上12 个行业为自变量做多元线性回归，数据见表3.1，其中自变量单位为亿元人民币。,浙江财经学院 倪伟才,22,残差（abstracted from Greene ECONOMETRIC ANALYSIS chapter 3),浙江财经学院 倪伟才,23,we can interpret M as a matrix that produ

9、ces the vector of least squares residuals in the regression of y on X when it premultiplies any vector y.It is convenient to refer to this matrix as a“residual maker.”It follows that MX=0.One way to interpret this result is that if X is regressed on X,a perfect fit will result and the residuals will

10、 be zero.,The n n matrix M defined is fundamental in regression analysis.You can easily show that M is both symmetric(M=M)and idempotent(M=M2).,residual maker:M,浙江财经学院 倪伟才,24,fit value,浙江财经学院 倪伟才,25,The matrix P,which is also symmetric and idempotent,is a projection matrix.It is the matrix formed fr

11、om X such that when a vector y is premultiplied by P,the result is the fitted values in the least squares regression of y on X.This is also the projection of the vector y into the column space of X.,projection(or hat)matrix:P,浙江财经学院 倪伟才,26,projection(or hat)matrix:P性质,1:对称矩阵2:幂等矩阵,浙江财经学院 倪伟才,27,残差的方

12、差协方差矩阵,浙江财经学院 倪伟才,28,随机误差项的方差2的无偏估计为,浙江财经学院 倪伟才,29,课堂练习,数据见题目来源于Greene Notes3解释变量y=G,x=(one,pg,y)请用Stata计算:1:xx,xy,(xx)-1,b2:M3:xe=xMy,where e is residuals4:MX,浙江财经学院 倪伟才,30,Stata命令,egen one=fill(1,1)mkmat G,mat(y)mkmat one Pg Y,mat(x)mat b=inv(xx)*x*y mat e=m*ymat xte=x*m*ymat list xtemat m=I(36)-x

13、*inv(x*x)*xmat mx=m*xmat list mx,浙江财经学院 倪伟才,31,补充内容（矩阵计算）,Applied Linear Regression Models(Fourth Edition)chapter5 simple linear regressionProblems 5.23,5.25例题 Problems 5.23学生练习：Problems 5.25具体请见word格式:回归模型的矩阵计算(stata).doc,浙江财经学院 倪伟才,32,Homework,浙江财经学院 倪伟才,33,3.3参数估计的性质(BLUE),浙江财经学院 倪伟才,34,浙江财经学院 倪伟

14、才,35,特例(一元线性回归模型),当p=1时,浙江财经学院 倪伟才,36,性质 3:D()=2(X X)-1的意义:,浙江财经学院 倪伟才,37,Calculating Parameter and Standard Error Estimates for Multiple Regression Models,Example:The following model with k=3 is estimated over 15 observations:and the following data have been calculated from the original Xs.Calculat

15、e the coefficient estimates and their standard errors.To calculate the coefficients,just multiply the matrix by the vector to obtainTo calculate the standard errors,we need an estimate of 2.,浙江财经学院 倪伟才,38,(contd),The variance-covariance matrix of is given by The variances are on the leading diagonal

16、:We write:,浙江财经学院 倪伟才,39,性质4:高斯马尔可夫定理,Gauss-Markov theorem:在高斯马尔可夫条件下,即E()=0,E()=2I,在的所有线性无偏估计中,由最小二乘法得到的估计值 的方差最小.(即BLUE)注可能存在非线性函数(指的是y1,y2,y n的函数),是无偏估计,但它的方差比由最小二乘法得到的估计值 的方差要小可能存在有偏估计,它的方差比由最小二乘法得到的估计值 的方差要小本定理的一个前提是在的线性,无偏估计中.本定理的证明采用矩阵形式.详细过程请参考Econometric Models and Economic ForecastsPindyck

17、Appendix 4.3 The Multiple Regression Model in Matrix Form 该书110,111页,此种证明方法较繁琐！建议采用Greene Econometric Analysis的方法！,浙江财经学院 倪伟才,40,Greene的方法！（要求掌握!）,浙江财经学院 倪伟才,41,Cond,浙江财经学院 倪伟才,42,注解：,Gauss-Markov theorem的证明可以参考 James H.Stock,Mark W.Watson Introduction to EconometricsAPPENDIX 16.5,浙江财经学院 倪伟才,43,参数估计

18、量的性质,性质5 cov（,，e）=0,此性质说明 与e不相关,在正态假定下等价于与e独立,从而与 独立。,性质6 在正态假设,(1),(2),浙江财经学院 倪伟才,44,3.4 回归方程的显著性检验,浙江财经学院 倪伟才,45,M0 很方便的记号！,浙江财经学院 倪伟才,46,M0 性质,浙江财经学院 倪伟才,47,SST,SSR,SSE,浙江财经学院 倪伟才,48,SST=SSR+SSE,浙江财经学院 倪伟才,49,请用矩阵计算重点是3种平方和,浙江财经学院 倪伟才,50,Stata,例：数据：chap05pr04.dta gen one=1mkmat one x,mat(x)mkmat

19、y,mat(y)mat b=inv(x*x)*x*ymat p=x*inv(x*x)*xmat m=I(5)-pmat yhat=p*ymat e=m*ymat i=J(5,1,1)mat list imat m0=I(5)-i*inv(i*i)*imat list m0,浙江财经学院 倪伟才,51,Cond,mat ssr=e*emat sse=yhat*m0*yhatmat sst=y*m0*ymat list ssemat list ssrmat list sstreg y x具体的输出结果请参考：3种平方和的矩阵计算.doc练习：Applied Linear Regression Mo

20、delschapter5 problems5.24数据：chap05pr21.dta,浙江财经学院 倪伟才,52,样本决定系数 R2=SSE/SST=1 SSR/SSTR2 measures the proportion of variation in Y which is explained by the multiple regression equation.R2 is often used informally as a goodness of fit statistic and to compare the validity of regression results under a

21、lternative specifications of the independent variables in the model.However,there are several problems with the use of R2.First,all our statistic results follow from the initial assumption that the model is correct;we have no procedure that compares alternative specifications.Second,R2 is sensitive

22、to the number of independent variables included in the regression model.The addition of more independent variables to the regression equation can never lower R2 and is likely to raise it.(The addition of a new explanatory variable does not alter SST but is likely to increase SSE.)Thus,one could simp

23、ly add more variables to an equation if one wished only to maximize R2.,浙江财经学院 倪伟才,53,Adjusted R2 要掌握！,The difficulty with R2 as a measure of goodness of fit is that R2 pertains only to explained and unexplained variation in Y and therefore does not account for the number of degree of freedom.A natu

24、ral solution is to use variances,not variations,thus eliminating the dependence of goodness of fit in the number of independent variables in the model.,浙江财经学院 倪伟才,54,Adjusted R2性质,浙江财经学院 倪伟才,55,三.统计量：回归方程总体显著性的检验,浙江财经学院 倪伟才,56,Cond,浙江财经学院 倪伟才,57,联合排除性约束的F检验 很重要，务必掌握,浙江财经学院 倪伟才,58,联合排除性约束的F检验的公式记住,浙江

25、财经学院 倪伟才,59,联合排除性约束的F检验和一般F检验的关系,一般F检验实际上就是联合排除性约束的F检验的特例！,浙江财经学院 倪伟才,60,一道有趣的题目：Wooldridge question4.5,浙江财经学院 倪伟才,61,练习,Consider Patient satisfaction chap06pr15.dta1:Test whether X3 can be dropped from the regression model given that X1 and X2 are retained.Use F test statistic and level of signific

26、ance 0.05.State the alternatives,decision rule,and conclusion.What is the P-value of the test?2:Test whether 1=-1 and 2=0.State the alternatives,full model and reduced model,decision rule,and conclusion.What is the P-value of the test?abstracted from Applied Linear Regression Models Problems 7.5 and

27、 7.9,浙江财经学院 倪伟才,62,Stata chap06pr15.dta,1:reg y x1 x2 x3 test x3di 3.600.51.89736662:quireg y x1 x2 x3 test x1=-1F(1,42)=0.43 Prob F=0.5133test x2=0,accumulateF(2,42)=0.88 Prob F=0.4208,浙江财经学院 倪伟才,63,四.t统计量：个别回归系数的显著性检验,浙江财经学院 倪伟才,64,五.讲解课本例3.1(12元.dta),注：全体12个自变量做为整体对y有显著性的线性关系；但每一个自变量对y没有显著性的线性关系。

28、：对于多元回归而言，回归方程总体性的显著性F检验和回归系数的个别显著性的t检验不同；原因在于多重共线性。如何才能使每一个变量都对y具有显著性影响：方法：剔除多余变量，一个一个剔除，先剔除p值最大的，进行检验，依次进行，直到所有的变量对y的影响都是显著的（即每个p值均小于）.,浙江财经学院 倪伟才,65,Stata for 课本例3.1,数据:12元.dtareg y x*reg y x2-x12reg y x3-x12reg y x3-x11 reg y x3 x5-x11 reg y x3 x5 x6 x8-x11reg y x3 x5 x8-x11 reg y x3 x8-x11reg y

29、 x3 x8 x9 x11,浙江财经学院 倪伟才,66,六.对F显著，t不显著的直观解释,y:the total travel time;X1:the number of miles traveled;X2:the number of gallons of gasoline consumed.Assume that we obtain the equation and find that the F test shows the relationship to be significant.Then suppose we conduct a test on 1 to determine whe

30、ther 1 0，and we cannot reject H0:1=0.Does this mean that travel time y is not related to miles traveled x1?,浙江财经学院 倪伟才,67,Cond,Not necessarily.What it probably means is that with x2 already in the model,x1 does not make a significant contribution to determining the value of y.This interpretation mak

31、es sense in our example:if we know the amount of gasoline consumed,we do not gain much additional information useful in predicting y by knowing the miles traveled.Similarly,a t test might lead us to conclude 2=0 on the grounds that,with x1 in the model,knowledge of the amount of gasoline consumed do

32、es not add much.,浙江财经学院 倪伟才,68,3.5 标准化系数,请参考Applied Linear Regression Modelschapter7,7.5部分,Standardized Multiple Regession Model,Purpose:A standardized form of the general multiple regression model is employed to control round off errors in normal equations calculations and to permit comparisons of

33、the estimated regression coefficients in common units.,浙江财经学院 倪伟才,69,Round off Errors in calculations,The results from normal equations calculations can be sensitive to rounding of data in intermediate stages of calculations.When the number of X variables is small-say,three or less-round off effects

34、 can be controlled by carrying a sufficient number of digits in intermediate calculations.Indeed,most computer regression programs use double-precision arithmetic in all computations to control round off effects.Still,with a large number of X variables,serious round off effects can arise despite the

35、 use of many digits in intermediate calculations.,浙江财经学院 倪伟才,70,Cond,Round off errors tend to enter calculations primarily when the inverse of XX is taken.The danger of serious round off errors in(XX)-1 is particularly great when(1)XX has a determinant that is close to zero and/or(2)the elements of

36、XX differ substantially in order to magnitude.The first condition arises when some or all of the X variables are highly intercorrelated.The second condition arises when the X variables have substantially different magnitudes so that the entries in the XX matrix cover a wide range,say,from 15 to 4900

37、0000,浙江财经学院 倪伟才,71,Lack of Comparability in Regression Coefficients,例：考虑二元回归方程,从表面上看，2000比2大，误解为x1对y的平均影响要比x2对y的平均影响要大,复习回归系数的实际意义,联系：可口可乐：比2升多送250毫升！其它的：如买一送三。,浙江财经学院 倪伟才,72,2000表示x1增加1吨，y平均增加2000个单位；,实际上，x1，x2对y的平均影响是相同的。,x1，x2的单位不同，x1的单位为吨，x2的单位为公斤,2表示x2增加1公斤，y平均增加2个单位即：表示x2增加1吨，y平均增加2000个单位,浙江财经

38、学院 倪伟才,73,Why to transform variables,The transformation to obtain the standardized regression model,called the correlation transformation,makes all enries in the XX matrix for the transformed variables fall between-1 and 1 inclusive,so that the calculation of the inverse matrix becomes much less sub

39、ject to round off errors due to dissimilar orders of magnitudes than with the original variables.,浙江财经学院 倪伟才,74,Correlation Transformation,The correlation transformation is a simple modification of the usual standardization of a variable.The correlation transformation is a simple function of the sta

40、ndardized variables,浙江财经学院 倪伟才,75,变量标准化,浙江财经学院 倪伟才,76,Formula for correlation transformation,浙江财经学院 倪伟才,77,Standardized Regression Model,The regression model with the transformed y*and xk*as defined by the correlation transformation formula is called a standardized regression model and is as follows

41、:The reason why there is no intercept parameter in the standardized regression model is that the least squares calculations always would lead to an estimated intercept term of zero if an intercept parameter were present in the model.,浙江财经学院 倪伟才,78,Relation,It is easy to show that the parameters*0,*1

42、,.,*p in the standardized regression model and the original parameters 0,1,.,p in the ordinary multiple regression model are related as follows:,We see that the standardized regression coefficients*k and the original regression coefficients k are related by simple scaling factors involving ratios of

43、 standard deviations,浙江财经学院 倪伟才,79,标准化系数的推导,浙江财经学院 倪伟才,80,In order to compare the importance of the independent variables in determining the dependent variable,the standardized coefficients are more appropriate for the purpose.,浙江财经学院 倪伟才,81,标准化回归系数与普通最小二乘回归系数之间的关系：,标准化回归方程：,注意：标准化回归方程回归常数项为0!,浙江财经学

44、院 倪伟才,82,求标准化系数的步骤,1：先将y,x1,x2,x p 标准化,2：然后将y的z 得分对x1,x2,x p的z 得分进行回归,3：最后根据回归方程得到标准化系数,浙江财经学院 倪伟才,83,例：污染对住房价格的影响（Wooldridge p175 example 6.1）（数据见hprice2.sav）,被解释变量：price;解释变量：nox（氧化亚氮）,crime,rooms,dist,stratio（学生和教师的比例）,例题讲解(spss),考虑price关于nox,crime,room,dist，stratio 的多元线性回归,浙江财经学院 倪伟才,84,结合例子回归方程

45、标准化的3种方法：,首先descriptive statistics-descriptives options-mean,std.deviation;然后利用compute 产生各个变量的z得分；最后进行回归.(该方法主要为了熟悉标准化的具体过程！),利用regression直接得到！,利用descriptives-save standardized values as variables直接产生z得分，再进行回归.(该方法可以马上得到z得分！),浙江财经学院 倪伟才,85,(3)根据结果，在环境污染、犯罪率、住房的大小的3个自变量中对价格影响最大的是哪种因素？最小呢？,如果想知道每个自变量对

46、平均住房价格的美元价值的影响，那就应该使用未经标准化的变量。,思 考：,提 醒,(1)写出标准化回归方程：,(2)标准化回归方程中的标准化系数的符号与实际一致吗？,浙江财经学院 倪伟才,86,Stata,data:chap07ta05.dtaegen ys=std(y)egen x1s=std(x1)egen x2s=std(x2)regress ys x1s x2s,noconstantregress y x1 x2,beta,浙江财经学院 倪伟才,87,练习,Consider Patient satisfaction chap06pr15.dta1:Transform the variab

47、les by means of the correlation transformation formula and fit the standardized regression model.2:Transform the estimated standardized regression coefficients back to the ones for fitted regression model in the original variables.abstracted from Applied Linear Regression Models Problems 7.18,浙江财经学院

48、 倪伟才,88,Stata for 练习chap06pr15.dta,egen ys=std(y)egen x1s=std(x1)egen x2s=std(x2)egen x3s=std(x3)reg ys x1s x2s x3s,nocons sum y x1 x2 x3di-.5906664*17.23646/8.918092 di-.1106149*17.23646/4.313556di-.2339312*17.23646/.2993391reg y x1 x2 x3,beta,浙江财经学院 倪伟才,89,XX matrix for transform variables,其中r i j

49、为xi,x j之间的简单相关系数,浙江财经学院 倪伟才,90,Cont.,We consider the XX matrix for the transformed variables.The X variable here is:,Remember that the standardized regression model does not contain an intercept term;hence,there is no column of 1s in the X matrix.,浙江财经学院 倪伟才,91,Formula:XX=rxx,It can be shown that th

50、e XX matrix for the transformed variables is simply the correlation matrix of the X variables.XX=rxxSince the XX matrix for the transformed variables consists of coefficients of correlation between the X variables,all of its elements are between 1 and 1 and thus are of the same order of magnitude.As