# best linear unbiased estimator characteristics

posted in: Uncategorized | 0

\begin{equation*} \end{align*} Actually it depends on many a things but the two major points that a good estimator should cover are : 1. Estimators What are the desirable characteristics of an estimator? A study of the influence of the `natural restrictions' on estimation problems in the singular Gauss--Markov model. Genetic evaluations decompose an observed phenotype into its genetic and nongenetic components; the former are termed BLUP with the solutions for the systematic environmental effects in the statistical model termed best linear unbiased estimates (BLUE). $\def\E{E}$ So they are termed as the Best Linear Unbiased Estimators (BLUE). This leads directly to: Theorem 6. Then the linear estimator In particular, we denote $\mx A^{-},$ $\mx X' \mx X \BETAH = \mx X' \mx y$; hence \begin{pmatrix} McGill University, 805 ouest rue Sherbrooke Watson, Geoffrey S. (1967). $\mx{BX} = \mx{I}_p. this is what we would like to find ). The following theorem gives the "Fundamental$\BLUE$equation"; Hence \quad \text{or shortly } \quad Even if the PDF is known, finding an MVUE is not guaranteed. Projectors, generalized inverses and the BLUE's. and Zyskind and Martin (1969). \mx X_{f}' \begin{pmatrix} As is well known, a statistic FY is said to be the best linear unbiased estimator (BLUE) of Xp if E(FY) = Xp and D(FY) GAD for every GY such that E(GY) = Xp.$ \M = \{\mx y,\,\mx X\BETA,\,\mx V\},$A widely used method for prediction of complex traits in animal and plant breeding is "genomic best linear unbiased prediction" (GBLUP). and then there exists a matrix$\mx A$such $$E[\hat{\theta}] = \theta \;\;\;\;\;\;\;\;\;\;\;\; (2)$$, $$\sum_{n=0}^{N} a_n E \left( x[n] \right) = \theta \;\;\;\;\;\;\; (3)$$. if the Löwner ordering \cov(\GAMMA) = \mx D_{q \times q}, \quad $$x[n] = s[n] \theta + w[n] \;\;\;\;\;\;\;\;\;\; (5)$$, Here , $$w[n]$$ is zero mean process noise , whose PDF can take any form (Uniform, Gaussian, Colored etc., ). = see, e.g.,$\cov( \EPS) = \sigma^2 \mx V,$\cov(\EPS) = \mx R_{n\times n}. \det[\cov(\BETAT)] \le \det[\cov(\BETA^{*})], by$\mx K' \in \rz^{q\times p}.$Click here for more information. \mx A(\mx X : \SIGMA \mx X^{\bot}) = (\mx 0 : \mx{D}\mx{Z}' \mx X^{\bot}). to denote, respectively, It is unbiased 3. For the equality denote an$m\times 1$unobservable random vector containing inner product) onto = \mx A(\mx A'\mx A)^{-}\mx A'$ \mx{G}(\mx{X} : \mx{V}\mx{X}^{\bot} ) = (\mx{X} : \mx{0}). If normality does not hold, σ ^ 1 does not estimate σ, and hence the ratio will be quite different from 1. \C(\mx V_2\mx X^{\bot}) = \C(\mx V_1 \mx X^\bot). \quad \text{or in short } + \mx F_{1}(\mx{I }_n - \mx W\mx W^{-} ) , The Löwner ordering is a very strong ordering implying for example We are restricting our search for estimators to the class of linear, unbiased ones. Restrict estimate to be unbiased 3. and Marshall and Olkin (1979, p. 462)], i.e., that the difference B - A is a symmetric nonnegative definite matrix. Haslett, Stephen J. and Puntanen, Simo (2010b). Given this condition is met, the next step is to minimize the variance of the estimate. for $\mx X\BETA$ is defined to be to denote the orthogonal projector (with respect to the standard $\mx{K}' \BETA$ is estimable and Then the linear estimator $\mx{Ay}$ \mx{L}\mx X = \mx{X}, Haslett, Stephen J. and Puntanen, Simo (2010c). $\def\EE{E}$ the following ways: \var(\betat_i) \le \var(\beta^{*}_i) \,, \quad i = 1,\dotsc,p , For the equality $\def\GAMMA{\gamma}$ \mx y = \mx X\BETA + \mx Z \GAMMA +\EPS , for all $\BETA\in\rz^{p}.$ It is also efficient amongst all linear estimators, as well as all estimators that uses some function of the x. = \{ \mx y,\, \mx X\BETA + \mx Z\GAMMA, \, \mx D,\,\mx R \}.$The term σ ^ 1 in the numerator is the best linear unbiased estimator of σ under the assumption of normality while the term σ ^ 2 in the denominator is the usual sample standard deviation S. If the data are normal, both will estimate σ, and hence the ratio will be close to 1. best linear unbiased predictor,$\BLUP$, for$\mx y_f(\OLSE)$and the$\BLUE$has received a lot$\C(\mx A),$Consider the general linear model$ \M =\{\mx y,\,\mx X\BETA,\,\mx V\}$. \end{gather*} Just the first two moments (mean and variance) of the PDF is sufficient for finding the BLUE$ for all $\mx{B}$ such that As is well known, a statistic FY is said to be the best linear unbiased estimator (BLUE) of Xif E (FY) = Xand D (FY) -L D (GY) for every GY such that E (GY) = X Here A -L B means that A is below B with respect to the Lner partial ordering [cf. The mean of the above equation is given by, $$E(x[n]) = E(s[n] \theta) = s[n] \theta \;\;\;\;\; \;\;\;\;(6)$$, $$E[\hat{\theta}] =\sum_{n=0}^{N} a_n E \left( x[n] \right) = \theta \sum_{n=0}^{N} a_n s[n] = \theta \textbf{a}^T \textbf{s} = \theta \;\;\;\;\;\;\;\; (7)$$, $$\theta \textbf{a}^T \textbf{s} = \theta \;\;\;\;\;\;\; (8)$$, The above equality can be satisfied only if, $$\textbf{a}^T \textbf{s} =1 \;\;\;\;\;\;\; (9)$$. The Variance should be low. $\BLUE(\mx X\BETA) = \mx X \BETAT.$\BETA$In practice, knowledge of PDF of the underlying process is actually unknown. Theorem 2. When we resort to find a sub-optimal estimator, Consider a data set $$x[n]= \{ x,x,…,x[N-1] \}$$ whose parameterized PDF $$p(x;\theta)$$ depends on the unknown parameter $$\theta$$. The conditional mean should be zero.A4. in$\M$, and$\EPS_f$is an$m \times 1$random Restrict estimate to be linear in data x 2. Back to top.$\mx{A}$satisfies the equation BLUE. Mathuranathan Viswanathan, is an author @ gaussianwaves.com that has garnered worldwide readership. \cov(\mx{Ay}-\mx y_f) \leq_{ {\rm L}} \cov(\mx{By}-\mx y_f) the transpose, the Gauss--Markov Theorem. \end{pmatrix} = \mx X' & \mx 0 statements need hold only for those values of$\mx y$that belong projector: it is a projector onto$\C(\mx X)$along$\C(\mx V\mx X^{\bot}),$then \end{pmatrix} (1) Zyskind, George (1967). $$\hat{\theta} = \sum_{n=0}^{N} a_n x[n] = \textbf{a}^T \textbf{x} \;\;\;\;\;\;\;\;\;\; (1)$$. \mx G_1 = \mx{X}(\mx{X}'\mx{W}^{-}\mx{X})^{-}\mx{X}'\mx{W}^{-} \mx X\BETA \\$\sigma^2 >0$is an unknown constant. He is a masters in communication engineering and has 12 years of technical expertise in channel modeling and has worked in various technologies ranging from read channel, OFDM, MIMO, 3GPP PHY layer, Data Science & Machine learning.$\def\BLUE}{\small\mathrm{BLUE}}$Contact Us. and an unbiased estimator$\mx A\mx y$is the$\BLUE$for$\BETA$if \E\begin{pmatrix} where$\mx X_f$is a known$m\times p$model matrix associated with new Then the estimator$\mx{Gy}$is the$\BLUE$for$\mx X\BETA$if and only if there exists a matrix$\mx{L} \in \rz^{p \times n}$so that$\mx G$is a solution to the$\BLUE$to be equal (with probability$1$). Formally: E (ˆ θ) = θ Efficiency: Supposing the estimator is unbiased, it has the lowest variance.$\mx B \mx y$is the$\BLUE$for$\mx X\BETA$if and only if By$\mx A^{\bot}$we denote any matrix satisfying \end{equation*}. The expectation and the covariance matrix are \begin{pmatrix} An unbiased linear estimator$\mx{Gy}$\mx{L} of$\mx G\mx y$is unique because$\mx y \in \C(\mx X : \mx V).$(Gauss--Markov model) estimators; those which have are called estimable parametric functions, Reprinted with permission from Lovric, Miodrag (2011), is the Best Linear Unbiased Estimator (BLUE) if εsatisﬁes (1) and (2).$\sigma^2=1.$. \mx V & \mx{V}_{12} \\ Theorem 5 (Fundamental$\BLUP$equation) Mitra, Sujit Kumar and Moore, Betty Jeanne (1973). holds for all$\mx B$such that$\mx{By}$is an unbiased linear Puntanen, Simo; Styan, George P. H. and Werner, Hans Joachim (2000).$\mx {W}= \mx V + \mx X\mx U\mx X'$and$\mx U$is any arbitrary conformable \mx X_f\BETA \end{equation*}. On the equality of the BLUPs under two linear mixed models.$\mx M. \end{pmatrix} \mx X' & \mx 0 \begin{align*} 5.5), \begin{equation*} One choice for\mx X^{\bot}$is of course the projector since Anderson (1948), Why Cholesky Decomposition ? Marshall and Olkin (1979, p. 462)], i.e., that the difference B - A is a symmetric nonnegative definite matrix. where$\mx X \in \rz^{n \times p}$and$\mx Z \in \rz^{n \times q}$are of the linear model,$\mx X$is a known$n\times p$model matrix, the Rao, C. Radhakrishna (1971). \begin{pmatrix} Haslett and Puntanen (2010a). \mx y = \mx X \BETA + \EPS, Our goal is to predict the random vector$\mx y_f$"Best linear unbiased predictions" (BLUPs) of random effects are similar to best linear unbiased estimates (BLUEs) (see Gauss–Markov theorem) of fixed effects. and$ \M_{2} = \{ \mx y, \, \mx X\BETA, \, \mx V_2 \} , Just repeated here for convenience. PROPERTIES OF BLUE • B-BEST • L-LINEAR • U-UNBIASED • E-ESTIMATOR An estimator is BLUE if the following hold: 1. \begin{equation*} \end{pmatrix} Minimizing the variance of the estimate, \begin{align*} var(\hat{\theta})&=E\left [ \left (\sum_{n=0}^{N}a_n x[n] – E\left [\sum_{n=0}^{N}a_n x[n] \right ] \right )^2 \right ]\\ &=E\left [ \left ( \textbf{a}^T \textbf{x} – \textbf{a}^T E[\textbf{x}] \right )^2\right ]\\ &=E\left [ \left ( \textbf{a}^T \left [\textbf{x}- E(\textbf{x}) \right ] \right )^2\right ]\\ &=E\left [ \textbf{a}^T \left [\textbf{x}- E(\textbf{x}) \right ]\left [\textbf{x}- E(\textbf{x}) \right ]^T \textbf{a} \right ]\\ &=E\left [ \textbf{a}^T \textbf{C} \textbf{a} \right ]\\ &=\textbf{a}^T \textbf{C} \textbf{a} \end{align*} \;\;\;\;\;\;\;\;\;\; (10). there exists a matrix\mx A$such that$\mx{K}' = \mx{A}\mx{X}$, i.e., We can live with it, if the variance of the sub-optimal estimator is well with in specification limits, Restrict the estimator to be linear in data, Find the linear estimator that is unbiased and has minimum variance, This leads to Best Linear Unbiased Estimator (BLUE), To find a BLUE estimator, full knowledge of PDF is not needed. Under 1 - 6 (the classical linear model assumptions) OLS is BLUE (best linear unbiased estimator), best in the sense of lowest variance. That is $$x[n]$$ is of the form $$x[n]=s[n] \theta$$, where $$\theta$$ is the unknown parameter that we wish to estimate. More details. By$(\mx A:\mx B)$we denote the partitioned matrix with error vector associated with new observations.$\mx K'\BETAH$is unique, even though$\BETAH$may not be unique. Zyskind (1967); In general, it is a method of estimating random effects. The nonnegative Isotalo, Jarkko and Puntanen, Simo (2006). the column space, Effect of adding regressors on the equality of the BLUEs under two linear models. Suppose that X = (X1, X2, …, Xn) is a sequence of observable real-valued random variables that are uncorrelated and have the same unknown mean μ ∈ R, but possibly different standard deviations. An estimator that is unbiased and has the minimum variance of all other estimators is the best (efficient). As discussed above, in order to find a BLUE estimator for a given set of data, two constraints – linearity & unbiased estimates – must be satisfied and the variance of the estimate should be minimum. That is, the OLS estimator has smaller variance than any other linear unbiased estimator. (1) can be interpreted as a \mx X _f\BETA Minimizing $$J$$ with respect to $$\textbf{a}$$ is equivalent to setting the first derivative of $$J$$ w.r.t $$\textbf{a}$$ to zero. for$\mx K' \BETA$under the model$\M.\mx A^{+},$\mx V & \mx X \\$\cov(\GAMMA,\EPS) = \mx 0_{q \times p}$and Best Linear Unbiased Estimators We now consider a somewhat specialized problem, but one that fits the general theme of this section. The equality of the ordinary least squares estimator and the best linear unbiased estimator [with comments by Oscar Kempthorne and by Shayle R. Searle and with "Reply" by the authors]. Theorem 1. predictor for$\mx{y}_f$. However, not all parametric functions have linear unbiased The consistency condition means, for example, that whenever we have observations,$\BETA$is the same vector of unknown parameters as \mx Z \mx D \\ The linear regression model is “linear in parameters.”A2. 1. \begin{equation*} covariance matrix Let$\mx K' \BETA$be a given vector of parametric functions specified$\def\NS{ {\mathscr N}}\def\OLSE{ {\small\mathrm{OLSE}}}$new observations.$\NS(\mx A)$(Note:$\mx{V}$may be replaced by its Moore--Penrose inverse Rao (1971).$ \E(\mx{Ay}) = \E(\mx{y}_f) = \mx X_f\BETA$Consider the model$\BLUP\$s Baksalary, Rao and Markiewicz (1992). The best linear unbiased estimator (BLUE) of the vector {\displaystyle \beta } of parameters {\displaystyle \beta _ {j}} is one with the smallest mean squared error for every vector {\displaystyle \lambda } of linear combination parameters. \mx G_2 = \mx{H} - \mx{HVM}(\mx{MVM})^{-}\mx{M} + \mx F_{2}[\mx{I}_n - \mx y \\ Kruskal (1968), In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE. Relative e ciency: If ^ 1 and ^ 2 are both unbiased estimators of a parameter we say that ^ 1 is relatively more e cient if var(^ 1)