Density Estimation by Histograms (Part III)

Before to continue with today’s post we will answer the question of last week, Is it $latex {\hat{f}_{h}(x)}&fg=000000$ a consistent estimator? The answer is yes. Because convergence in mean squared implies convergence in probability.

Continuing the ideas of Part II , we arrive into a problem applying the Mean Squared Error. Recall that

$latex \displaystyle \displaystyle \mathrm{MSE}(\hat{f}_{h}(x))= \frac{1}{nh}f(x) + f^{\prime}(x)^2\left(h\left(j-\frac{1}{2}\right)-x\right)^2+o\left(\frac{1}{nh}\right)+o(h), \ \ \ \ \ (1)&fg=000000$


which depends directly on the unknown function $latex {f}&fg=000000$ in every point. Instead, it is worthwhile have a global measure of accuracy. One of the most used are the Mean Integrated Squared Error (MISE):

$latex \displaystyle \begin{array}{rcl} \mathrm{MISE}(\hat{f}_h) &=& \displaystyle \mathop{\mathbb E} \left(\int_{-\infty}^\infty \{\hat{f}_h(x)-f_h(x)\}^2 dx\right) \\ &=& \displaystyle \int_{-\infty}^\infty \mathop{\mathbb E} \left( (\hat{f}_h(x)-f_h(x))^2 dx\right) \\ &=& \displaystyle \int_{-\infty}^\infty \mathrm{MSE}(\hat{f}_h(x)) dx \end{array} &fg=000000$

Using (1) we can write for any $latex {x}&fg=000000$,

$latex \displaystyle \begin{array}{rcl} \mathrm{MISE}(\hat{f}_h) & =& \displaystyle \int\frac{1}{nh}f(x)dx + \int \sum_j I(x\in B_j) f^{\prime}(x)\left(h\left(j-\frac{1}{2}\right)-x\right)dx\\ & =& \displaystyle \frac{1}{nh} + \sum_j \int_{B_j} f^{\prime}(x)^2\left(h\left(j-\frac{1}{2}\right)-x\right)^2dx\\ & \approx& \displaystyle \frac{1}{nh} + \sum_j \int_{B_j} f^{\prime}(x)^2 dx \int_{B_j} \left(h\left(j-\frac{1}{2}\right)-x\right)^2dx\\ & =& \displaystyle \frac{1}{nh} + \frac{h^2}{12} \int f^{\prime}(x)^2 dx \\ & =& \displaystyle \frac{1}{nh} + \frac{h^2}{12} \|f^\prime \|_2^2 \text{ for } h\rightarrow 0. \\ \end{array} &fg=000000$

Here, $latex {\|f^\prime \|_2^2}&fg=000000$ denotes the squared $latex {L_2}&fg=000000$-norm of the function $latex {f^\prime}&fg=000000$. Now, the Asymptotic MISE (AMISE), is given by

$latex \displaystyle \displaystyle \mathrm{AMISE}(\hat{f}_h) = \frac{1}{nh} + \frac{h^2}{12} \|f^\prime \|_2^2. &fg=000000$

Optimal Binwidth

We can choose an optimal value of $latex {h}&fg=000000$ differentiating the AMISE with respect to $latex {h}&fg=000000$,

$latex \displaystyle \displaystyle \frac{\partial \mathrm{AMISE}(\hat{f}_h) }{\partial h} = \frac{-1}{nh^2} + \frac{1}{6}h\|f^\prime \|_2^2=0, &fg=000000$


$latex \displaystyle \displaystyle h_{opt}= \left(\frac{6}{n\|f^\prime \|_2^2}\right)\approx n^{-1/3}, &fg=000000$

where $latex {h_{opt}}&fg=000000$ es the optimal bindwidth. Of course we could have some problems trying to calculate $latex {h_{opt}}&fg=000000$ because $latex {\|f^\prime \|_2^2}&fg=000000$ is still an unknown value.

Question for you: What is exactly the $latex {h_{opt}}&fg=000000$ if we assume that

$latex \displaystyle \displaystyle f(x) = \frac{1}{\sqrt{2\pi}} \text{exp}\left(\frac{-x^2}{2}\right)? &fg=000000$

The next week we will do some simulations to show this theory in practice.



  1. Pingback: Density Estimation by Histograms (Part IV) | Blog about Statistics.

  2. Pingback: Kernel density estimation | Blog about Statistics.

  3. Pingback: Optimizing the binwidth for the histogram using cross-validation | Maximum Entropy

  4. Pingback: Optimizing the binwidth for the histogram using cross validation - Maikol Solís

Leave a Reply