6626070
2997924

MATH05, Parameters estimation

Back to the previous page ｜ Statistics
List of posts to read before reading this article

Contents

Point estimation : Method of moment

Asumption : parameter is determined by sample statistic(population moment = sample moment)
Weekness : it is difficult to determine whether a parameter is accurate
Weekness : it's hard to figure out the confidence level and interval




estimate for beta distribution

from scipy import stats
import numpy as np

def estimate_beta(x):
    x_bar = x.mean()
    s2 = x.var()
    a = x_bar * (x_bar * (1 - x_bar) / s2 - 1)
    b = (1 - x_bar) * (x_bar * (1 - x_bar) / s2 - 1)
    return a, b
    
np.random.seed(0)
x = stats.beta(15, 12).rvs(10000)
estimate_beta(x)

(15.346682046700685, 12.2121537049535)


Point estimation : Maximum Likelihood Estimation(MLE)

Caution : parameter ≠ random variable
Weekness : it's hard to figure out the confidence level and interval




likelihood for normal distribution about single sample data

$$given\ dataset\ x\ :\ \{0\}$$ $$given\ parameter\ \sigma \ =\ 1$$ $$likelihood\ :\ L(\theta = \mu ;x) = \frac{1}{\sqrt{2 \pi \sigma}}e^{-\frac{(\mu-x)^{2}}{2\sigma^{2}}}$$

for mu

import numpy as np
from scipy import optimize

# objective function(= -likelihood)
def objective(MU, SIGMA2=1):
    return - np.exp(-(MU) ** 2 / (2 * SIGMA2)) / np.sqrt(2 * np.pi * SIGMA2)

# optimize
optimize.brent(objective, brack=(-10,10))

optimal point for mu = 1.3008039364016205e-11

Visualization 
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np

# range of mu and sigma^2 at x = 0
x = 0
mus = np.linspace(-5, 5, 1000)
sigma2s = np.linspace(0.1, 10, 1000)
MU, SIGMA2 = np.meshgrid(mus, sigma2s)

# likelihood
L = np.exp(-(MU-x) ** 2 / (2 * SIGMA2)) / np.sqrt(2 * np.pi * SIGMA2)

# plot
fig = plt.figure()
ax = Axes3D(fig)
ax = fig.gca(projection='3d')
ax.plot_surface(MU, SIGMA2, L, linewidth=0.1)
plt.xlabel('$\mu$')
plt.ylabel('$\sigma^2$')
plt.title('likelihood $L(\mu, \sigma^2)$')
plt.show()

download




likelihood for normal distribution about multi-sample data

$$given\ dataset\ x\ :\ \{-3,\ 0,\ 1\}$$ $$given\ parameter\ \sigma \ =\ 1$$ $$likelihood\ :\ L(\theta = \mu ;x) = \frac{1}{(\sqrt{2 \pi \sigma})^{3/2}}e^{-\frac{3(\mu+\frac{2}{3})^{2}+\frac{26}{3}}{2\sigma^{2}}}$$ 
import numpy as np
from scipy import optimize

def objective(mu, sigma2=1):
    return - (2 * np.pi * sigma2) ** (3 / 2) * np.exp(-(3 * mu ** 2 + 4 * mu + 10) / (2 * sigma2))

# optimize
optimize.brent(objective, brack=(0,1))

optimal point for mu = -0.6666666665981932


Interval estimation : Bayesian estimation

Asumption : parameter = random variable

List of posts followed by this article

Reference

OUTPUT 
<details markdown="1">
<summary class='jb-small' style="color:red">OUTPUT</summary>
<hr class='division3_1'>
<hr class='division3_1'>
</details>