京都大学 情報学研究科 システム科学専攻 2017年8月実施 専門科目 確率統計 Author uogxtc , 祭音Myyura
Description 問題1
確率変数 X 1 , … , X n , Y 1 , … , Y m X_1, \ldots, X_n, Y_1, \ldots, Y_m X 1 , … , X n , Y 1 , … , Y m X i ∼ N ( a θ , σ 2 ) X_i \sim N(a\theta, \sigma^2) X i ∼ N ( a θ , σ 2 ) Y j ∼ N ( b θ , σ 2 ) Y_j \sim N(b\theta, \sigma^2) Y j ∼ N ( b θ , σ 2 ) i = 1 , … , n i = 1, \ldots, n i = 1 , … , n j = 1 , … , m j = 1, \ldots, m j = 1 , … , m N ( μ , σ 2 ) N(\mu, \sigma^2) N ( μ , σ 2 ) μ \mu μ σ 2 \sigma^2 σ 2 n , m n, m n , m a , b a, b a , b θ , σ 2 \theta, \sigma^2 θ , σ 2
(1) θ , σ 2 \theta, \sigma^2 θ , σ 2 X 1 , … , X n , Y 1 , … , Y m X_1, \ldots, X_n, Y_1, \ldots, Y_m X 1 , … , X n , Y 1 , … , Y m
(2) 定数 α , β \alpha, \beta α , β θ ~ = α X ˉ + β Y ˉ \tilde{\theta} = \alpha \bar{X} + \beta \bar{Y} θ ~ = α X ˉ + β Y ˉ X ˉ = ( X 1 + ⋯ + X n ) / n \bar{X} = (X_1 + \cdots + X_n)/n X ˉ = ( X 1 + ⋯ + X n ) / n Y ˉ = ( Y 1 + ⋯ + Y m ) / m \bar{Y} = (Y_1 + \cdots + Y_m)/m Y ˉ = ( Y 1 + ⋯ + Y m ) / m θ ~ \tilde{\theta} θ ~ E ( θ ~ ) E(\tilde{\theta}) E ( θ ~ ) V ( θ ~ ) V(\tilde{\theta}) V ( θ ~ )
(3) θ ~ \tilde{\theta} θ ~ θ \theta θ α , β \alpha, \beta α , β θ ~ \tilde{\theta} θ ~ V ( θ ~ ) V(\tilde{\theta}) V ( θ ~ ) α , β \alpha, \beta α , β
問題2
あるコインを投げると、確率 p ( 0 < p < 1 ) p \ (0 < p < 1) p ( 0 < p < 1 ) q ( = 1 − p ) q \ (= 1 - p) q ( = 1 − p ) T T T
(1) T = n T = n T = n n = 1 , 2 , … n = 1, 2, \ldots n = 1 , 2 , … P ( T = n ) P(T = n) P ( T = n ) P ( ⋅ ) P(\cdot) P ( ⋅ )
(2) 確率変数 T T T
あるスロットマシン（窓は一つとする）を引くと、 m m m m = 1 , 2 , … m = 1, 2, \ldots m = 1 , 2 , … m m m { 1 , 2 , … , m } \{1, 2, \ldots, m\} { 1 , 2 , … , m } n n n n = 1 , 2 , … n = 1, 2, \ldots n = 1 , 2 , … X n X_n X n ∈ { 1 , 2 , … , m } \in \{1, 2, \ldots, m\} ∈ { 1 , 2 , … , m } m m m i i i i = 1 , 2 , … , m i = 1, 2, \ldots, m i = 1 , 2 , … , m T m , i T_{m,i} T m , i
T m , i = { 1 ( i = 1 ) min { n > T m , i − 1 ∣ X n ≠ X j ; j = 1 , … , n − 1 } ( i = 2 , … , m ) T_{m,i} = \begin{cases}
1 & (i = 1) \\
\min \{ n > T_{m,i-1} \mid X_n \neq X_j; j = 1, \ldots, n-1 \} & (i = 2, \ldots, m)
\end{cases} T m , i = { 1 min { n > T m , i − 1 ∣ X n = X j ; j = 1 , … , n − 1 } ( i = 1 ) ( i = 2 , … , m ) と再帰的に定義できる。以下の設問に答えなさい。
(3) 確率変数 U m , i U_{m,i} U m , i i = 2 , … , m i = 2, \ldots, m i = 2 , … , m
U m , i ≡ T m , i − T m , i − 1 U_{m,i} \equiv T_{m,i} - T_{m,i-1} U m , i ≡ T m , i − T m , i − 1 とする。
U m , i = k U_{m,i} = k U m , i = k k = 1 , 2 , … k = 1, 2, \ldots k = 1 , 2 , … P ( U m , i = k ) P(U_{m,i} = k) P ( U m , i = k )
(4) すべての図柄が初めて出るまでスロットマシンを引いた回数 T m , m T_{m,m} T m , m
1 + m ∑ j = 1 m − 1 1 j 1 + m \sum_{j=1}^{m-1} \frac{1}{j} 1 + m j = 1 ∑ m − 1 j 1 Kai 問題 1
(1)
The likelihood function is
L ( θ , σ 2 ) = ∏ i = 1 n f X ( X i ) ∏ i = 1 m f Y ( Y i ) = ∏ i = 1 n 1 2 π σ e − ( x − a θ ) 2 2 σ 2 ∏ i = 1 m 1 2 π σ e − ( x − b θ ) 2 2 σ 2 , \begin{aligned}
L(\theta,\sigma^{2})&=\prod_{i=1}^nf_X(X_i)\prod_{i=1}^mf_Y(Y_i)\\
&=\prod_{i=1}^n\frac1{\sqrt{2\pi}\sigma}e^{-\frac{(x-a\theta)^2}{2\sigma^2}}\prod_{i=1}^m\frac1{\sqrt{2\pi}\sigma}e^{-\frac{(x-b\theta)^2}{2\sigma^2}},
\end{aligned} L ( θ , σ 2 ) = i = 1 ∏ n f X ( X i ) i = 1 ∏ m f Y ( Y i ) = i = 1 ∏ n 2 π σ 1 e − 2 σ 2 ( x − a θ ) 2 i = 1 ∏ m 2 π σ 1 e − 2 σ 2 ( x − b θ ) 2 , from which we know that the log-likelihood is:
log L = − ( n + m ) log σ − ( n + m ) log 2 π − { ∑ i = 1 n ( a θ − X i ) 2 2 σ 2 + ∑ j = 1 m ( b θ − Y j ) 2 2 σ 2 } . \begin{aligned}
\log L&=-(n+m)\log \sigma-(n+m)\log\sqrt{2\pi}\\
&-\bigg\{\sum_{i=1}^{n}\frac{(a\theta-X_{i})^{2}}{2\sigma^{2}}+\sum_{j=1}^{m}\frac{(b\theta-Y_{j})^{2}}{2\sigma^{2}}\bigg\}.
\end{aligned} log L = − ( n + m ) log σ − ( n + m ) log 2 π − { i = 1 ∑ n 2 σ 2 ( a θ − X i ) 2 + j = 1 ∑ m 2 σ 2 ( b θ − Y j ) 2 } . Set ∂ log L ∂ σ = 0 \frac{\partial\log L}{\partial\sigma}=0 ∂ σ ∂ l o g L = 0
∂ log L ∂ σ = − n + m σ − { ∑ i = 1 n ( a θ − X i ) 2 + ∑ j = 1 m ( b θ − Y j ) 2 } ⋅ 1 2 ⋅ ( − 2 ) 1 σ 3 = 0 ⇒ σ ^ 2 = 1 n + m { ∑ i = 1 n ( a θ − X i ) 2 + ∑ j = 1 m ( b θ − Y j ) 2 } \begin{aligned}
\frac{\partial\log L}{\partial\sigma}& =-\frac{n+m}{\sigma}-\left \{ \sum_{i=1}^{n}(a\theta-X_{i})^{2} +\sum_{j=1}^m(b\theta-Y_j)^2\right\}\cdot\frac12\cdot(-2)\frac1{\sigma^3} \\
&=0 \\
\Rightarrow\quad\hat{\sigma}^2=& \frac{1}{n+m}\bigg\{\sum_{i=1}^{n}{(a\theta-X_{i})^{2}}+\sum_{j=1}^{m}{(b\theta-Y_{j})^{2}}\bigg\}
\end{aligned} ∂ σ ∂ log L ⇒ σ ^ 2 = = − σ n + m − { i = 1 ∑ n ( a θ − X i ) 2 + j = 1 ∑ m ( b θ − Y j ) 2 } ⋅ 2 1 ⋅ ( − 2 ) σ 3 1 = 0 n + m 1 { i = 1 ∑ n ( a θ − X i ) 2 + j = 1 ∑ m ( b θ − Y j ) 2 } The MLE of θ \theta θ
∂ log L ∂ θ = − 1 2 σ 2 { ∑ i = 1 n 2 ( a θ − X i ) a + ∑ j = 1 m 2 ( b θ − Y j ) b } = 0 ⇒ θ ^ = a ∑ i = 1 n X i + b ∑ j = 1 m Y j n a 2 + m b 2 . \begin{aligned}
\frac{\partial\log L}{\partial\theta}&=-\frac{1}{2\sigma^{2}}\bigg\{\sum_{i=1}^{n}2(a\theta-X_{i})a+\sum_{j=1}^{m}2(b\theta-Y_{j})b\bigg\}=0\\
&\Rightarrow\quad\hat{\theta}=\frac{a\sum_{i=1}^{n}X_{i}+b\sum_{j=1}^{m}Y_{j}}{na^{2}+mb^{2}}.
\end{aligned} ∂ θ ∂ log L = − 2 σ 2 1 { i = 1 ∑ n 2 ( a θ − X i ) a + j = 1 ∑ m 2 ( b θ − Y j ) b } = 0 ⇒ θ ^ = n a 2 + m b 2 a ∑ i = 1 n X i + b ∑ j = 1 m Y j . (2)
E [ X ˉ ] = E [ X 1 + ⋯ + X n n ] = ⏞ i . i . d . 1 n ⋅ n ⋅ E [ X i ] = a θ . \mathbb{E}[\bar{X}]=\mathbb{E}\left[\dfrac{X_1+\cdots+X_n}{n}\right]\overbrace{=}^{i.i.d.}\dfrac{1}{n}\cdot n\cdot\mathbb{E}[X_i]=a\theta. E [ X ˉ ] = E [ n X 1 + ⋯ + X n ] = i . i . d . n 1 ⋅ n ⋅ E [ X i ] = a θ . Similarly,
E [ Y ˉ ] = b θ . \mathbb{E}[\bar{Y}]=b\theta. E [ Y ˉ ] = b θ . Then
E [ θ ~ ] = α a θ + β b θ . \mathbb{E}[\tilde{\theta}]=\alpha a\theta+\beta b\theta. E [ θ ~ ] = α a θ + β b θ . For variance we have
V a r ( X ˉ ) = 1 n V a r ( X i ) = σ 2 n , V a r ( Y ˉ ) = σ 2 m , \mathrm{Var}(\bar{X})=\frac{1}{n}\mathrm{Var}(X_{i})=\frac{\sigma^{2}}{n},\quad\mathrm{Var}(\bar{Y})=\frac{\sigma^{2}}{m}, Var ( X ˉ ) = n 1 Var ( X i ) = n σ 2 , Var ( Y ˉ ) = m σ 2 , V a r ( α X ˉ + β Y ˉ ) = ( α 2 n + β 2 m ) σ 2 . \mathrm{Var}(\alpha\bar{X}+\beta\bar{Y})=(\frac{\alpha^2}{n}+\frac{\beta^2}{m})\sigma^2. Var ( α X ˉ + β Y ˉ ) = ( n α 2 + m β 2 ) σ 2 . (3)
When θ ~ \tilde{\theta} θ ~ θ \theta θ
E [ θ ~ ] = α a θ + β b θ = θ , \mathbb{E}[\tilde\theta]=\alpha a\theta+\beta b\theta=\theta, E [ θ ~ ] = α a θ + β b θ = θ , which is equivalent to
β = 1 − α a b . \beta=\frac{1-\alpha a}{b}. β = b 1 − α a . The variance of θ ~ \tilde{\theta} θ ~
V a r ( θ ~ ) = σ 2 n ( α 2 + 1 + a 2 α 2 − 2 a α b 2 ) = σ 2 n { ( 1 + a 2 b 2 ) α 2 − 2 a b α + 1 b 2 } . \begin{aligned}\mathrm{Var}(\tilde{\theta})&=\frac{\sigma^{2}}{n}\left(\alpha^{2}+\frac{1+a^{2}\alpha^{2}-2a\alpha}{b^{2}}\right)\\
&=\frac{\sigma^{2}}{n}\bigg\{(1+\frac{a^{2}}{b^{2}})\alpha^{2}-\frac{2a}{b}\alpha+\frac{1}{b^{2}}\bigg\}.
\end{aligned} Var ( θ ~ ) = n σ 2 ( α 2 + b 2 1 + a 2 α 2 − 2 a α ) = n σ 2 { ( 1 + b 2 a 2 ) α 2 − b 2 a α + b 2 1 } . The minima of Var ( θ ~ ) \text{Var}(\tilde{\theta}) Var ( θ ~ )
α = 2 a b 2 2 ( 1 + a 2 b 2 ) = a a 2 + b 2 , \alpha=\frac{\frac{2a}{b^2}}{2(1+\frac{a^2}{b^2})}=\frac{a}{a^2+b^2}, α = 2 ( 1 + b 2 a 2 ) b 2 2 a = a 2 + b 2 a , and
β = 1 − a 2 a 2 + b 2 b = b a 2 + b 2 . \beta=\frac{1-\frac{a^2}{a^2+b^2}}b=\frac b{a^2+b^2}. β = b 1 − a 2 + b 2 a 2 = a 2 + b 2 b . 問題2
(For this question, readers may refer to geometric distribution, 幾何分布.)
(1)
Pr ( T = n ) = Pr ( t 1 = t a i l ) × Pr ( t 2 = t a i l ) × ⋯ × P ( t n = h e a d ) = q n − 1 p , \begin{aligned}\Pr(T=n)&=\Pr(t_{1}=tail) \times \Pr(t_{2}=tail) \times \cdots \times P(t_{n}=head)\\&=q^{n-1}p,\end{aligned} Pr ( T = n ) = Pr ( t 1 = t ai l ) × Pr ( t 2 = t ai l ) × ⋯ × P ( t n = h e a d ) = q n − 1 p , where t i t_i t i i i i
(2)
E [ T ] = ∑ k = 1 n ( p k ) q k − 1 = ∑ k = 0 n − 1 p ( k + 1 ) q k , (i) \mathbb{E}[T]=\sum_{k=1}^n(pk)q^{k-1}=\sum_{k=0}^{n-1}p(k+1)q^k, \tag{i} E [ T ] = k = 1 ∑ n ( p k ) q k − 1 = k = 0 ∑ n − 1 p ( k + 1 ) q k , ( i ) This is a commonly seen series. We consider
q E [ T ] = ∑ k = 1 n ( p k ) q k . (ii) q\mathbb{E}[T]=\sum_{k=1}^n(pk)q^k. \tag{ii} q E [ T ] = k = 1 ∑ n ( p k ) q k . ( ii ) Eq. (i) subtracted by Eq. (ii) is
( 1 − q ) E [ T ] = p − p n ⋅ q n + ∑ k = 1 n − 1 p ⋅ q k . (1-q)\mathbb{E}[T]=p-pn\cdot q^n+\sum_{k=1}^{n-1}p\cdot q^k. ( 1 − q ) E [ T ] = p − p n ⋅ q n + k = 1 ∑ n − 1 p ⋅ q k . Then
E [ T ] = ( p q 1 − q + p ) / ( 1 − q ) = p ( 1 − q ) 2 = 1 p . \begin{aligned}
\mathbb{E}[T]&=\left(\frac{pq}{1-q}+p\right)/(1-q)\\
&=\frac{p}{(1-q)^{2}}=\frac{1}{p}.
\end{aligned} E [ T ] = ( 1 − q pq + p ) / ( 1 − q ) = ( 1 − q ) 2 p = p 1 . Now consider the second moment.
E [ T 2 ] = ∑ k = 1 ∞ k 2 q k − 1 p = ∑ k = 0 ∞ ( k + 1 ) 2 q k p . \mathbb{E}[T^2]=\sum_{k=1}^\infty k^2q^{k-1}p=\sum_{k=0}^\infty(k+1)^2q^kp. E [ T 2 ] = k = 1 ∑ ∞ k 2 q k − 1 p = k = 0 ∑ ∞ ( k + 1 ) 2 q k p . q E [ T 2 ] = ∑ k = 1 ∞ k 2 q k p , q\mathbb{E}[T^2]=\sum_{k=1}^\infty k^2q^kp, q E [ T 2 ] = k = 1 ∑ ∞ k 2 q k p , ( 1 − q ) E [ T 2 ] = p + ∑ k = 1 ∞ ( 2 k + 1 ) q k p = 2 q p + p q 1 − q + p = 2 − p p , \begin{aligned}(1-q)\mathbb{E}[T^{2}]&=p+\sum_{k=1}^{\infty}(2k+1)q^{k}p\\
&=\frac{2q}p+\frac{pq}{1-q}+p\\
&=\frac{2-p}p,
\end{aligned} ( 1 − q ) E [ T 2 ] = p + k = 1 ∑ ∞ ( 2 k + 1 ) q k p = p 2 q + 1 − q pq + p = p 2 − p , Thus
E [ T 2 ] = 2 − p p 2 , \mathbb{E}[T^2]=\frac{2-p}{p^2}, E [ T 2 ] = p 2 2 − p , and
V a r ( T ) = 2 − p p 2 − 1 p 2 = 1 − p p 2 . \mathrm{Var}(T)=\frac{2-p}{p^2}-\frac{1}{p^2}=\frac{1-p}{p^2}. Var ( T ) = p 2 2 − p − p 2 1 = p 2 1 − p . (3)
The most tricky part is to understand the question (at least for me the formulation of T m , i T_{m,i} T m , i T m , i T_{m,i} T m , i i i i T m , 1 = 1. T_{m,1}=1. T m , 1 = 1. T m , 2 = 1 + 3 + 1 = 5. T_{m,2}=1+3+1=5. T m , 2 = 1 + 3 + 1 = 5. T m , 3 , … , T m , m . T_m,3,\ldots,T_{m,m}. T m , 3 , … , T m , m .
Then we will notice that T m , i − T m , i − 1 T_{m,i}-T_{m,i-1} T m , i − T m , i − 1 ( i − 1 ) (i-1) ( i − 1 ) ( m − i + 1 ) (m-i+1) ( m − i + 1 )
Then U m , i U_m,i U m , i
Pr ( U m , i = k ) = ( i − 1 m ) k − 1 ( m − i + 1 m ) . \Pr(U_{m,i}=k)=\left(\frac{i-1}{m}\right)^{k-1}\left(\frac{m-i+1}{m}\right). Pr ( U m , i = k ) = ( m i − 1 ) k − 1 ( m m − i + 1 ) . (4)
We first find E [ U m , i ] \mathbb{E}[U_m,i] E [ U m , i ]
i.e.,
E [ U m , i ] = 1 m − i + 1 m = m m − i + 1 . \mathbb{E}[U_{m,i}]=\frac{1}{\frac{m-i+1}{m}}=\frac{m}{m-i+1}. E [ U m , i ] = m m − i + 1 1 = m − i + 1 m . Therefore,
T m , m = 1 + U m , 2 + U m , 3 + … + U m , m E [ T m , m ] = 1 + ∑ i = 2 m E [ U m , i ] = 1 + ∑ i = 2 m m m − i + 1 = 1 + m ∑ i = 2 m 1 m − i + 1 = 1 + m ∑ i = 2 m 1 m − i + 1 = 1 + m ∑ j = 2 m 1 j − 1 . \begin{aligned}
T_{m,m}&=1+U_{m,2}+U_{m,3}+\ldots+U_{m,m}\\
\mathbb{E}[T_{m,m}]&=1+\sum_{i=2}^m\mathbb{E}[U_{m,i}]\\
&=1+\sum_{i=2}^m\frac m{m-i+1}=1+m\sum_{i=2}^m\frac1{m-i+1}\\
&=1+m\sum_{i=2}^m\frac1{m-i+1}=1+m\sum_{j=2}^m\frac1{j-1}.
\end{aligned} T m , m E [ T m , m ] = 1 + U m , 2 + U m , 3 + … + U m , m = 1 + i = 2 ∑ m E [ U m , i ] = 1 + i = 2 ∑ m m − i + 1 m = 1 + m i = 2 ∑ m m − i + 1 1 = 1 + m i = 2 ∑ m m − i + 1 1 = 1 + m j = 2 ∑ m j − 1 1 .