# variational inference

传统的MCMC去近似，有lib，较容易。但是用VI，每个问题都得推导。但是现在出现 **自动变分推断算法**，可以直接用lib，比如PyMC3。 [贝叶斯深度学习——基于PyMC3的变分推理](http://geek.csdn.net/news/detail/80255)

## 欧拉-拉格朗日方程

找一个$$f(x)$$,在$$\[a,b]$$之间使得$$J = \int\_a^b F(x,f(x),f^\prime (x)) dx$$积分达到最大值。\
假设$$g(x) = f(x) + \varepsilon \eta(x)$$。$$\varepsilon$$是个很小的正数，这就相当于$$g(x)$$为最佳函数$$f(x)$$增加了一个很小的扰动。当使用$$g(x)$$求得极值时候，$$g(x) = f(x)$$。 将 $$g(x)$$带入原方程，并求导。

$$
J = \int\_a^b F(x,g(x),g^\prime (x)) dx \\
\frac {\partial J}{\partial \varepsilon} = \frac {\partial x}{\partial \varepsilon} \frac {\partial F}{\partial x} + \frac {\partial g}{\partial \varepsilon} \frac {\partial F}{\partial g} + \frac {\partial g^\prime}{\partial \varepsilon} \frac {\partial F}{\partial g^\prime} = \eta(x) \frac {\partial F}{\partial g} + \eta^\prime (x) \frac {\partial F}{\partial g^\prime}  \\
\text{当}\varepsilon=0 \text{时，函数能取到极值。且此时：} g(x) = f(x),  g^\prime(x) = f^\prime(x)\\
\frac {\partial J}{\partial \varepsilon} \mid\_{\varepsilon=0} = \int\_a^b \[\eta(x) \frac {\partial F}{\partial f} + \eta^\prime (x) \frac {\partial F}{\partial f^\prime}] dx = 0 \\
\int\_a^b  \eta^\prime (x) \frac {\partial F}{\partial f^\prime} dx = \int\_a^b \frac {\partial F}{\partial f^\prime} d \eta (x) = \[\frac {\partial F}{\partial f^\prime} \eta (x)]\_a^b - \int\_a^b \eta (x) d \frac {\partial F}{\partial f^\prime} = - \int\_a^b \eta (x) d \frac {\partial F}{\partial f^\prime} \\
\int\_a^b \eta(x) \[\frac {\partial F}{\partial f} +  \frac {\partial }{\partial x} \frac {\partial F}{\partial f^\prime}] dx = 0 \\
\frac {\partial F}{\partial f} +  \frac {\partial }{\partial x} \frac {\partial F}{\partial f^\prime} = 0 \\
$$

引入“$$\delta$$算子”来描述上述过程。定义$$\delta \[y(x)] = \tilde{y}-y$$。\
在本例中：\
$$\delta y = \tilde{y}-y = a \eta, \delta y^\prime = \tilde{y^\prime}-y^\prime = a \eta^\prime$$

![](https://2270971654-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M7DcNFhVrwIk3Tks_pB%2Fsync%2F77d54404c283a66dce24387ce01574a368c41e33.png?generation=1589383949945760\&alt=media)\
![](https://2270971654-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M7DcNFhVrwIk3Tks_pB%2Fsync%2Fa0c7e597aefb8f993de0df9be0ceff1738481ccc.png?generation=1589383949584372\&alt=media)\
![](https://2270971654-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M7DcNFhVrwIk3Tks_pB%2Fsync%2F70184873c73999516a58e9b0c8adba63439e3401.png?generation=1589383949806764\&alt=media)\
![](https://2270971654-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-M7DcNFhVrwIk3Tks_pB%2Fsync%2F0a1b03cdd5c82f2c4336f44f175176f791ba8d85.png?generation=1589383949557172\&alt=media)

## 欧拉-拉格朗日方程两种形式

## 变分EM

### 参考佳文

[变分贝叶斯](http://www.junnanzhu.com/?p=386)

[变分原理正文](http://wenku.baidu.com/view/2747cc43336c1eb91a375d30.html?re=view)\
[变分原理的直接方法](http://wenku.baidu.com/view/33020d92fd0a79563c1e72ee.html)\
[变分方法](http://wenku.baidu.com/view/32d1e11aff00bed5b9f31d0f.html)

[数学变分法](https://www.youtube.com/channel/UCci8XOD1wSL4U91IzLqC3fQ)\
[变分法](http://wenku.baidu.com/link?url=xScZ1TYKIa9psjarFO8jip8jSmUsVrahv-v163KYZDECk3XU9FvN1oni6X0Oh0IhK5ivMT83GYt5RdZ41S_6Kvj0D8ubzqT5meb5Ficch0y)\
[徐亦达的机器学习视频](http://i.youku.com/u/UMzIzNDgxNTg5Ng==/videos)