variational inference

传统的MCMC去近似，有lib，较容易。但是用VI，每个问题都得推导。但是现在出现 自动变分推断算法，可以直接用lib，比如PyMC3。贝叶斯深度学习——基于PyMC3的变分推理

欧拉-拉格朗日方程

找一个 $f(x)$ ,在 $[a,b]$ 之间使得 $J = \int_a^b F(x,f(x),f^\prime (x)) dx$ 积分达到最大值。假设 $g(x) = f(x) + \varepsilon \eta(x)$ 。 $\varepsilon$ 是个很小的正数，这就相当于 $g(x)$ 为最佳函数 $f(x)$ 增加了一个很小的扰动。当使用 $g(x)$ 求得极值时候， $g(x) = f(x)$ 。将 $g(x)$ 带入原方程，并求导。

J = \int_a^b F(x,g(x),g^\prime (x)) dx \\ \frac {\partial J}{\partial \varepsilon} = \frac {\partial x}{\partial \varepsilon} \frac {\partial F}{\partial x} + \frac {\partial g}{\partial \varepsilon} \frac {\partial F}{\partial g} + \frac {\partial g^\prime}{\partial \varepsilon} \frac {\partial F}{\partial g^\prime} = \eta(x) \frac {\partial F}{\partial g} + \eta^\prime (x) \frac {\partial F}{\partial g^\prime} \\ \text{当}\varepsilon=0 \text{时，函数能取到极值。且此时：} g(x) = f(x), g^\prime(x) = f^\prime(x)\\ \frac {\partial J}{\partial \varepsilon} \mid_{\varepsilon=0} = \int_a^b [\eta(x) \frac {\partial F}{\partial f} + \eta^\prime (x) \frac {\partial F}{\partial f^\prime}] dx = 0 \\ \int_a^b \eta^\prime (x) \frac {\partial F}{\partial f^\prime} dx = \int_a^b \frac {\partial F}{\partial f^\prime} d \eta (x) = [\frac {\partial F}{\partial f^\prime} \eta (x)]_a^b - \int_a^b \eta (x) d \frac {\partial F}{\partial f^\prime} = - \int_a^b \eta (x) d \frac {\partial F}{\partial f^\prime} \\ \int_a^b \eta(x) [\frac {\partial F}{\partial f} + \frac {\partial }{\partial x} \frac {\partial F}{\partial f^\prime}] dx = 0 \\ \frac {\partial F}{\partial f} + \frac {\partial }{\partial x} \frac {\partial F}{\partial f^\prime} = 0 \\

引入“ $\delta$ 算子”来描述上述过程。定义 $\delta [y(x)] = \tilde{y}-y$ 。在本例中： $\delta y = \tilde{y}-y = a \eta, \delta y^\prime = \tilde{y^\prime}-y^\prime = a \eta^\prime$