《A Guide to Singular Value Decomp osition for Collab orative Filtering》
用户对电影的打分可以用feature建立联系：用户喜欢动作片还是科幻片，该电影是喜剧还是传记。
建立两个矩阵：每个用户对各个feature的喜欢程度，电影的各个feature程度。则两矩阵相乘的结果与原来的评分越近越好，即期望越销越好。
E = 1 2 ∑ i = 1 n ∑ j = 1 m I i j ( V i j − p ( U i , M j ) ) 2 + k u 2 ∑ i = 1 n ∣ ∣ U i ∣ ∣ 2 + k m 2 ∑ j = 1 m ∣ ∣ M j ∣ ∣ 2 E=\frac 12 \sum_{i=1}^n \sum_{j=1}^m I_{ij}(V_{ij}-p(U_i,M_j))^2 + \frac {k_u}{2}\sum_{i=1}^n ||U_i||^2 + \frac {k_m}{2} \sum_{j=1}^m ||M_j||^2 E = 2 1 i = 1 ∑ n j = 1 ∑ m I ij ( V ij − p ( U i , M j ) ) 2 + 2 k u i = 1 ∑ n ∣∣ U i ∣ ∣ 2 + 2 k m j = 1 ∑ m ∣∣ M j ∣ ∣ 2 其中n表示用户数目，m表示物品数目，I i j I_ij I i j V i j V_ij V i j p ( U i , M j ) p(U_i,M_j) p ( U i , M j )
用梯度下降法求解,算法流程：
1. Set the starting values of matrices U,M.
2. Repeat
- (a) Compute gradients \Delta_U and \Delta_M
- (b) Set U \leftarrow U-\mu\Delta_U , M \leftarrow M-\mu_\Delta_M .
until the vaildation RMSE start to increase
其中梯度：
∂ E ∂ U i = ∑ j = 1 m I i j ( ( V i j − p ( U i , M j ) ) M j ) − K u U i , i = 1 , … , n ∂ E ∂ M j = ∑ i = 1 n I i j ( ( V i j − p ( U i , M j ) ) U j ) − K m M j , j = 1 , … , m \frac{\partial E}{\partial U_i} = \sum_{j=1}^m I_{ij}((V_{ij}-p(U_i,M_j))M_j)-K_uU_i , i=1,\ldots,n \\
\frac{\partial E}{\partial M_j} = \sum_{i=1}^n I_{ij}((V_{ij}-p(U_i,M_j))U_j)-K_mM_j , j=1,\ldots,m ∂ U i ∂ E = j = 1 ∑ m I ij (( V ij − p ( U i , M j )) M j ) − K u U i , i = 1 , … , n ∂ M j ∂ E = i = 1 ∑ n I ij (( V ij − p ( U i , M j )) U j ) − K m M j , j = 1 , … , m 上述算法被称为批处理式学习算法，因为它计算的是整个矩阵。
不完全增量式学习
E i = 1 2 ∑ i = 1 n ∑ j = 1 m I i j ( V i j − p ( U i , M j ) ) 2 + k u 2 ∑ i = 1 n ∣ ∣ U i ∣ ∣ 2 + k m 2 ∑ j = 1 m I i j ( ∣ ∣ M j ∣ ∣ ) 2 E_i = \frac 12 \sum_{i=1}^n \sum_{j=1}^m I_{ij}(V_{ij}-p(U_i,M_j))^2 + \frac {k_u}{2}\sum_{i=1}^n ||U_i||^2 + \frac {k_m}{2} \sum_{j=1}^m I_{ij}(||M_j||)^2 E i = 2 1 i = 1 ∑ n j = 1 ∑ m I ij ( V ij − p ( U i , M j ) ) 2 + 2 k u i = 1 ∑ n ∣∣ U i ∣ ∣ 2 + 2 k m j = 1 ∑ m I ij ( ∣∣ M j ∣∣ ) 2 梯度:
∂ E i ∂ U i = ∑ j = 1 m I i j ( ( V i j − p ( U i , M j ) ) M j ) − K u U i , i = 1 , … , n ∂ E i ∂ M j = I i j ( ( V i j − p ( U i , M j ) ) U j ) − K m I i j ( M j ) = I i j [ ( V i j − p ( U i , M j ) ) U j ) − K m M j ] , j = 1 , … , m \frac{\partial E_i}{\partial U_i} = \sum_{j=1}^m I_{ij}((V_{ij}-p(U_i,M_j))M_j)-K_uU_i , i=1,\ldots,n \\
\frac{\partial E_i}{\partial M_j}
= I_{ij}((V_{ij}-p(U_i,M_j))U_j)-K_mI_{ij}(M_j) \\
=I_{ij}[(V_{ij}-p(U_i,M_j))U_j)-K_mM_j], j=1,\ldots,m ∂ U i ∂ E i = j = 1 ∑ m I ij (( V ij − p ( U i , M j )) M j ) − K u U i , i = 1 , … , n ∂ M j ∂ E i = I ij (( V ij − p ( U i , M j )) U j ) − K m I ij ( M j ) = I ij [( V ij − p ( U i , M j )) U j ) − K m M j ] , j = 1 , … , m 完全增量式学习是对每一个评分进行期望计算，期望如下：
还有些SVD算法考虑了每个用户，每个物品的bias,这里所谓的bias就是每个人的偏差，比如一个电影a,b两人都认为不错，但是a评分方便比较保守，不错的给3分，b评分比较宽松，不错的给4分，故一下的评分方式考虑到了每个用户，每个物品的bias，要比上述算法更加精准。原来评分的话是直接计算u s e r f e a t u r e ∗ i t e m f e a t u r e T userfeature * itemfeature^T u ser f e a t u re ∗ i t e m f e a t u r e T
p ( U i , M j , α i , β j ) = a + U i T M j + α i + β j p(U_i,M_j,\alpha_i,\beta_j) = a+U_i^TM_j+\alpha_i+\beta_j p ( U i , M j , α i , β j ) = a + U i T M j + α i + β j 其中a表示所有评分的平均数，α i \alpha_i α i β i \beta_i β i
E i j = 1 2 ( V i j − p ( U i , M j , α i , β j ) ) 2 + k u 2 ∣ ∣ U i ∣ ∣ 2 + k m 2 I i j ( ∣ ∣ M j ∣ ∣ ) 2 + k b 2 ( α i 2 + β j 2 ) ∂ E i j ∂ U i = ( V i j − p ( U i , M j , α i , β j ) ) − K b α i ∂ E i j ∂ M j = ( V i j − p ( U i , M j , α i , β j ) ) − K b β j E_{ij} = \frac 12 (V_{ij}-p(U_i,M_j,\alpha_i,\beta_j))^2 + \frac {k_u}{2} ||U_i||^2 + \frac {k_m}{2} I_{ij}(||M_j||)^2+ \frac {k_b}{2} (\alpha_i^2 + \beta_j^2) \\
\frac{\partial E_{ij}}{\partial U_i} = (V_{ij}-p(U_i,M_j,\alpha_i,\beta_j))-K_b\alpha_i \\
\frac{\partial E_{ij}}{\partial M_j}
= (V_{ij}-p(U_i,M_j,\alpha_i,\beta_j))-K_b\beta_j E ij = 2 1 ( V ij − p ( U i , M j , α i , β j ) ) 2 + 2 k u ∣∣ U i ∣ ∣ 2 + 2 k m I ij ( ∣∣ M j ∣∣ ) 2 + 2 k b ( α i 2 + β j 2 ) ∂ U i ∂ E ij = ( V ij − p ( U i , M j , α i , β j )) − K b α i ∂ M j ∂ E ij = ( V ij − p ( U i , M j , α i , β j )) − K b β j 评测效果，就是误差平方和：
R M S E ( P , A ) = ∑ i = 1 n ∑ j = 1 m J i j ( A i j − P i j ) 2 ∑ i = 1 n ∑ j = 1 m J i j RMSE(P,A) = \sqrt \frac {\sum_{i=1}^n \sum_{j=1}^m J_{ij}(A_{ij}-P_{ij})^2}{\sum_{i=1}^n \sum_{j=1}^m J_{ij}} RMSE ( P , A ) = ∑ i = 1 n ∑ j = 1 m J ij ∑ i = 1 n ∑ j = 1 m J ij ( A ij − P ij ) 2 基于矩阵分解的推荐算法
参考佳文