feng's Blog

Happy coding

Relevance Vector Machine 1

feng posted @ Sun, 30 Sep 2012 22:13:08 +0800 in PRML , 1265 readers

内容关于PRML7.2节的

---------------------------------------------------------------------------------------------------------------------------

这个Relevance Vector Machine(RVM)跟SVM差不多,可以处理Regression问题,也可以处理classification问题。这里我们先讨论regression问题。

 

首先讨论一下什么是regression问题。我们拿到一些训练的样本$(\mathbf{x}_i, t_i)$,希望从中训练出来一个映射关系$t=y(\mathbf{x})$。在测试的时候,给定一个$\mathbf{x}$,我们就可以估计出来$t$的数值。其中$t$一般认为是连续的,$\mathbf{x}$和$\mathbf{t}$一般也都是实数。如何求解这个映射关系$y$,是该问题的重点所在。

 

在RVM这个模型中,假设$t$是这样产生的,

\[p(t|\mathbf{x})=\mathcal{N}(t|y(\mathbf{x}), \beta^-1)$\]

也就是说,给定了$\mathbf{x}$之后,通过映射得到$y(\mathbf{x})$,然后加上一个高斯噪声,输出$t$。这个高斯噪声的平均值是0,方差是$\beta^-1$,其中$\beta$也称之为准确率,未知的。而$y(\mathbf{x})$的形式假定为线性的,也就是说

\[y(\mathbf{x}) = \sum_{i = 1}^{M}{w_i \phi_i(\mathbf{x})} = \mathbf{w}^T \phi(\mathbf{x})\]

这是一种很广义的写法。其中$w_i$是未知的;$M$是未知参数的个数;$\phi_i(\mathbf{x})$是一个预先设置好的函数映射。其实具体在做的时候,$\phi_i(\mathbf{x})=k(\mathbf{x}, \mathbf{x}_i)$。也就是说

\[y(\mathbf{x}) = \sum_{n = 1}^N{w_{n}k(\mathbf{x}, \mathbf{x}_n)} + b\]

其中$N$是训练样本的个数;$k(\cdot, \cdot)$是一个核函数,如果不是很清楚什么是核函数的话,姑且认为是一个普通的函数就好,这里的核函数也是预先设置好,不需要求解的;$b$是一个常数,未知。

实际中用到的是核函数的表达式子,但是以下的推到,也同样适用于广义的写法。所以,我们就直接用$y(\mathbf{x})=\mathbf{w}^T \phi(\mathbf{x})$的写法。

 

现在的问题是,如何求解$\mathbf{w}$和$\beta$。

 

RVM中又假定了$\mathbf{w}$这样的分布性质

\[p(\mathbf{w}|\mathbf{\alpha}) = \prod_{i = 1}^{M}\mathcal{N}(w_i | 0, \alpha_i^{-1})\]

也就是说,每一个$w_i$都是互相独立的,都是符合高斯分布的,而这个分布的均值是0,方差是$\alpha_i^{-1}$。

再接下来的问题就是,如何估计这个方差$\mathbf{\alpha}$和$\beta$。

Avatar_small
buy instagram likes said:
Tue, 20 Apr 2021 08:49:59 +0800

Very interesting blog. Alot of blogs I see these days don't really provide anything that I'm interested in, but I'm most definately interested in this one. Just thought that I would post and let you know.

Avatar_small
InventHelp said:
Wed, 21 Apr 2021 06:01:33 +0800

Great job for publishing such a beneficial web site. Your web log isn’t only useful but it is additionally really creative too. There tend to be not many people who can certainly write not so simple posts that artistically. Continue the nice writing

Avatar_small
Car Bluetooth Receiv said:
Sun, 25 Apr 2021 05:52:14 +0800

Great article with excellent idea!Thank you for such a valuable article. I really appreciate for this great information..

Avatar_small
Concrete Driveway Re said:
Tue, 27 Apr 2021 04:44:10 +0800

Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work!

Avatar_small
AAA said:
Sat, 01 Jan 2022 15:09:39 +0800

This was a very nice post. In concept I wish to put in writing like this additionally – taking time and actual effort to make a very good article… however what can I say… I procrastinate alot and not at all seem to get something done. 토토사이트

Avatar_small
AAA said:
Thu, 17 Feb 2022 04:01:41 +0800

kitchen aids have a variety of different appliances that can help you cook your food easier;; 토토사이트


Login *


loading captcha image...
(type the code from the image)
or Ctrl+Enter