Hi,

I would like to find out how to add regulariztion to an error function
in netlab tool box.

Thank you very much.

David

On Sep 27, 6:58 pm, David <dtian.ty... at (no spam) googlemail.com> wrote:

Hi,

I would like to find out how to add regulariztion to an error function
in netlab tool box.

Thank you very much.

David

netlab already includes regularisation, see help mlp

NET = MLP(NIN, NHIDDEN, NOUT, FUNC, PRIOR)

here if prior is a scalar value (e.g. 0.01) then it is the
regularisation constant for the weight-decay type regulariser that
NETLAB uses.  For setting the value of the parameter, it is probably
best to use the evidence framework (help evidence), there are a couple
of demos that give a good idea, e.g, demev1.m

HTH

Gavin

On 28 Sep, 12:06, "GavinCaw... at (no spam) googlemail.com"



gavincaw... at (no spam) googlemail.com> wrote:
On Sep 27, 6:58 pm, David <dtian.ty... at (no spam) googlemail.com> wrote:

Hi,

I would like to find out how to add regulariztion to an error function
in netlab tool box.

Thank you very much.

David

netlab already includes regularisation, see help mlp

NET = MLP(NIN, NHIDDEN, NOUT, FUNC, PRIOR)

here if prior is a scalar value (e.g. 0.01) then it is the
regularisation constant for the weight-decay type regulariser that
NETLAB uses. For setting the value of the parameter, it is probably
best to use the evidence framework (help evidence), there are a couple
of demos that give a good idea, e.g, demev1.m

HTH

Gavin

Gavin

Thank you very much. In fact, I had a look at the help information of
mlp. It does not mention regularization at all, so I asked the
question. I guess a lot of new users of netlab do not know about the
meaning of PRIOR in mlp function.

David

On Sep 28, 3:31 pm, David <dtian.ty... at (no spam) googlemail.com> wrote:





On 28 Sep, 12:06, "GavinCaw... at (no spam) googlemail.com"

gavincaw... at (no spam) googlemail.com> wrote:
On Sep 27, 6:58 pm, David <dtian.ty... at (no spam) googlemail.com> wrote:

Hi,

I would like to find out how to add regulariztion to an error function
in netlab tool box.

Thank you very much.

David

netlab already includes regularisation, see help mlp

NET = MLP(NIN, NHIDDEN, NOUT, FUNC, PRIOR)

here if prior is a scalar value (e.g. 0.01) then it is the
regularisation constant for the weight-decay type regulariser that
NETLAB uses.  For setting the value of the parameter, it is probably
best to use the evidence framework (help evidence), there are a couple
of demos that give a good idea, e.g, demev1.m

HTH

Gavin

Gavin

Thank you very much. In fact, I had a look at the help information of
mlp. It does not mention regularization at all, so I asked the
question. I guess a lot of new users of netlab do not know about the
meaning of PRIOR in mlp function.

David

I think it probably helps to have read the book by Bishop, which has a
subtle Bayesian bias, which explains why the regularisation parameters
are discussed in terms of priors (the weight decay regulariser
corresponds to a Gaussian prior over the weights in Bayesian
terminology).  Netlab is a great peice of kit, I am using it for a
study on Bayesian regularisation that I am running at the moment (and
have been for about a year so far!) and it has proven remarkably
robust for free software.

There is a book by Ian Nabney, "NETLAB algorithms for pattern
recognition", Springer Verlag, series on advances in pattern
recognition, ISBN 1-85233-440-1, which describes the inner workings of
NETLAB in greater detail.  If you use NETLAB a lot, or want to extend
it in someway, I can strongly recommend getting hold of a copy.- Hide quoted text -

- Show quoted text -