 |
|
| Computers Forum Index » Computer Artificial Intelligence - Neural Nets » Column Scaling, Relevance, and Weights... |
|
Page 1 of 1 |
|
| Author |
Message |
| TomH488... |
 Posted: Mon Aug 24, 2009 6:18 pm |
|
|
|
Guest
|
It seems that there is a lot of concern when there are very large
weights (>>1). All sorts of methods are used to limit this.
With this in mind, consider a simple Nnet model teaching addition:
a+b=c
where a, b, c are the columns.
But consider the following constraints on these columns: we are going
to add one very small number to one very large number. Therefore let:
a=(0,1) x 10e6, and
b=(0,1)
the result is that
c=(0,1) x 10e6 + epsilon, where epsilon=(0,1)
For all intents and purposes, col b is irrelevant for c up to 5
significant figures.
If this training input was not scaled, the weight matrix would be of
Order 1,
however, if all col are scaled to (0,1) or standardized,
there would have to weight ratios of Order 6 present in the weight
matrix to preserve the relevance of the actual input.
This big weight variation would not be welcome.
Which leads to the following implied assumtion:
When scaling inputs, it is assumed that they share the same
approximate relevance.
If not, the weight matrix will contain ratios necessary to drive
irrelevant columns into the noise.
Comments? Any will be greatly appreciated.
Thanks in advance,
Tom |
|
|
| Back to top |
|
|
|
|
|
All times are GMT
The time now is Tue Nov 24, 2009 9:14 pm
|
|