| Computers Forum Index » Computer Artificial Intelligence - Neural Nets » Increasing errors against training set with more... |
|
Page 1 of 1 |
|
| Author |
Message |
| garyrob... |
 Posted: Fri Oct 09, 2009 8:06 pm |
|
|
|
Guest
|
Wait... I think I see what's happening. It's always decreasing the sum
squared error as the number of iterations increases, based on the raw
output from the neural net... but after a certain point, that decrease
in sum squared error corresponds to a decrease in the binary
classification task I'm training for. Hmm... interesting...
Again, the problem seems to go away with an increase in the number of
nodes in the hidden layer...
Gary |
|
|
| Back to top |
|
|
|
| Greg... |
| Posted: Sun Oct 11, 2009 2:56 pm |
|
|
|
Guest
|
On Oct 9, 4:06 pm, garyrob <garyr... at (no spam) gmail.com> wrote:
Quote: Wait... I think I see what's happening. It's always decreasing the sum
squared error as the number of iterations increases, based on the raw
output from the neural net... but after a certain point, that decrease
in sum squared error corresponds to a decrease in the binary
classification task I'm training for. Hmm... interesting...
Again, the problem seems to go away with an increase in the number of
nodes in the hidden layer...
Gary
How many input variables?
How many output nodes?
What are the output targets?
Greg |
|
|
| Back to top |
|
|
|
| garyrob... |
| Posted: Wed Oct 14, 2009 6:20 pm |
|
|
|
Guest
|
Quote: How many input variables?
How many output nodes?
What are the output targets?
Greg
Hmm. Now I'm not even seeing the steady decrease in squared error...
that's bouncing around too.
To answer your questions:
6 input variables plus one constant -1 input for bias (I read in a
book that this can be helpful) for a total of 7.
One hidden layer with 3 nodes.
One output node.
The targets are 0 and 1.
There are 127 training samples.
For the first couple of training iterations the error goes down. After
that it goes up and bounces around. For instance, I'm classifying the
output node as 0 if the output is < .5, and otherwise, 1.
0 iterations: Hits: 19; Misses: 116
1 iteration: Hits: 125; Misses: 10
2 iterations: Hits: 129; Misses: 6
3 iterations: Hits: 110; Misses: 25
.... and the misses bounce around from there.
I'm surprised that the NN gets to such a decent classification level
(the data is somewhat noisy) and then becomes worse.
As I mentioned earlier, I'm using the network at http://python.ca/nas/python/bpnn.py,
which is also used in this IBM tutorial on NN's:
http://www.ibm.com/developerworks/library/l-neural/.
It defaults to "learing rate"=.5 and "momentum factor"=.1; I'm using
those default values. |
|
|
| Back to top |
|
|
|
| garyrob... |
| Posted: Thu Oct 15, 2009 6:06 pm |
|
|
|
Guest
|
Folks, I must apologize. I finally found the bug in my code. I am
still getting some cases where there is more error in a generation
than in the preceding one, but now, overall, the trend is to fewer
errors. So I believe things are probably working fine.
I appreciate any time anyone took to think about the problem I posed,
and am sorry it didn't lead anywhere meaningful. |
|
|
| Back to top |
|
|
|
|
