Hello,

I'm completely new with AI programming and for the last few days I have
been tinkering with this example implementation of a backpropagation NN;
http://arctrix.com/nas/python/bpnn.py.

So far I have only made an attempt to a small application in wich a 'dot'
can find it's way to another dot (representing a heatsource) by using
some virtual 'sensors'.

Any time the NN 'fires' it makes a decision to go left-right-up-down or
to do nothing. Whatever it decides, the sensors pick up the new
measurements (amount of virtual heat) to determine it distance to the
other dot. These sensors act as an input to the NN, but are also
connected to a virtual 'hormonal system'. Whenever the measurements are
becoming increasingly more positive, the level of 'happiness' is adjusted
accordingly and vice versa. These levels act as a teacher for the NN. It
teaches wether to continue, stop, of retreat from the current direction
according to the change in 'happiness' level.

It still has it's flaws, but with a little tweaking (schoolvacations are
still going on , I believe I must be able to get it running perfectly.
All in all it is very fun to do.

I want to make a next step, saving inputweight data, connecting NNs to
other NNs to, in the end, enlarge the knowledge of my NN. But to do this
I must first really understand backpropagation NNs. Using Google I find a
lot of mathematical representations, that are not really helping me since
my mathematical background is limited. And very simple approached that
also do not help me in this case, because they are to limited. Can
someone point me to an complete explanation of a BPNN where the
mathematical model is explained in laymen (or at least easier) terms?

Thanks in advance,
Bart Smeets

I must first really understand backpropagation NNs. Using Google I find a
lot of mathematical representations, that are not really helping me since
my mathematical background is limited. And very simple approached that
also do not help me in this case, because they are to limited. Can
someone point me to an complete explanation of a BPNN where the
mathematical model is explained in laymen (or at least easier) terms?

On  9-Sep-2009, Bart Smeets <bartsmeet... at (no spam) gmail.com> wrote:

I must first really understand backpropagation NNs. Using Google I find a
lot of mathematical representations, that are not really helping me since
my mathematical background is limited. And very simple approached that
also do not help me in this case, because they are to limited. Can
someone point me to an complete explanation of a BPNN where the
mathematical model is explained in laymen (or at least easier) terms?

Ok, here is a non-mathematical, intuitive explanation:

Backprop feeds back into the training process information about how the
values predicted by the current network compares with the true training
values.  The closer the network gets to the true value, the smaller the
backprop error is.

Suppose you got into a car for the first time and did not know how to steer
it.  Looking out the windshield you can observe how the car moves as you
turn the steering wheel.  Your goal is to stay in your lane.  You start out
with an initial steering angle and watch how the car moves relative to the
lane.  If it starts to go too far to the right your eyes feed back (back
prop) that information to your brain which then "learns" that the wheel is
turned too far to the right and turns it to the left.  However, the left
turn is too much and you veer into the left lane.  This information is back
into the brain which then adjusts back to the right.  Hopefully after
driving the car for a while, your brain learns the correct amount of wheel
turn required to adjust for lane movement.

1. In an artificial neural network (ANN), you start out with an initial set
of weights which corresponds to your initial steering wheel angle.

2. For each training cycle the predicted values are compared to the actual
values, and the error magnitude and direction are fed back into net
training. This corresponds to your eyes telling your brain where your car is
relative to the center of the lane.

3. Based on the error magnitude and direction, weights are adjusted in the
network.  This corresponds to you turning the wheel left as the car drifts
right.

4. Weight adjustments become smaller for each training cycle as the training
process converges to the optimal set of weight values.  This corresponds to
your steering skills improving as you gain experience driving.

Hi Phil, I see that you have lot of knowledge about neural networks,
maybe you can also answer my questions?

I'm also new in this, I gave my network a very simple patterns of
horizontal and vertical lines, and I expected my network to learn it
very easily (Classification) but to my surprise my network unable to
learn this simple patterns, what can be the problem?

 Hopefully after
driving the car for a while, your brain learns the correct amount of
wheel
turn required to adjust for lane movement.

Hoping doesn't help much. You need an upper bound on the learning
rate to prevent nonconverging oscillations.

Sorry, I'm not going to take the time to write a program to generate data
for your problem. I might not even reproduce the salient aspects of your
data.

On 12-Sep-2009, Greg <he... at (no spam) alumni.brown.edu> wrote:

 Hopefully after
driving the car for a while, your brain learns the correct amount of
wheel
turn required to adjust for lane movement.

Hoping doesn't help much. You need an upper bound on the learning
rate to prevent nonconverging oscillations.

What!  You mean you don't use a "Hoping Parameter" in your training?
 That
combined with the Prayer Parameter are my favorite ways to accelerate
convergence

Sorry, I'm not going to take the time to write a program to generate data
for your problem. I might not even reproduce the salient aspects of your
data.

Hi Phil, Sorry maybe I didn't understand you, I thought that you
wanted the code of the program itself, so do you want only the
training images? I have 100 images, 50 of them with horizontal lines
and 50 with vertical lines, do you want me to send you this 100
images? (each one is 15x15 pixels) as a text files?

Now I have this images only in  a memory buffer while the program is
running, do you want me to turn it into text files and sent it to you
by mail? I can save each image as a small text file with 225 lines in
it, when each line contain one pixel in the image as a float number,
so the file will look something like this -

 0.821576
 0.268764
-0.498147
  .
  .
and so on....

Is that what you wanted?

Before you do any thing more, why don't you generate the gradient
images and look at them.