 |
|
| Computers Forum Index » Computer Artificial Intelligence - Neural Nets » Choosing ANN structure... |
|
Page 1 of 1 |
|
| Author |
Message |
| bim-bom... |
 Posted: Sat Oct 24, 2009 4:09 pm |
|
|
|
Guest
|
Hello,
I will make my masters thesis about Artificial Neural Networks. But the
ANN will not run on the CPU, but on a GPU (nVidia CUDA technology) -
graphic card - when the algorithm is well parallelised, it can be many
times faster than on a CPU.
Could you please give me an advice which ANN structure (MLP, RBF,
Kohonen, etc.) to use?
My requirements for the structure are:
1. There should be at least 5 public test sets available on the internet
that could be used to train/test quality of this ANN.
2. Programs running on a GPU should have thousands of threads running in
parallel get maximum speed (ANN with just 10 neurons wouldn't be faster
on a GPU than on a CPU). So, the optimal number of neurons in most test
sets should be as big as possible.
3. For example, MLP network with input layer, one hidden layer and
output layer, there is only one parameter describing the structure of
ANN(it is number of neurons in the hidden layer). Number of neurons in
input and output layers are determined by a test set. I want ANN which
has at least 2 parameters describing the structure for a particular problem.
4. It would be nice if the ANN type rather new (less than 15-20 years
old). I prefer less known ones. There currently exist some
implementations of MLP running on CUDA (I would prefer ANN which wasn't
implemented yet in CUDA).
5. It would be nice if this ANN type was really good in what it does (it
gives better results than most of other ANNs).
It is not very important for me, if this ANN is used to function
approximation, classification, or anything else.
Thank you for your help! |
|
|
| Back to top |
|
|
|
| Phil Sherrod... |
| Posted: Sat Oct 24, 2009 5:17 pm |
|
|
|
Guest
|
On 24-Oct-2009, bim-bom <julekmen at (no spam) go2.pl> wrote:
Quote: I will make my masters thesis about Artificial Neural Networks. But the
ANN will not run on the CPU, but on a GPU (nVidia CUDA technology) -
graphic card - when the algorithm is well parallelised, it can be many
times faster than on a CPU.
Are you going to be traiining the ANN on the GPU or just executing the ANN
to make predictions? In other words, will the ANN be trained on a
conventional CPU before you begin your test?
If execution speed is of is of interest (rather than training), then you
could assign a separate GPU execution unit (CPU within the GPU?) to each
neuron. Each GPU unit will wait for data to arrive from the lower level
layer, then it can perform the weight multiplications, additions, and
compute the non-linear transfer function. (For speed, use a look-up table
for the transfer function evaluation.) Then it will mark its output as
valid and notify all neurons in the next higher level that this value is
ready for processing by them.
How many GPU execution units are available for simultaneous use?
--
Phil Sherrod
http://www.dtreg.com -- Neural networks, SVM, Decision trees |
|
|
| Back to top |
|
|
|
| Stephen Wolstenholme... |
| Posted: Sat Oct 24, 2009 5:25 pm |
|
|
|
Guest
|
On Sat, 24 Oct 2009 14:09:55 +0200, bim-bom <julekmen at (no spam) go2.pl> wrote:
Quote: Hello,
I will make my masters thesis about Artificial Neural Networks. But the
ANN will not run on the CPU, but on a GPU (nVidia CUDA technology) -
graphic card - when the algorithm is well parallelised, it can be many
times faster than on a CPU.
Could you please give me an advice which ANN structure (MLP, RBF,
Kohonen, etc.) to use?
I have been considering a CUDA based neural network but haven't made
much progress yet. It became obvious when I first considered using a
GPU that multithread learning would be essential to take advantage of
the parallel processing. I have included multithread learning in the
latest CPU version of EasyNN-plus to see just how difficult it can be
to control and what is possible. It is a MLP with BP learning. It is
interesting that some networks learn much faster with just eight
threads and two or four processors. A CPU based neural network
application that also uses CUDA must be a better solution than using
either type of processor separately. I think that using a CPU
application to construct the network and then passing the training to
the GPU is the way to go.
Steve
--
Neural Planner Software Ltd www.NPSL1.com |
|
|
| Back to top |
|
|
|
| bim-bom... |
| Posted: Sat Oct 24, 2009 8:39 pm |
|
|
|
Guest
|
Phil Sherrod pisze:
Quote: On 24-Oct-2009, bim-bom <julekmen at (no spam) go2.pl> wrote:
I will make my masters thesis about Artificial Neural Networks. But the
ANN will not run on the CPU, but on a GPU (nVidia CUDA technology) -
graphic card - when the algorithm is well parallelised, it can be many
times faster than on a CPU.
Are you going to be traiining the ANN on the GPU or just executing the ANN
to make predictions? In other words, will the ANN be trained on a
conventional CPU before you begin your test?
Well, if I will be able to parallelise the training stage, I will be
doing it on a GPU.
Quote:
If execution speed is of is of interest (rather than training), then you
could assign a separate GPU execution unit (CPU within the GPU?) to each
neuron. Each GPU unit will wait for data to arrive from the lower level
layer, then it can perform the weight multiplications, additions, and
compute the non-linear transfer function. (For speed, use a look-up table
for the transfer function evaluation.) Then it will mark its output as
valid and notify all neurons in the next higher level that this value is
ready for processing by them.
I am planning to do something like this. I will find a way to
parallelise training and execution algorithm - I just need to know a
certain ANN type that will meet my need that I have written.
Quote:
How many GPU execution units are available for simultaneous use?
My GPU has 32 processors, but the minimum number of concurrent threads
should be about 1000 to achieve the best speed. |
|
|
| Back to top |
|
|
|
| bim-bom... |
| Posted: Sat Oct 24, 2009 8:47 pm |
|
|
|
Guest
|
Stephen Wolstenholme pisze:
Quote: On Sat, 24 Oct 2009 14:09:55 +0200, bim-bom <julekmen at (no spam) go2.pl> wrote:
Hello,
I will make my masters thesis about Artificial Neural Networks. But the
ANN will not run on the CPU, but on a GPU (nVidia CUDA technology) -
graphic card - when the algorithm is well parallelised, it can be many
times faster than on a CPU.
Could you please give me an advice which ANN structure (MLP, RBF,
Kohonen, etc.) to use?
I have been considering a CUDA based neural network but haven't made
much progress yet. It became obvious when I first considered using a
GPU that multithread learning would be essential to take advantage of
the parallel processing. I have included multithread learning in the
latest CPU version of EasyNN-plus to see just how difficult it can be
to control and what is possible. It is a MLP with BP learning. It is
interesting that some networks learn much faster with just eight
threads and two or four processors. A CPU based neural network
application that also uses CUDA must be a better solution than using
either type of processor separately. I think that using a CPU
application to construct the network and then passing the training to
the GPU is the way to go.
Well, executing MLP is a very parallelisable algorithm - Such algorithms
can be even 50 times faster on a fast GPU than on a fast CPU. But it is
very important how your program uses memory. Look here:
http://www.codeproject.com/KB/graphics/GPUNN.aspx
I have written the last (first on the list) comment. By some simple
optimizations, I made one CUDA kernel 10 times faster...
OK, I'm still waiting for your propositions... |
|
|
| Back to top |
|
|
|
|
|
All times are GMT
The time now is Sun Nov 29, 2009 7:42 am
|
|