 |
|
| Computers Forum Index » Computer Artificial Intelligence - Philosophy » Curt is correct.... |
|
Page 1 of 1 |
|
| Author |
Message |
| casey... |
 Posted: Wed Aug 12, 2009 8:40 am |
|
|
|
Guest
|
Date: 01 Aug 2009
Subject: Re: logic from a neural network
JC:
Quote: However this helicopter operates in the real world
and yet it only uses the current state and a
probability look up table to determine its best
action just like the ttt game.
Curt wrote:
Quote: And what exactly do you think this "current state"
is that you are talking about? Where did it "come
from"?
The point is that the "current state" is not
something given to the system as a sensory input.
It's _calculated_ using temporal patterning matching
functions.
On Aug 9, 7:28 pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
Quote: casey <jgkjca... at (no spam) yahoo.com.au> wrote:
You make many assumption which you need to test. One
of them is the need for a temporal pattern for real
time behaviors yet the helicopter RL example operates
in real time and apparently doesn't require anything
more that state/actions to work.
John you have said that 3 times now. You were wrong
every time you said it. I've explained it twice why
you were wrong, but yet, apparently, not only does the
dynamics of helicopter flight go over you head, when
I take the time to explain to you how you are wrong,
you still don't understand it.
It's physically impossible to fly a helicopter without
using temporal pattern recognition. Go study physics
and come back when you understand why this is true.
Yes I am wrong!
I was thinking only about the network itself.
The temporal pattern is the velocity or acceleration.
So although the net simply associates an action with
each possible state the input spatial pattern is a
precomputed temporal pattern, a velocity value.
So essentially any simple state/action machine cannot
learn to fly a helicopter unless some of those values
in the state value are temporal patterns that have to
be computed by some other means than the network itself.
JC |
|
|
| Back to top |
|
|
|
| casey... |
| Posted: Thu Aug 13, 2009 7:57 am |
|
|
|
Guest
|
On Aug 12, 8:06 pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
Quote:
But we are really fooling ourselves when we think
like that because in no real sense is velocity
really part of the "state" at some instant point
in time.
Fooling ourselves or functionally the difference
is not detectable?
As a god like viewer we may know that the current
state of network 2 is made up of the position at
time = n and the position at time = n-1 but how
would that effect how the network 2 would treat
its current input?
+===============================+
| network 1 |
| +==========+ |
input -----+--[delay]--->| | |
| | | network 2|---------->
| +------------>| | |
| +==========+ |
+===============================+
Quote: When we take a very high speed picture in an attempt
to capture current state at some instantaneous point
in time, we certainly have no direct indication of
velocity in the picture.
If I take a picture of my car's speedometer I can tell
how fast the car was moving with just one picture :)
The current voltage at the output of electric generator
is a measure of the rate of rotation.
In the helicopter reinforcement learning project they
may not talk the way you do but their projects *work*.
The helicopter does learn to fly upside down.
This project is beyond my resources and expertise but
the principles should be the same in a simpler task.
I am more interested in "thinking" with real things
that either work or don't work than with "high level"
abstractions which cannot be tested.
JC |
|
|
| Back to top |
|
|
|
| Curt Welch... |
| Posted: Fri Aug 14, 2009 5:15 am |
|
|
|
Guest
|
casey <jgkjcasey at (no spam) yahoo.com.au> wrote:
Quote: On Aug 12, 8:06=A0pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
But we are really fooling ourselves when we think
like that because in no real sense is velocity
really part of the "state" at some instant point
in time.
Fooling ourselves or functionally the difference
is not detectable?
As a god like viewer we may know that the current
state of network 2 is made up of the position at
time =3D n and the position at time =3D n-1 but how
would that effect how the network 2 would treat
its current input?
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+
| network 1 |
| +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ |
input -----+--[delay]--->| | |
| | | network 2|----------
| +------------>| | |
| +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ |
+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D+
Hum, my newsreader is not dealing with the = format above...
Anyway, sure, from the perspective of what the network "sees" it is
"current state" to network2 in your example. But only because you built
some form of temporal memory hardware that translates historic state, in to
current state for network 2 - which is exactly what the "temporal pattern
matching function" I keep talking about does. It's exactly why I say it's
so important in solving the AI problem. It takes the temporal problem, and
turns it into a "current state" problem.
In the above, I was not talking about what the world "looks like" to some
network, but instead, what the world is really like. Meaning that, the
true current state of the helicopter does not include velocity. Velocity
is just our way of talking about a type of temporal pattern - a temporal
pattern of position changing at a certain rate.
Quote: When we take a very high speed picture in an attempt
to capture current state at some instantaneous point
in time, we certainly have no direct indication of
velocity in the picture.
If I take a picture of my car's speedometer I can tell
how fast the car was moving with just one picture 
Yes, very true. But that's not my point however. Again, you know that
only because you have used temporal memory hardware (the speedometer).
Quote: The current voltage at the output of electric generator
is a measure of the rate of rotation.
Yes, interesting example. And even with a snap-shot, I suspect that
voltage could be calculated by knowing the instantaneous location of the
elections. But again, you have pointed out an example of building a
machine to measure the temporal property by combining past events together
to create a current event the _represents_ the velocity.
Being able to build these temporal pattern matching machines is a key part
of AI in my view but because we can build an instrument which allows us to
know the velocity, doesn't mean the velocity was an inherent part of the
instantaneous state of the falling rock.
Quote: In the helicopter reinforcement learning project they
may not talk the way you do but their projects *work*.
The helicopter does learn to fly upside down.
This project is beyond my resources and expertise but
the principles should be the same in a simpler task.
I am more interested in "thinking" with real things
that either work or don't work than with "high level"
abstractions which cannot be tested.
JC
Yes, the principle is that any sort of real world control problems needs a
certain amount of historic data to solve it. You have to use some sort of
hardware that "remembers" (in one way or another) past events. They solved
the helicopter problem by using their own intelligence to figure out what
historic data was needed and then hard coded it into the solution instead
of building a learning system that could figure that out for itself. Their
reinforcement learning module didn't learn on it's own what historic data
was needed - and I believe that's a real weakness that can be ignored for a
very limited domain application like this one, but can't be ignored for
creating full human level intelligence.
This is a recurring problem with the way people solve these problems. They
hard-code a solution for a limited domain, instead finding a generic
solution that can solve all the domains that humans can learn to solve.
It's the same for how reinforcement learning was applied to backgammon in
TD-gammon as it is for this helicopter problem. There are no hard-coded
modules for calculating velocity in TD-Gammon, and there are no hard-coded
modules for mapping board positions to the neural network inputs in the
helicopter solution. What they hard coded for each of these limited domain
problems, would fail to be of any use in the other domain. Though both are
examples of using reinforcement learning to learn important parts of the
solution, neither was able to use reinforcement learning to learn the
entire solution - they hard to hard code significant parts of the solution.
To create full human intelligence, we have to do better than that. The
reinforcement learning nodule must be able to solve the whole problem, and
not just part of it. In the helicopter problem, the reinforcement learning
system should have been able to figure out on it's own that velocity (or
something like it) was required to solve the problem. And in Backgammon,
the learning module needed to figure out on it's own how to abstract
important features of board positions from many examples.
Both solutions were able to use a reinforcement learning algorithm that was
totally spatial in nature and which had no power to learn temporal
patterns. To solve AI, we need to create reinforcement learning algorithms
that work in the temporal domain and the spatial domain at the same time,
not just one that works in the spatial domain and which then relies on a
human to use their intelligence to pick out the important temporal features
for the problem, and design the hardware to translate those temporal
features into spatial inputs, for the spatial learning system to work with.
--
Curt Welch http://CurtWelch.Com/
curt at (no spam) kcwc.com http://NewsReader.Com/ |
|
|
| Back to top |
|
|
|
| casey... |
| Posted: Mon Aug 24, 2009 8:52 pm |
|
|
|
Guest
|
On Aug 13, 9:08 pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
Quote:
... I was not talking about what the world "looks
like" to some network, but instead, what the world
is really like.
What the world "looks like" is *all we have*.
We don't know what the real world is really like we
simply model it from our inputs and simpler systems
with simpler inputs have simpler world models.
Quote: If I take a picture of my car's speedometer I can tell
how fast the car was moving with just one picture :)
Yes, very true. But that's not my point however.
Again, you know that only because you have used temporal
memory hardware (the speedometer).
It is not "temporal" memory hardware it is simply memory
hardware which can hold a temporal pattern as a spatial
pattern or a spatial pattern as a spatial pattern.
JC |
|
|
| Back to top |
|
|
|
| Don Stockbauer... |
| Posted: Thu Aug 27, 2009 11:46 am |
|
|
|
Guest
|
On Aug 26, 6:27 pm, casey <jgkjca... at (no spam) yahoo.com.au> wrote:
Quote: On Aug 26, 9:55 am, c... at (no spam) kcwc.com (Curt Welch) wrote:
casey <jgkjca... at (no spam) yahoo.com.au> wrote:
On Aug 13, 9:08=A0pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
... I was not talking about what the world "looks
like" to some network, but instead, what the world
is really like.
What the world "looks like" is *all we have*.
I said "what the world looks like TO SOME NETWORK".
And what the world *looks* like to ANY network is
all its has. There is no "real world " available to
ANY network only a model of that real world.
You claim to be talking about what the world is really
like and I was making the point neither you nor any
other system can know what the world is really like.
We or it can only model A world based on our inputs.
5 MODEL SYSTEM
TEST MODEL
IF (MODEL GIVES CORRECT RESULTS, MATCHING "REALITY") GO TO 10
GO TO 5
10 CORNTINUE |
|
|
| Back to top |
|
|
|
| casey... |
| Posted: Fri Aug 28, 2009 12:59 am |
|
|
|
Guest
|
On Aug 27, 4:34 pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
Quote:
If you measure time, and "store it" in any way, then you
have created what I'm talking about as temporal hardware.
No, it is being USED to store temporal information. The
hardware itself is not "temporal". I made that clear when
I wrote that a memory device can store a temporal pattern
or a spatial pattern. The device itself exists in space
at a particular time (NOW) and its parts have spatial
relationships (which can change over time). In plain easy
to understand language your push button flip flop 1 second
timer is just that, a 1 second timer. There is no extra
understanding added by calling it a "temporal" device.
There is nothing special about it and there is no deeper
insight achieved by calling it a "temporal" device.
I understand timers and if you use them to illustrate any
AI mechanisms I will fully understand it without the need
for calling them "temporal hardware".
Take a simple oscillator. Where is the temporal pattern
"stored" that it generates as an output?
Quote: This concept of "temporal memory device" was a concept
I made up ...
No kidding!!
Quote: The way I talk IS strange.
No kidding!!
Quote: When I read books written by experts in their field I
don't find myself wondering what the hell they are on
about the way I find myself wondering about your views.
There are reasons not to always "follow the experts" John.
I never said I "follow the experts" I said I don't have any
problem understanding their point of view. At least when they
use plain language. I recognize when I am out of my depth
and that is a separate issue to "not understanding".
None of them talk about "temporal", "asynchronous", "high
resolution in time" as the magic of AI and you have not
shown any of those things doing anything special in regards
to AI.
The reason for reading the works of those who have come
before you in trying to solve the problem of getting a
machine to behave "intelligently" is to avoid duplicating
what has been tried before or running afoul of well
established facts or theories you are not aware of.
JC |
|
|
| Back to top |
|
|
|
| Curt Welch... |
| Posted: Fri Aug 28, 2009 5:15 am |
|
|
|
Guest
|
casey <jgkjcasey at (no spam) yahoo.com.au> wrote:
Quote: On Aug 27, 4:34=A0pm, c... at (no spam) kcwc.com (Curt Welch) wrote:
If you measure time, and "store it" in any way, then you
have created what I'm talking about as temporal hardware.
No, it is being USED to store temporal information. The
hardware itself is not "temporal". I made that clear when
I wrote that a memory device can store a temporal pattern
or a spatial pattern.
How could you "make clear" a term I made up???? Are you saying you changed
the definition of the term I invented so as to fit what you wanted it to be
and to add nearly infinite confusion to the conversation?
Quote: The device itself exists in space
at a particular time (NOW) and its parts have spatial
relationships (which can change over time). In plain easy
to understand language your push button flip flop 1 second
timer is just that, a 1 second timer. There is no extra
understanding added by calling it a "temporal" device.
There is nothing special about it and there is no deeper
insight achieved by calling it a "temporal" device.
Well, yes, there is important understanding in the concept - but you never
seem to grasp that. You seem to just want to keep changing the deification
of my words to fit the only way you seem able to think about these things.
Quote: I understand timers and if you use them to illustrate any
AI mechanisms I will fully understand it without the need
for calling them "temporal hardware".
Take a simple oscillator. Where is the temporal pattern
"stored" that it generates as an output?
Talking about an oscillator "storing a temporal pattern" really doesn't
make any sense to me. It does however make sense to describe it as a
temporal device because it's current state is a function of _when_ some
past events happened, and not just what happened in the past.
Quote: This concept of "temporal memory device" was a concept
I made up ...
No kidding!!
The way I talk IS strange.
No kidding!!
When I read books written by experts in their field I
don't find myself wondering what the hell they are on
about the way I find myself wondering about your views.
There are reasons not to always "follow the experts" John.
I never said I "follow the experts" I said I don't have any
problem understanding their point of view. At least when they
use plain language. I recognize when I am out of my depth
and that is a separate issue to "not understanding".
None of them talk about "temporal", "asynchronous", "high
resolution in time" as the magic of AI and you have not
shown any of those things doing anything special in regards
to AI.
The reason for reading the works of those who have come
before you in trying to solve the problem of getting a
machine to behave "intelligently" is to avoid duplicating
what has been tried before or running afoul of well
established facts or theories you are not aware of.
JC
And I never said there was any reason not to study what others had done. I
wouldn't be having this conversation and I wouldn't understand any of these
concepts if I had not spend decades studying the work of people who came
before me.
The basic concept that started all this was that I believe hardware that
learns by adjusting a temporal function is not only more powerful than one
which learns by adjusting only a spatial function, but that it's required
for strong AI.
The helicopter program (as far as I can tell) does all it's learning in the
spatial domain - and as such, has ZERO ability to _learn_ new temporal
functions. It solves the temporal problem of flight control by hard-coding
minimal temporal functions into the machine in the form of the
pre-processors which calculate things values velocity and acceleration -
and then does all it's learning strictly in the spatial domain.
Clearly, for this application, that approach worked just fine. But in
using that approach, the machine has ZERO ability to learn any NEW temporal
functions. The only ones it has, are the ones hard-coded into it by the
designers.
My argument is (and has been for about 6 years now), that if you don't give
it the power to _LEARN_ in the TEMPORAL DOMAIN - you will never create
strong AI. It's temporal behavior ability will always be limited by the
temporal functions you hard-code into the system if you don't give it the
power to learn temporal functions on its own. This is a very simple
concept I've been trying to show you for 6 years - and one you still don't
seem to understand. You don't understand the concept of a temporal
function, and instead of learning it, you just want to go back to "well
just add a timer! I can understand timers!".
My pulse sorting network is one working example of how you give a network
the power to LEARN IN THE TEMPORAL DOMAIN. It was designed the way it was
designed for the explicit reason that it was a temporal learning system.
--
Curt Welch http://CurtWelch.Com/
curt at (no spam) kcwc.com http://NewsReader.Com/ |
|
|
| Back to top |
|
|
|
| J.A. Legris... |
| Posted: Fri Aug 28, 2009 12:59 pm |
|
|
|
Guest
|
Sorry guys, but I fear the old subject line had taken on a life of
it's own.
--
Joe |
|
|
| Back to top |
|
|
|
| Curt Welch... |
| Posted: Fri Aug 28, 2009 6:56 pm |
|
|
|
Guest
|
"J.A. Legris" <jalegris at (no spam) sympatico.ca> wrote:
Quote: Sorry guys, but I fear the old subject line had taken on a life of
it's own.
Yeah, that's accurate - both the subject and the comment.
--
Curt Welch http://CurtWelch.Com/
curt at (no spam) kcwc.com http://NewsReader.Com/ |
|
|
| Back to top |
|
|
|
| Curt Welch... |
| Posted: Fri Aug 28, 2009 7:35 pm |
|
|
|
Guest
|
curt at (no spam) kcwc.com (Curt Welch) wrote:
Quote: casey <jgkjcasey at (no spam) yahoo.com.au> wrote:
My argument is (and has been for about 6 years now), that if you don't
give it the power to _LEARN_ in the TEMPORAL DOMAIN - you will never
create strong AI. It's temporal behavior ability will always be limited
by the temporal functions you hard-code into the system if you don't give
it the power to learn temporal functions on its own. This is a very
simple concept I've been trying to show you for 6 years - and one you
still don't seem to understand. You don't understand the concept of a
temporal function, and instead of learning it, you just want to go back
to "well just add a timer! I can understand timers!".
Just to try and make the point yet again.
The reason I like to talk about it as "temporal hardware" instead of just
talking about it as timers and logic functions, is because of the learning
aspect of the problem. If a human solves an engineering problem by using a
traditional approach with timers and finite state machines, the engineering
problem will be solved just fine. But to solve AI, we have to build a
machine that can design itself in response to rewards. It has to take
different modules and hook them up in the right configuration so that the
machine will perform some new complex behavior that gets it more rewards.
If that learning system has a box of timers and AND gates, and OR gates, it
becomes very difficult to make a learning algorithm find a workable design
for the hardware. Most the ways it would connect up timers and AND gates
do nothing useful at all. So the question is, as it always has been - how
do you build a learning machine, that can build hardware, which includes
the type of time-based functions we get by using timers, logic gates, and
memory chips?
My answer to this question is that instead of giving the learning function
8 different low level modules (some purely spatial like AND gates, and some
temporal like the timers) to try and build hardware from, it will learn
much faster, if you give it one low level module to build hardware from.
And if you give it only one low level module to work with, that module must
compute some type of temporal function, it's behavior must be time
sensitive so that the overall hardware built out many of these signal
components will be able to have temporal functions.
The reason I make up this langauge (and these concepts) and talk in this
way (about temporal hardware) is because it's a concept that I think is key
to solving AI (which as you know - I believe to be a problem of building a
strong learning machine). As long as you don't grasp the concept, and keep
saying - lets just talk about timers and spatial functions and finite state
machines - you will never understand the type of solution I'm talking about
and looking for.
--
Curt Welch http://CurtWelch.Com/
curt at (no spam) kcwc.com http://NewsReader.Com/ |
|
|
| Back to top |
|
|
|
| casey... |
| Posted: Sat Aug 29, 2009 4:13 am |
|
|
|
Guest
|
Curt is incorrigible and John can't understand that either
On Aug 28, 5:59 am, "J.A. Legris" <jaleg... at (no spam) sympatico.ca> wrote:
Quote:
Sorry guys, but I fear the old subject line had
taken on a life of it's own.
Well its all over now, Curt has had enough.
I always felt Curt as being intransigent rather than
incorrigible. I agreed with a lot of his views but
not so much with the way he went about it.
JC |
|
|
| Back to top |
|
|
|
| J.A. Legris... |
| Posted: Sat Aug 29, 2009 1:49 pm |
|
|
|
Guest
|
On Aug 29, 12:13 am, casey <jgkjca... at (no spam) yahoo.com.au> wrote:
Quote: Curt is incorrigible and John can't understand that either
On Aug 28, 5:59 am, "J.A. Legris" <jaleg... at (no spam) sympatico.ca> wrote:
Sorry guys, but I fear the old subject line had
taken on a life of it's own.
Well its all over now, Curt has had enough.
I always felt Curt as being intransigent rather than
incorrigible. I agreed with a lot of his views but
not so much with the way he went about it.
JC
Let's change the subject.
Recently, someone here referred to psychology as a "basket-case",
meaning (I think) that psychology lacks the conceptual soundness and
obvious success of "real" sciences such as biology, etc. AGI seems to
be similarly hobbled.
Do you have any ideas for getting around the impasse? Without
returning to the brave new world of radical behaviourism, how can AGI
obtain a solid conceptual basis that encourages scientific creativity
while remaining firmly attached to the real world?
--
Joe |
|
|
| Back to top |
|
|
|
|
|
All times are GMT
The time now is Mon Dec 07, 2009 10:13 am
|
|