Main Page | Report this Page
 
   
Science Forum Index  »  Statistics - Education Forum  »  Looking for feedback for an online multivariate regression t
Page 3 of 3    Goto page Previous  1, 2, 3
Author Message
Greg Heath
Posted: Sun Dec 10, 2006 5:41 am
Guest
Reef Fish wrote:
Quote:
David A. Heiser wrote:
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in message
news:1165596313.275506.72380@n67g2000cwd.googlegroups.com...

Greg Heath wrote:
... So in summary,

1. Greg Heath claims

It can be useful to inspect the components of Cxx
and Cyx, submatrices of the all-variable correlation
coefficient matrix, to help understand the relationships
among variables.

Although it is not necessary and is seldom sufficient, the
inspection can be very helpful.

Quote:
The most useful time is when multicollinearities are present.

That is the LEAST useful time.

Of course I don't agree.

Quote:
A simple eigenvalue
eigenvector analysis and the use of model-building
technique of eliminating the REDUNDUNT variable
in the equation is all that's necessary as the CURE.

It is sufficient but not always necessary.

"Simple" is in the eyes of the beholder. You have no
idea of how fast the eyes of a contract monitor with a BA
in Geology can cloud over when you attempt to explain
your PhD level electrical and mechanical engineering
research using an eigenvector analysis.

If a simple explanation is needed, sometimes using
a display of [Cxx Cyx] is helpful.

Quote:
++++++++++++++++++++++++++++++++++++
-----SNIP
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4, #1, page
120.

What's a "Scree plot"?

What you have termed an elbow plot: SSE/(n-p+1) vs p.

Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Quote:
In the method of eliminating REDUNDANT variables in a multiple
regression, there are very well-defined and analytic methods of
examining the partial correlations in deciding which is the variable
to REMOVE that will least affect the FIT.

I think David is referring to PCR and which PCs to remove.

Quote:
No one is talking about the cut off point or what value to use.

True.

Quote:
I think you are jumping into the middle of the SUMMARY Greg and
I were discussing about OUR points about multicollinearity and
its accommodation with something that is completely off the wall
on YOUR part.

Lighten up Reef. You can't expect every replier to have a clear
picture of our months-long disagreement . He's a little OT. So
what?

Quote:
If you don't use a covariance matriz, but the intial data matrix then you
may get complex eigenvalues, which actually are more informative (in polar
form), but can't be nicely plotted.as a Scree plot.

?? Eigenvalues of a nonsquare data matrix ??

Quote:
You are talking through your HAT, and I don't mean the HAT matrix
associated with a regression problem or any of the regression
estimates that are called the beta-hats either.

Yes David, you seem to be way out in left field with that last
paragraph.

Greg
Reef Fish
Posted: Sun Dec 10, 2006 9:49 am
Guest
Greg Heath wrote:

GH> ... So in summary,

GH> 1. Greg Heath claims
<see claim below>
GH> 2. Reef Fish claims
<see claim below>
GH> Hope that ends this thread.

I AGREED that both claims were CORRECT, and made
a brief summary of why you were wrong. The rest are
ALL in the archives.

I thought for a minute you were back arguing again. But I see
you added a summary comment to repeat your claim, so I assume
that OUR thread of disagreement has indeed ENDED.

Each reader can go back to the archives and read what we each
had to say.

Quote:
1. Greg Heath claims

It can be useful to inspect the components of Cxx
and Cyx, submatrices of the all-variable correlation
coefficient matrix, to help understand the relationships
among variables.
The most useful time is when multicollinearities are present.

I agreed that was your claim and merely explained WHY
RF > That is the LEAST useful time.

Quote:
If a simple explanation is needed, sometimes using
a display of [Cxx Cyx] is helpful.

That is a repeat of the same claim.
Quote:
++++++++++++++++++++++++++++++++++++
-----SNIP

The rest of your post was about David Heiser's unwarranted
comments when he jumped into the middle of our disagreement
without even knowing that we were discussing ordinary
multiple REGRESSION!

I said he was "talking through his HAT"

and you said he was "way out in left field"

so we AGREED at least on that part, and let's see David Heiser's
worthless as well as improper comments:

David Heiser wrote,
Quote:
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4, #1, page
120.

RF > What's a "Scree plot"?

GH> What you have termed an elbow plot: SSE/(n-p+1) vs p.


Then why not call it so? The term "Scree plot" was never used in any
previous discussion. Why introduce it at this point? Besides, my
"elbow plot" was the Standard deviation of residuals (sqrt of MSE)
vs p (the number of independent variables).

The MSE of a regression is SSE/(n-p-1).

Was the SSE/(n-p+1) your error/typo or Karyongo's/Heiser's error?

Quote:
Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Curvature is undefined. "small marginal decrease" and "increase"
are well-defined and well-understood terms.

Quote:

In the method of eliminating REDUNDANT variables in a multiple
regression, there are very well-defined and analytic methods of
examining the partial correlations in deciding which is the variable
to REMOVE that will least affect the FIT.

I think David is referring to PCR and which PCs to remove.

PCR was never mentioned. WE were discussion ordinary multiple
regression and David Heiser jumped in with a completely nonsensical
comment:

DH> There are no p values associated with eigenvalues. Consequently
DH> it is a matter of eyeball interpretation of a Scree plot.
Quote:

RF> > No one is talking about the cut off point or what value to use.

GH> True.

I think you are jumping into the middle of the SUMMARY Greg and
I were discussing about OUR points about multicollinearity and
its accommodation with something that is completely off the wall
on YOUR part.

Lighten up Reef. You can't expect every replier to have a clear
picture of our months-long disagreement . He's a little OT. So
what?

So what? I expect everyone who BUTTS into someone else's
disagreement to know at least what the PROBLEM was, let
alone what the disagreement was.

So. if he jumps into a month-long disagreement, he should at least
KNOW what the disagreement was about!

That is his RESPONSIBILITY as a responsible discussant.

So what? So, he should have kept his mouth SHUT.


Quote:

If you don't use a covariance matriz, but the intial data matrix then you
may get complex eigenvalues, which actually are more informative (in polar
form), but can't be nicely plotted.as a Scree plot.

?? Eigenvalues of a nonsquare data matrix ??

You are talking through your HAT, and I don't mean the HAT matrix
associated with a regression problem or any of the regression
estimates that are called the beta-hats either.

Yes David, you seem to be way out in left field with that last
paragraph.

Greg

So, I take it that OUR disagreement STOOD and we agreed to have
no more argument about the disagreement and let the readers read
what is in the archives.

That's a MAJOR step toward progress. Richard Ulrich should take
very careful note!

-- Reef Fish Bob.
David A. Heiser
Posted: Sun Dec 10, 2006 2:49 pm
Guest
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in message
news:1165758550.122579.111320@j72g2000cwa.googlegroups.com...
Quote:

Greg Heath wrote:

GH> ... So in summary,

GH> 1. Greg Heath claims
see claim below
GH> 2. Reef Fish claims
see claim below
GH> Hope that ends this thread.

I AGREED that both claims were CORRECT, and made
a brief summary of why you were wrong. The rest are
ALL in the archives.

I thought for a minute you were back arguing again. But I see
you added a summary comment to repeat your claim, so I assume
that OUR thread of disagreement has indeed ENDED.

Each reader can go back to the archives and read what we each
had to say.

1. Greg Heath claims

It can be useful to inspect the components of Cxx
and Cyx, submatrices of the all-variable correlation
coefficient matrix, to help understand the relationships
among variables.
The most useful time is when multicollinearities are present.

I agreed that was your claim and merely explained WHY
RF > That is the LEAST useful time.

If a simple explanation is needed, sometimes using
a display of [Cxx Cyx] is helpful.

That is a repeat of the same claim.
++++++++++++++++++++++++++++++++++++
-----SNIP

The rest of your post was about David Heiser's unwarranted
comments when he jumped into the middle of our disagreement
without even knowing that we were discussing ordinary
multiple REGRESSION!

I said he was "talking through his HAT"

and you said he was "way out in left field"

so we AGREED at least on that part, and let's see David Heiser's
worthless as well as improper comments:

David Heiser wrote,
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail
is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should
be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4,
#1, page
120.

RF > What's a "Scree plot"?

GH> What you have termed an elbow plot: SSE/(n-p+1) vs p.

Then why not call it so? The term "Scree plot" was never used in any
previous discussion. Why introduce it at this point? Besides, my
"elbow plot" was the Standard deviation of residuals (sqrt of MSE)
vs p (the number of independent variables).

The MSE of a regression is SSE/(n-p-1).

Was the SSE/(n-p+1) your error/typo or Karyongo's/Heiser's error?

Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Curvature is undefined. "small marginal decrease" and "increase"
are well-defined and well-understood terms.


In the method of eliminating REDUNDANT variables in a multiple
regression, there are very well-defined and analytic methods of
examining the partial correlations in deciding which is the variable
to REMOVE that will least affect the FIT.

I think David is referring to PCR and which PCs to remove.

PCR was never mentioned. WE were discussion ordinary multiple
regression and David Heiser jumped in with a completely nonsensical
comment:

DH> There are no p values associated with eigenvalues. Consequently
DH> it is a matter of eyeball interpretation of a Scree plot.

RF> > No one is talking about the cut off point or what value to use.

GH> True.

I think you are jumping into the middle of the SUMMARY Greg and
I were discussing about OUR points about multicollinearity and
its accommodation with something that is completely off the wall
on YOUR part.

Lighten up Reef. You can't expect every replier to have a clear
picture of our months-long disagreement . He's a little OT. So
what?

So what? I expect everyone who BUTTS into someone else's
disagreement to know at least what the PROBLEM was, let
alone what the disagreement was.

So. if he jumps into a month-long disagreement, he should at least
KNOW what the disagreement was about!

That is his RESPONSIBILITY as a responsible discussant.

So what? So, he should have kept his mouth SHUT.



If you don't use a covariance matriz, but the intial data matrix then
you
may get complex eigenvalues, which actually are more informative (in
polar
form), but can't be nicely plotted.as a Scree plot.

?? Eigenvalues of a nonsquare data matrix ??

You are talking through your HAT, and I don't mean the HAT matrix
associated with a regression problem or any of the regression
estimates that are called the beta-hats either.

Yes David, you seem to be way out in left field with that last
paragraph.

Greg

So, I take it that OUR disagreement STOOD and we agreed to have
no more argument about the disagreement and let the readers read
what is in the archives.

That's a MAJOR step toward progress. Richard Ulrich should take
very careful note!

-- Reef Fish Bob.
+++++++++++++++++++++++++++++++++++++++++++++++

I'm sorry that I didn't know that the sci.stat.* groups are only there to
provide a vehicle for Reef Fish, and is closed to everyone else. Apparently
only those "selected" responders are allowed to debate. The territory and
views are very narrow. No references outside of the "circle". No reading of
current ASA publications or of books outside of the "circle of Reef Fish's
memory".

David Heiser
Reef Fish
Posted: Sun Dec 10, 2006 10:05 pm
Guest
David A. Heiser wrote:
Quote:
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in message
news:1165758550.122579.111320@j72g2000cwa.googlegroups.com...

Greg Heath wrote:

GH> ... So in summary,

GH> 1. Greg Heath claims
see claim below
GH> 2. Reef Fish claims
see claim below
GH> Hope that ends this thread.

I AGREED that both claims were CORRECT, and made
a brief summary of why you were wrong. The rest are
ALL in the archives.

I thought for a minute you were back arguing again. But I see
you added a summary comment to repeat your claim, so I assume
that OUR thread of disagreement has indeed ENDED.

Each reader can go back to the archives and read what we each
had to say.

1. Greg Heath claims

It can be useful to inspect the components of Cxx
and Cyx, submatrices of the all-variable correlation
coefficient matrix, to help understand the relationships
among variables.
The most useful time is when multicollinearities are present.

I agreed that was your claim and merely explained WHY
RF > That is the LEAST useful time.

If a simple explanation is needed, sometimes using
a display of [Cxx Cyx] is helpful.

That is a repeat of the same claim.
++++++++++++++++++++++++++++++++++++
-----SNIP

The rest of your post was about David Heiser's unwarranted
comments when he jumped into the middle of our disagreement
without even knowing that we were discussing ordinary
multiple REGRESSION!

I said he was "talking through his HAT"

and you said he was "way out in left field"

so we AGREED at least on that part, and let's see David Heiser's
worthless as well as improper comments:

David Heiser wrote,
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail
is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should
be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4,
#1, page
120.

RF > What's a "Scree plot"?

GH> What you have termed an elbow plot: SSE/(n-p+1) vs p.

Then why not call it so? The term "Scree plot" was never used in any
previous discussion. Why introduce it at this point? Besides, my
"elbow plot" was the Standard deviation of residuals (sqrt of MSE)
vs p (the number of independent variables).

The MSE of a regression is SSE/(n-p-1).

Was the SSE/(n-p+1) your error/typo or Karyongo's/Heiser's error?

Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Curvature is undefined. "small marginal decrease" and "increase"
are well-defined and well-understood terms.


In the method of eliminating REDUNDANT variables in a multiple
regression, there are very well-defined and analytic methods of
examining the partial correlations in deciding which is the variable
to REMOVE that will least affect the FIT.

I think David is referring to PCR and which PCs to remove.

PCR was never mentioned. WE were discussion ordinary multiple
regression and David Heiser jumped in with a completely nonsensical
comment:

DH> There are no p values associated with eigenvalues. Consequently
DH> it is a matter of eyeball interpretation of a Scree plot.

RF> > No one is talking about the cut off point or what value to use.

GH> True.

I think you are jumping into the middle of the SUMMARY Greg and
I were discussing about OUR points about multicollinearity and
its accommodation with something that is completely off the wall
on YOUR part.

Lighten up Reef. You can't expect every replier to have a clear
picture of our months-long disagreement . He's a little OT. So
what?

So what? I expect everyone who BUTTS into someone else's
disagreement to know at least what the PROBLEM was, let
alone what the disagreement was.

So. if he jumps into a month-long disagreement, he should at least
KNOW what the disagreement was about!

That is his RESPONSIBILITY as a responsible discussant.

So what? So, he should have kept his mouth SHUT.



If you don't use a covariance matriz, but the intial data matrix then
you
may get complex eigenvalues, which actually are more informative (in
polar
form), but can't be nicely plotted.as a Scree plot.

?? Eigenvalues of a nonsquare data matrix ??

You are talking through your HAT, and I don't mean the HAT matrix
associated with a regression problem or any of the regression
estimates that are called the beta-hats either.

Yes David, you seem to be way out in left field with that last
paragraph.

Greg

So, I take it that OUR disagreement STOOD and we agreed to have
no more argument about the disagreement and let the readers read
what is in the archives.

That's a MAJOR step toward progress. Richard Ulrich should take
very careful note!

-- Reef Fish Bob.
+++++++++++++++++++++++++++++++++++++++++++++++
I'm sorry that I didn't know that the sci.stat.* groups are only there to
provide a vehicle for Reef Fish, and is closed to everyone else.

That's not a smart thing for anyone to say who jumped into the midst
of a month-old discussion and didn't have the slightest idea that it
was about MULTIPLE REGRESSION.

You just made a fool of yourself, in this group. If you had done the
same in any other group, with or without Reef Fish, it would have
been the same foolish thing on your part.

Why don't you read over what YOU said, and see that it didn't
apply to ANYTHING Greg Heath and I were discussing!

Quote:
Apparently
only those "selected" responders are allowed to debate.

What debate? You didn't even know the SUBJECT under
discussion!

Quote:
The territory and
views are very narrow. No references outside of the "circle". No reading of
current ASA publications or of books outside of the "circle of Reef Fish's
memory".

David Heiser

You acted foolishly and made irrelevant and OT comments on
what Greg Heath and I were discussing. Greg and I BOTH saw
it that way.

You are compounding your foolishness by making this gratuitous
post to try to excuse yourself.

-- Reef Fish Bob.
Greg Heath
Posted: Mon Dec 11, 2006 10:12 am
Guest
Reef Fish wrote:
Quote:
Greg Heath wrote:

-----SNIP

Quote:
The rest of your post was about David Heiser's unwarranted
comments when he jumped into the middle of our disagreement
without even knowing that we were discussing ordinary
multiple REGRESSION!

I said he was "talking through his HAT"

and you said he was "way out in left field"

so we AGREED at least on that part, and let's see David Heiser's
worthless as well as improper comments:

David Heiser wrote,
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4, #1, page
120.

RF > What's a "Scree plot"?

GH> What you have termed an elbow plot: SSE/(n-p+1) vs p.

Corrected typo: SSE/(n-p-1) vs p.

Quote:
Then why not call it so? The term "Scree plot" was never used
in any previous discussion. Why introduce it at this point?

1.Because PCA texts (Jackson and/or Jollife ?) call it a scree plot.
2. AFAIK the term "elbow" was coined by you
3. David probably never read your elbow post

Quote:
Besides, my "elbow plot" was the Standard deviation of residuals
(sqrt of MSE) vs p (the number of independent variables).

The MSE of a regression is SSE/(n-p-1).

Sorry. Let's back up:

Quote:
RF > What's a "Scree plot"?

GH> The square of what you have termed an elbow plot: SSE/(n-p-1) vs p.

Satisfied?

Quote:
Was the SSE/(n-p+1) your error/typo or Karyongo's/Heiser's error?

My typo. It was never mentioned by Dave and I didn't look at the
K reference.

Quote:
Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Curvature is undefined. "small marginal decrease" and "increase"
are well-defined and well-understood terms.

Curvature of a smooth curve is well defined in 1st year calculus. To
use with data, estimate 2nd derivatives with 2nd differences.

Quote:
In the method of eliminating REDUNDANT variables in a multiple
regression, there are very well-defined and analytic methods of
examining the partial correlations in deciding which is the variable
to REMOVE that will least affect the FIT.

I think David is referring to PCR and which PCs to remove.

PCR was never mentioned. WE were discussion ordinary multiple
regression and David Heiser jumped in with a completely nonsensical
comment:

DH> There are no p values associated with eigenvalues. Consequently
DH> it is a matter of eyeball interpretation of a Scree plot.

It's not nonsensical if you realize that he was thinking of variable
reduction in the context of PCR. However, he is probably wrong
w.r.t. the use of p-values with PCR

Quote:
RF> > No one is talking about the cut off point or what value to use.

GH> True.

I think you are jumping into the middle of the SUMMARY Greg and
I were discussing about OUR points about multicollinearity and
its accommodation with something that is completely off the wall
on YOUR part.

Lighten up Reef. You can't expect every replier to have a clear
picture of our months-long disagreement . He's a little OT.
So what?

So what? I expect everyone who BUTTS into someone else's
disagreement to know at least what the PROBLEM was, let
alone what the disagreement was.

So. if he jumps into a month-long disagreement, he should at least
KNOW what the disagreement was about!

That is his RESPONSIBILITY as a responsible discussant.

So what? So, he should have kept his mouth SHUT.

Unwarranted...worthless...improper...keep mouth shut

Hmm...

Some time ago Reef Fish, the owner of this newsgroup, mentioned
that he was going to let it be a public forum. When I read that I got
the impression that he was going to let anyone post and reply
to any thread without getting the owner's permission.

I guess I was mistaken.

Maybe he should have made it clear where on the internet one
can find the application forms for
a. beginning a thread
b. replying to a thread

Or can it both be done with the same form?

Maybe he should have also made it clear whether or not
these forms are copywrited and whether or not breaching
the copywrite would result in fines and/or prosecution.

Quote:
If you don't use a covariance matriz, but the intial data matrix then you
may get complex eigenvalues, which actually are more informative (in polar
form), but can't be nicely plotted.as a Scree plot.

?? Eigenvalues of a nonsquare data matrix ??

You are talking through your HAT, and I don't mean the HAT matrix
associated with a regression problem or any of the regression
estimates that are called the beta-hats either.

Yes David, you seem to be way out in left field with that last
paragraph.

David, will you please clarify what you were thinking about when
you wrote that?

Greg
Reef Fish
Posted: Mon Dec 11, 2006 6:56 pm
Guest
Greg Heath wrote:
Quote:
Reef Fish wrote:
Greg Heath wrote:

-----SNIP

The rest of your post was about David Heiser's unwarranted
comments when he jumped into the middle of our disagreement
without even knowing that we were discussing ordinary
multiple REGRESSION!

I said he was "talking through his HAT"

and you said he was "way out in left field"

so we AGREED at least on that part, and let's see David Heiser's
worthless as well as improper comments:

David Heiser wrote,
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4, #1, page
120.

RF > What's a "Scree plot"?

GH> What you have termed an elbow plot: SSE/(n-p+1) vs p.

Corrected typo: SSE/(n-p-1) vs p.

Then why not call it so? The term "Scree plot" was never used
in any previous discussion. Why introduce it at this point?

1.Because PCA texts (Jackson and/or Jollife ?) call it a scree plot.

I see you've been reading the wrong books too. Besides, nobody was
talking about PCA except David Heiser.

Quote:
2. AFAIK the term "elbow" was coined by you

No. I borrowed that term from Joe Kruskal (or Multidimensional
Scaling fame) who recommended using the "Elbow Rule" to
decide on the number of dimensions to use, his his Stress plot
vs dimension. Obvious, the idea I used in choosing the number
of variables to use is the same (look at the marginal decrease to
see when the diminishing return is not worth an additional variable).

When I was talking about the elbow rule, several of the discussants
mentioned they saw that kind of plots in STATISTICAL textbooks,
as soon as I pointed out one. m00es (remember him?) said he
recognized the DATA immediately in the example I cited such a
plot in a textbook.

Quote:
3. David probably never read your elbow post

David never read 99% of the discussions related to multiple
regression in these groups. I even pointed him to read some of
the threads in the archives. He is just like Ulrich in that regard.


Quote:

Besides, my "elbow plot" was the Standard deviation of residuals
(sqrt of MSE) vs p (the number of independent variables).

The MSE of a regression is SSE/(n-p-1).

Sorry. Let's back up:

RF > What's a "Scree plot"?

GH> The square of what you have termed an elbow plot: SSE/(n-p-1) vs p.

Satisfied?

You already answered it in this post above. I asked the question only
ONCE. Are you just repeating the same clip and answered it twice, and
threw in your smart ass remark "Satistified?" the second time.


Quote:
Was the SSE/(n-p+1) your error/typo or Karyongo's/Heiser's error?

My typo. It was never mentioned by Dave and I didn't look at the
K reference.

He also misspelled the name of the author -- not that it would make a
bit of different who that clown is, Koyongo or Karyongo.

Quote:

Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Curvature is undefined. "small marginal decrease" and "increase"
are well-defined and well-understood terms.

Curvature of a smooth curve is well defined in 1st year calculus. To
use with data, estimate 2nd derivatives with 2nd differences.

I see you're still at the 1st year calculus level? Never kind what
you
said is an ABSURD idea to use even if the curve IS continuouly
differentiable. The CURVE to which the "Elbow rule" (Kruskal's
or mind) is a piecewise linear function with joints (without
derivatives) at the dimensional points.

I think your post has deteriorated to the point of just you ARGUING
without using your brains to do any THINKING.

You FREE lesson privilege has NOT be restored, although you have
agreed to terminate your month-long argument about your use of
correlations.

We DID terminate that.

If David Heiser want to show his ignorance, let HIM -- I am suggesting
that to spare you of further nonsense such as the curvature and
second derivatives on piecewise linear functions.

Quote:
Greg

Rest of Greg Heath's smart ass remarks (with no more substance over
what had already been covered) deleted.

-- Reef Fish Bob.
Richard Ulrich
Posted: Tue Dec 12, 2006 12:58 am
Guest
On 11 Dec 2006 14:56:27 -0800, "Reef Fish"
<large_nassua_grouper@yahoo.com> wrote:

[snip]
Quote:

3. David probably never read your elbow post

David never read 99% of the discussions related to multiple
regression in these groups. I even pointed him to read some of
the threads in the archives. He is just like Ulrich in that regard.

Ulrich, like everyone else, finds it painful to look back
at any old Reef Fish posts because Reef Fish contaminates
almost every thread with rhetorical ugliness, and otherwise
fills his posts (very commonly) with repetition.


By the way, I recognized the idea of the elbow plot
immediately, though it was mainly familiar to me as a
scree plot. I also remembered an earlier discussion, which
I find in sci.stat.math, Googling groups for
< group:sci.stat.* elbow scree >.

Jerry Dallal Replied to Bob on Feb. 10 --
=== JD, near the beginning of Reply to RF.
Kruskal's "Elbow Rule Plots" are called Scree Plots in some quarters,
"scree" being "A slope of loose rock debris at the base of a steep
incline or cliff." The plots are not uncommon. See, for example, p477
of Neter, Wasserman, and Kutner "Applied Linear Statistical Models,
3rd" for the plot, but not the name. ...
=== end of excerpt.



--
Rich Ulrich, wpilib@pitt.edu
http://www.pitt.edu/~wpilib/index.html
Reef Fish
Posted: Tue Dec 12, 2006 2:25 am
Guest
Richard Ulrich wrote:
Quote:
On 11 Dec 2006 14:56:27 -0800, "Reef Fish"
large_nassua_grouper@yahoo.com> wrote:

[snip]

3. David probably never read your elbow post

David never read 99% of the discussions related to multiple
regression in these groups. I even pointed him to read some of
the threads in the archives. He is just like Ulrich in that regard.

Ulrich, like everyone else, finds it painful to look back
at any old Reef Fish posts because Reef Fish

pointed out so many of Richard Ulrich's statistical ERRORS and
his malpractice of statistics.


Quote:
By the way, I recognized the idea of the elbow plot
immediately, though it was mainly familiar to me as a
scree plot. I also remembered an earlier discussion, which
I find in sci.stat.math, Googling groups for
group:sci.stat.* elbow scree >.

Jerry Dallal Replied to Bob on Feb. 10 --
=== JD, near the beginning of Reply to RF.
Kruskal's "Elbow Rule Plots" are called Scree Plots in some quarters,
"scree" being "A slope of loose rock debris at the base of a steep
incline or cliff." The plots are not uncommon. See, for example, p477
of Neter, Wasserman, and Kutner "Applied Linear Statistical Models,
3rd" for the plot, but not the name. ...
=== end of excerpt.

I do remember his reference to Neter et al, since I've used that book
as reference or textbooks in my courses for over 20 years. Those
plots are NOT called Scree Plots in any books in statistics.

-- Reef Fish Bob.
Reef Fish
Posted: Tue Dec 12, 2006 4:05 am
Guest
The ERROR in the subject of mislabeling a MULTIPLE regression as
a "multivariate regression" originated in a post by Owen in
sci.stat.edu.

In my very first post, I pointed out that "multivariate regression" in
the
subject was a misnomer, as I said to Old Mac User:

RF> On the other link, it had "multiple regression" which is NOT
RF> "multivariate regression", so that's like Richard Ulrich calling

RF> Y = bX a nonlinear regression model. :-)

NOW, we are 40 posts into that thread, so it's about time that we
change the subject to the correct MULTIPLE regresssion label.

Richard Ulrich entered on the cited line above calling that factual
statement a "slur" -- the FACT that he called it a nonlinear regression
model. My response was then cross-posted to all three groups
because it was about the subject of "variable selection in multiple
regression" which had been widely discussed in all three groups.

That was the 7th post in the thread in sci.stat.edu, and the OP in
the other two groups.

In bounced David Heiser, not having read anything about the subject,

Quote:
Reef Fish Bob's messages over the year
(and prior years) have stressed to fact that linear regression depends on
the predictor variables being independent.

He meant statistically independent (and uncorrelated) -- which is
COMPLETELY opposite to what I had been posting about LINEAR
independence of the indep vars. and not stochastic independence.

That and other comments by David Heiser including his confusion
about a polynomial model not being a LINEAR model (same error
as Richard Ulrich had made) made me conclude the post with this
assessment:

RF> Right now, you are VERY confused and deficient about your
RF> understanding of linear regression problems. My diagnosis
RF> is with 100% confidence.

David Heiser's further misconceptions brought in Greg Heath with
his rehash of the use of correlations for multicollinearity diagnosis
which I said was WRONG. (The one we had just concluded on
Greg's statement of his position and mine).

It was toward the end of the discussion of Greg and me about the
multicollinearity problem that David Heiser butted in AGAIN, this
time with his competely off-the-wall statements about p-values of
eigenvalues and complex eigenvalues none of which had
ANYTHING to do with the entire thread.

That's the summary of the 40 posts in the thread up to now.


Reef Fish wrote:
Quote:
Richard Ulrich wrote:
On 11 Dec 2006 14:56:27 -0800, "Reef Fish"
large_nassua_grouper@yahoo.com> wrote:

[snip]

3. David probably never read your elbow post

David never read 99% of the discussions related to multiple
regression in these groups. I even pointed him to read some of
the threads in the archives. He is just like Ulrich in that regard.

Ulrich, like everyone else, finds it painful to look back
at any old Reef Fish posts because Reef Fish

pointed out so many of Richard Ulrich's statistical ERRORS and
his malpractice of statistics.

So, in this thread, after Owen asked his initial question mislabeling
"multivariate regression" and Old Mac User made some comments
about multiple regression, the rest of the "discussants" (from post
#6 to post #40 of the original thread) were Richard Ulrich, David
Heiser, and Greg Heath, each contributed their own errors in the
multiple regression problems.

My discussion with Greg Heath had terminated, by mutual consent.

This was my final remark to Greg:

RF> If David Heiser want to show his ignorance, let HIM -- I am
RF> suggesting that to spare you of further nonsense such as
RF> the curvature and second derivatives on piecewise linear
RF> functions.

As for Richard Ulrich, here's the advice you gave Ethan.johnsons
that you should follow yourself:

Quote:
You apparently need to go back to the start, and learn what
these tests do, and something about how they do it.
Pick up a textbook or two and browse.

-- Reef Fish Bob.
Greg Heath
Posted: Tue Dec 12, 2006 7:00 am
Guest
Reef Fish wrote:
Quote:
Greg Heath wrote:
Reef Fish wrote:
Greg Heath wrote:

-----SNIP

The rest of your post was about David Heiser's unwarranted
comments when he jumped into the middle of our disagreement
without even knowing that we were discussing ordinary
multiple REGRESSION!

I said he was "talking through his HAT"

and you said he was "way out in left field"

so we AGREED at least on that part, and let's see David Heiser's
worthless as well as improper comments:

David Heiser wrote,
Consequently it is a
matter of eyeball interpretation of a Scree plot. When there are many
variables (common in social research), and the Scree plot right tail is
generally decreasing without a defined break, there are different
intreptations by different people on where the cutoff point should be. See
Karyongo in Journal of Modern Applied Statistical Methods, Vol. 4, #1, page
120.

RF > What's a "Scree plot"?

GH> What you have termed an elbow plot: SSE/(n-p+1) vs p.

Corrected typo: SSE/(n-p-1) vs p.

Then why not call it so? The term "Scree plot" was never used
in any previous discussion. Why introduce it at this point?

1.Because PCA texts (Jackson and/or Jollife ?) call it a scree plot.

I see you've been reading the wrong books too. Besides, nobody was
talking about PCA except David Heiser.

2. AFAIK the term "elbow" was coined by you

No. I borrowed that term from Joe Kruskal (or Multidimensional
Scaling fame) who recommended using the "Elbow Rule" to
decide on the number of dimensions to use, his his Stress plot
vs dimension. Obvious, the idea I used in choosing the number
of variables to use is the same (look at the marginal decrease to
see when the diminishing return is not worth an additional variable).

When I was talking about the elbow rule, several of the discussants
mentioned they saw that kind of plots in STATISTICAL textbooks,
as soon as I pointed out one. m00es (remember him?) said he
recognized the DATA immediately in the example I cited such a
plot in a textbook.

Hmm, I thought I remember the term "scree" used in Kruskal's
MDS book also.

Quote:
3. David probably never read your elbow post

David never read 99% of the discussions related to multiple
regression in these groups. I even pointed him to read some of
the threads in the archives. He is just like Ulrich in that regard.

Besides, my "elbow plot" was the Standard deviation of residuals
(sqrt of MSE) vs p (the number of independent variables).

The MSE of a regression is SSE/(n-p-1).

Sorry. Let's back up:

RF > What's a "Scree plot"?

GH> The square of what you have termed an elbow plot: SSE/(n-p-1) vs p.

Satisfied?

You already answered it in this post above. I asked the question only
ONCE. Are you just repeating the same clip and answered it twice, and
threw in your smart ass remark "Satistified?" the second time.

Yep.

Quote:
Was the SSE/(n-p+1) your error/typo or Karyongo's/Heiser's error?

My typo. It was never mentioned by Dave and I didn't look at the
K reference.

He also misspelled the name of the author -- not that it would make a
bit of different who that clown is, Koyongo or Karyongo.

Some have chosen the cutoff w.r.t. where the plot has
maximum curvature.

Curvature is undefined. "small marginal decrease" and "increase"
are well-defined and well-understood terms.

Curvature of a smooth curve is well defined in 1st year calculus. To
use with data, estimate 2nd derivatives with 2nd differences.

I see you're still at the 1st year calculus level? Never kind what
you
said is an ABSURD idea to use even if the curve IS continuouly
differentiable. The CURVE to which the "Elbow rule" (Kruskal's
or mind) is a piecewise linear function with joints (without
derivatives) at the dimensional points.

I was just reporting a result in a published article I downloaded
from the net. The results looked reasonable. The finite difference
approximation to the curvature did yield a maximum near where
some would claim there is an elbow.

-----SNIP
Quote:
If David Heiser want to show his ignorance, let HIM -- I am suggesting
that to spare you of further nonsense such as the curvature and
second derivatives on piecewise linear functions.

Hey, don't shoot the messenger.

Greg
Reef Fish
Posted: Tue Dec 12, 2006 11:37 am
Guest
Greg Heath wrote:
Quote:

Hmm, I thought I remember the term "scree" used in Kruskal's
MDS book also.

I learned MDS from Joe Kruskal himself, in 1967, shortly after his
publication of his 1963 papers on MDS.

He used the "Elbow Rule" then.

What BOOK on MDS did he write?

Quote:
If David Heiser want to show his ignorance, let HIM -- I am suggesting
that to spare you of further nonsense such as the curvature and
second derivatives on piecewise linear functions.

Hey, don't shoot the messenger.

Only corrected the errors the messenger introduced himself!
Quote:

Greg

-- Reef Fish Bob,
David Winsemius
Posted: Sun Dec 17, 2006 12:22 pm
Guest
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in news:1165937871.090284.265210
@f1g2000cwa.googlegroups.com:

Quote:
Hmm, I thought I remember the term "scree" used in Kruskal's
MDS book also.

I learned MDS from Joe Kruskal himself, in 1967, shortly after his
publication of his 1963 papers on MDS.

He used the "Elbow Rule" then.

What BOOK on MDS did he write?

http://www.amazon.com/exec/obidos/ASIN/0803909403
http://www.amazon.co.uk/exec/obidos/ASIN/0803930682
Reef Fish
Posted: Sun Dec 17, 2006 12:32 pm
Guest
David Winsemius wrote:
Quote:
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in news:1165937871.090284.265210
@f1g2000cwa.googlegroups.com:

Hmm, I thought I remember the term "scree" used in Kruskal's
MDS book also.

I learned MDS from Joe Kruskal himself, in 1967, shortly after his
publication of his 1963 papers on MDS.

He used the "Elbow Rule" then.

What BOOK on MDS did he write?

http://www.amazon.com/exec/obidos/ASIN/0803909403
http://www.amazon.co.uk/exec/obidos/ASIN/0803930682

David, you came to this thread quite a bit late. The use of "scree"
had
been thoroughly identified in 5 or 6 different areas.

Go back up the Google tree of this thread (44 articles) and review
what had been discussed and clarified already.

If the word "scree" was ever used in the 1978 book co-authored by
Kruskal, I am SURE it was NOT used as a substitute (or even related
to Kruskal's use) of the "Elbow Rule" he used in his MDSCAL and MDS
published and unpublished articles.

-- Reef Fish Bob.
David Winsemius
Posted: Sun Dec 17, 2006 5:21 pm
Guest
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in
news:1166373175.068176.129360@73g2000cwn.googlegroups.com:

Quote:

David Winsemius wrote:
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in
news:1165937871.090284.265210 @f1g2000cwa.googlegroups.com:

Hmm, I thought I remember the term "scree" used in Kruskal's
MDS book also.

I learned MDS from Joe Kruskal himself, in 1967, shortly after his
publication of his 1963 papers on MDS.

He used the "Elbow Rule" then.

What BOOK on MDS did he write?

http://www.amazon.com/exec/obidos/ASIN/0803909403
http://www.amazon.co.uk/exec/obidos/ASIN/0803930682

David, you came to this thread quite a bit late. The use of "scree"
had been thoroughly identified in 5 or 6 different areas.

Go back up the Google tree of this thread (44 articles) and review
what had been discussed and clarified already.

If the word "scree" was ever used in the 1978 book co-authored by
Kruskal, I am SURE it was NOT used as a substitute (or even related
to Kruskal's use) of the "Elbow Rule" he used in his MDSCAL and MDS
published and unpublished articles.


I wrote _absolutely_nothing_ regarding "elbows" or "scree". You were
responding to Greg Heath, remember? You asked what book on MDS Kruskal
had written. I answered. I made no statements about what might or might
not be in those books.

Searching on "Kruskal, book" in sci.stat.math, I see that one of the two
was referenced in a different thread.

<------>
We now return you to our regular failure to communicate.

--
David Winsemius
Reef Fish
Posted: Sun Dec 17, 2006 6:20 pm
Guest
David Winsemius wrote:
Quote:
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in
news:1166373175.068176.129360@73g2000cwn.googlegroups.com:


David Winsemius wrote:
"Reef Fish" <large_nassua_grouper@yahoo.com> wrote in
news:1165937871.090284.265210 @f1g2000cwa.googlegroups.com:

Hmm, I thought I remember the term "scree" used in Kruskal's
MDS book also.

I learned MDS from Joe Kruskal himself, in 1967, shortly after his
publication of his 1963 papers on MDS.

He used the "Elbow Rule" then.

What BOOK on MDS did he write?

http://www.amazon.com/exec/obidos/ASIN/0803909403
http://www.amazon.co.uk/exec/obidos/ASIN/0803930682

David, you came to this thread quite a bit late. The use of "scree"
had been thoroughly identified in 5 or 6 different areas.

Go back up the Google tree of this thread (44 articles) and review
what had been discussed and clarified already.

If the word "scree" was ever used in the 1978 book co-authored by
Kruskal, I am SURE it was NOT used as a substitute (or even related
to Kruskal's use) of the "Elbow Rule" he used in his MDSCAL and MDS
published and unpublished articles.


I wrote _absolutely_nothing_ regarding "elbows" or "scree". You were
responding to Greg Heath, remember? You asked what book on MDS Kruskal
had written. I answered. I made no statements about what might or might
not be in those books.

You were posting today, OUT of CONTEXT, and OUT of DATE. That's
all I was informing you, that if you want to jump into some thread that
has
been ongoing, it is YOUR responsibility to be up-to-date on it. Else,
just wait until you know where the status of the thread it.

Quote:

Searching on "Kruskal, book" in sci.stat.math, I see that one of the two
was referenced in a different thread.

------
We now return you to our regular failure to communicate.

The failure was all yours, and you are quite right about YOUR
regular failure too.

-- Reef Fish Bob.
Quote:

--
David Winsemius
 
Page 3 of 3    Goto page Previous  1, 2, 3   All times are GMT - 5 Hours
The time now is Sat Oct 11, 2008 2:13 am