I trying to think about it like this. Suppose there is no variability
in a given variable say X. i.e. it is constant. I guess we would say
that such a variable would have no usefulness in predicting the value
of a response variable. I mean if X is always 5 then knowing what X is
will give us no chance to have any insight into the value of our
response variable (say Y).

But the converse of this idea, that if X is extremely variable -has a
lot of variability, then ..... what? I cannot go to the next step
here.

Any help would be appreciated!
Thank you.

Hi, I have read about PCA. Always there is reference to the fact that
the first principle component is the component with the greatest
variation. Also there is mention of the fact that we seek to rotate
the axes in the direction of maximum variability. And again there is
mention of the fact that much of the variability of the data can be
accounted for by a smaller set of variables.

I am missing something in these statements. I cannot see clearly the
importance of the variability of a variable. I think that this is
being considered in the context of predicting the value of another (a
dependent/response) variable.

I trying to think about it like this. Suppose there is no variability
in a given variable say X. i.e. it is constant. I guess we would say
that such a variable would have no usefulness in predicting the value
of a response variable. I mean if X is always 5 then knowing what X is
will give us no chance to have any insight into the value of our
response variable (say Y).

But the converse of this idea, that if X is extremely variable -has a
lot of variability, then ..... what? I cannot go to the next step
here.

Any help would be appreciated!
Thank you.

Also, at the risk of bringing up more confusion, if your original
variables are in different units, and have greatly varying variances
for each of them, you will want to do your PCA from the correlation
matrix of your original X variables, as opposed to the covariance
matrix of your original X variables.

From your original set of many X variables,PCAlets you create a new
set of variables. This new set will have just as many variables as
your original set, but the point here is that, with only a few of
them, you could still capture almost all of the variance of the
original X variables.

As an exaggerated example, your outcome might be that the 1st new
variable might capture 98% of the variance of all of the original
variables. This means that this one new variable does a good job of
representing the entire set of original variables. That is why there
is so much focus on wanting your new variable to have a large
variance.

As another example, if instead your outcome was that the 1st new
variable only accounted for 4% of the variance of the original
variables, we could say it does not do a good job of representing the
set of original variables.

Now, for a new PC variable that does capture a large percentage of the
original variance, does this mean this variable is good at predicting
some other response variable? The usual answer here is no, the new
variables are only chosen to explain the variance of some set of
variables, and do not take into account how any of the varibles relate
to some response variable. InPCA, there are no response or predictor
variables.

However, I am aware of another opinion, which I like. If you are using
some set of predictor variables to model some response variable, then
right off the bat, the reason you are doing this in the first place is
because you think this set of predictor variables is reasonable to try
and predict your response variable. If you then find one or two new PC
variables that can represent your original X variables, then it is
hardly any more of a stretch to use them for prediction, than it
already was to use your original X variables.

This could be a moot point though, because other methods do exist that
will create new variables that do explicitly take into account the
covariance between a response variable and some other set of
variables, such as partial least squares. I think there have been some
good posts on this topic in this group already.

Also, at the risk of bringing up more confusion, if your original
variables are in different units, and have greatly varying variances
for each of them, you will want to do yourPCAfrom the correlation
matrix of your original X variables, as opposed to the covariance
matrix of your original X variables. This should be as simple as
checking off some option on whatever software you are using. The
reasoning is that PC's from a covariance matrix will be greatly skewed
towards the original variables with large variances.

-CP

I trying to think about it like this. Suppose there is novariability
in a given variable say X. i.e. it is constant. I guess we would say
that such a variable would have no usefulness in predicting the value
of a response variable. I mean if X is always 5 then knowing what X is
will give us no chance to have any insight into the value of our
response variable (say Y).

But the converse of this idea, that if X is extremely variable -has a
lot ofvariability, then ..... what?  I cannot go to the next step
here.

Any help would be appreciated!
Thank you.

I recall from regression that we talk about the explained versus
unexplained variation. This is, as I recall, what percent of the total
variation of the response variable is explained by the explanatory
variable(s). So this is in the context of predicting the outcome of
the response variable and the higher the explained variation the
better the model (less error in the prediction).
So I am assuming that in PCA, the reason we are interested in how much
of the overall variation is covered by a variable is something to do
with how well it will work for predicting some response variable
outcome. But this does not seem to be right because we are not talking
about how much of the variation in the explained variable (if there is
an explained variable) is explained by our variable or variables
(either the original variables or the derived components). Rather we
are just talking about how much of the overall variation in our entire
set of variables is accounted for by one variable or one component or
a set of variables or a set of components. But I don't understand why
we are interested in this overall variation in the first place?

Hi and thanks to all you have responded to my query. My question is, I
suppose, say in regard to the above post, why are we interested in
whether one of the original variables or one of the new derived
variables might capture 98% of the variance of all of the original
variables or not. I mean, I guess this is a very basic question, but
why are we interested in the variation of the variables in the first
place.

onyourmark <william... at (no spam) gmail.com> wrote:

Hi and thanks to all you have responded to my query. My question is, I
suppose, say in regard to the above post, why are we interested in
whether one of the original variables or one of the new derived
variables might capture 98% of the variance of all of the original
variables or not. I mean, I guess this is a very basic question, but
why are we interested in the variation of the variables in the first
place.

Maybe you aren't interested in that.  In which case, you probably shouldn't
do aPCAanalysis.  It's a tool for a job.  If you have no need for that
job, you have no need for that tool.

Xho

--
--------------------http://NewsReader.Com/--------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Hi and thanks to all again. I am interested in PCA. May I ask, (sorry
for being obtuse), when you say "that is where all the action occurs"
you are saying that all the variability is in those two dimensions and
that only 0.01 percent of all the variability lies in the z axis, but
aren't you concerned with predicting a fourth variable (a response
variable, say Y)? Otherwise I don't understand what you mean by that
is where the action occurs. I mean, I understand that most of the
variation occurs in those two dimensions but why is the variation
important?

Hi and thanks to all again. I am interested in PCA. May I ask, (sorry
for being obtuse), when you say "that is where all the action occurs"
you are saying that all the variability is in those two dimensions and
that only 0.01 percent of all the variability lies in the z axis, but
aren't you concerned with predicting a fourth variable (a response
variable, say Y)? Otherwise I don't understand what you mean by that
is where the action occurs. I mean, I understand that most of the
variation occurs in those two dimensions but why is the variation
important?
I can see that, for example, if data is constant with respect to a
certain variable, say X1, so that for every case/individual X1 has the
same value, say X1=5, across all observations, then X1 will be useless
in predicting Y (or as it is sometimes said "variation in Y") because
X1 is 5 no matter what value Y is (if you tell me that for this
individual/observation/case X1 is 5, that is not going to help me to
predict Y at all). And by extension if X1 is not constant but has
almost no variation then it will be almost useless in predicting the
variation in Y.
So is this why we are interested in the variation of the variables?
Because they are input variables? Or is there some other reason?
Thanks again.

On Jun 27, 3:47 am, xhos... at (no spam) gmail.com wrote:
onyourmark <william... at (no spam) gmail.com> wrote:

Hi and thanks to all you have responded to my query. My question is, I
suppose, say in regard to the above post, why are we interested in
whether one of the original variables or one of the new derived
variables might capture 98% of the variance of all of the original
variables or not. I mean, I guess this is a very basic question, but
why are we interested in the variation of the variables in the first
place.
Maybe you aren't interested in that. In which case, you probably shouldn't
do aPCAanalysis. It's a tool for a job. If you have no need for that
job, you have no need for that tool.

Xho

--
--------------------http://NewsReader.Com/--------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Hi and thanks to all again. I am interested in PCA. May I ask, (sorry
for being obtuse), when you say "that is where all the action occurs"
you are saying that all the variability is in those two dimensions and
that only 0.01 percent of all the variability lies in the z axis, but
aren't you concerned with predicting a fourth variable (a response
variable, say Y)? Otherwise I don't understand what you mean by that
is where the action occurs. I mean, I understand that most of the
variation occurs in those two dimensions but why is the variation
important?
I can see that, for example, if data is constant with respect to a
certain variable, say X1, so that for every case/individual X1 has the
same value, say X1=5, across all observations, then X1 will be useless
in predicting Y (or as it is sometimes said "variation in Y") because
X1 is 5 no matter what value Y is (if you tell me that for this
individual/observation/case X1 is 5, that is not going to help me to
predict Y at all). And by extension if X1 is not constant but has
almost no variation then it will be almost useless in predicting the
variation in Y.
So is this why we are interested in the variation of the variables?
Because they are input variables? Or is there some other reason?
Thanks again.