Hello,

I have some more questions (related to my previous post
"Effect size with no control group").

According to: alpha, number n of subjects, and effect size,
it is possible to find the statistical power (1-beta),
by making a t-test with that alpha, n, and ES.

Tables with results are reported (for example) at:

http://www.science.mcmaster.ca/Psychology/psych2ra3/pwr1samp.pdf
(in the case of 1 Sample, or 2 Paired samples)

http://www.science.mcmaster.ca/Psychology/psych2ra3/pwr2samp.pdf
(in the case of 2 Indipendent samples)

Provided that ES = (Mean2 - Mean1) / SD
I understand why the table is different in the 2 cases:
if we have a single sample (to compare with a previous mean),
the t test is calculated with ES / SQR(1/n)
while if we have 2 independent samples to compare (n subjects each),
the t test is calculated with ES / SQR(2/n).
What is different is the SQR(2/n) instead of SQR(1/n) :
for independent samples, if m=n, SQR(1/m + 1/n) becomes SQR(2/n)

However, I am confused about the case of Paired data.
In this case ES should be calculated as usual,
ES = (Mean2 - Mean1) / SD

but the t-test must NOT be calculated with ES, but with the
paired differences, so that the SD is NOT the SD of the sample!
Usually SDdiff << SDsample, so the t-test comes out more
favorable (high statistical power, even with low alpha).

Hence, in the case of paired data, I consider wrong to
calculate power using the usual ES. The table would be fine
only if I use a "fake ES" calculated as (Mean2 - Mean1) / SDdiff
instead of the classic ES that is (Mean2 - Mean1) / SDsample
Question: am I wrong?

Another question, that is completely different, is this:
while studying several researches, I saw that, if there is an
attrition, this is not taken into account in the calculation
of ES! An example: pre-test with n=30 (experimental group)
and m=30 (control group). Post-test with n=25 and m=25.
I understand that ES is still calculated as
(MeanPost - MeanPre) / SDpooled
that does not take into account that 5 subjects have gone!
(So the ES is overestimated, in my understanding).
Question: is this a standard?

Probably the 5 subjects have gone because the treatment
is not effective, so I would consider the score of all the 30
subjects in the post-test, using, for the 5 that have gone,
their pre-test score, unchanged (I already said this in a
previous post of mine).
This would reduce the calculated ES to a more reliable value.

So, in my understanding, the ES's that I read of are
overestimated, because attrition has not been taken into
account.

Thanks for any explanation or comment.

Thanks for your answer.
There is still something that I am not sure about.
Since it is very specific, I am going to explain it
with a real example.

Consider these psychometric paired data (n=12) :

Subject Pre score Post score Difference

1 58 55 -3
2 65 58 -7
3 55 44 -11
4 60 52 -8
5 62 57 -5
6 49 45 -4
7 41 38 -3
8 53 48 -5
9 36 32 -4
10 49 51 +2
11 64 65 +1
12 52 47 -5

Mean 53.67 49.33 -4.33
St.Dev. 8.97 9.07 3.55
St.Dev/sqr(n) 2.59 2.62 1.02
t -1.66 -4.25

I calculate Effect Size
ES = (Mean2 - Mean1) / SD
(pooled SD is 9.02) so that
ES = (49.33-53.67)/9.02 = -4.34/9.02 =
ES = -0.481
(the minus is because the score has decreased,
but consider ES = 0.481)

I check if this result is significant (one-tailed t test)

Case A. I consider the 2 samples as paired
(as they really are):
t = -4.33/(9.02*sqr(1/12)) = -4.34/1.02 =
t = -4.25 (significant, p<0.001)

Of course, I consider this Case A as correct,
while the following Cases, B and C, are not correct:

Case B. I consider the 2 samples as independent:
t = (49.33-53.67)/(3.55*sqr(1/12+1/12)) = -4.34/3.69 =
t = -1.18 (non-significant)

Case C. I trivially compare the post-sample Mean2
with a fixed value that is equal to Mean1 = 53.67:
t = (49.33-53.67)/(9.02/sqr(12)) = -4.34/2.61 =
t = -1.66 (significant if alpha=.10, non-significant if alpha=.05)
- naturally this t is greater than (B), since it ignores the

Now, let's consider Statistical Power.
Remember that Effect Size is 0.481.

I calculate Statistical Power (1-beta), considering
alpha = 0.05, and I compare my result with the result
given by the famous Cohen's tables, given alpha, ES, n:
http://www.science.mcmaster.ca/Psychology/psych2ra3/pwr1samp.pdf
(in the case of 1 Sample, or 2 Paired samples)
http://www.science.mcmaster.ca/Psychology/psych2ra3/pwr2samp.pdf
(in the case of 2 Indipendent samples)

Valid results come out if I consider
the 2 "wrong" Cases, B or C:

Case B
I calculate Power (1-beta) = 0.28
On Cohen's tables for two independent samples,
given alpha=0.05 (one-tail), n=12, ES=0.50,
Power = 0.31
(it's OK, since in the tables I can find ES=0.50,
while in my case ES=0.481)

Case C
I calculate Power (1-beta) = 0.45
But, on Cohen's tables for one sample,
given alpha=0.05 (one-tail), n=12, ES=0.50,
Power = 0.48 (it's OK, compared with 0.45)

However, I do NOT get a correct result
if I use the correct Case A!

Case A
I calculate Power (1-beta) = 0.9989 (very good)
But, on Cohen's tables for paired data,
given alpha=0.05 (one-tail), n=12, ES=0.50
Power = 0.48 only!

Nevertheless, if I use a "fake ES" in Case A,
fakeES = (realES)*sqr(12) = 1.666

then I get the correct value of Power from
Cohen's tables for paired data.
But I don't think that my ES is 1.666.
I still think that ES is 0.481
(this is why I wrote "fake ES" in my previous post,
and I insist here in writing so).

This is a puzzle to me.

In other words: I can be confident that the treatment
works, form the analysis of the existing data (alpha=.05,
beta=.02), since they are paired.