jgpow...@gmail.com wrote:
Hello,
I am a biologist attempting to publish in a Scientific Journal. I
used t-tests to analyze some of my data and one of the reviewers made
a comment about this being inappropriate. This same reviewer made
other comments that led me to question whether or not he really
understood what was going on, but I wanted to get some input on
whether or not a t-test would be appropriate in this situation. My
statistics background is limited, I did take a class about 4 years ago
in statistics, but since I didn't need the information right away, the
majority of it left my brain almost immediately.
Reviewer's exact quote:
"A T-test or Anova are really for pairwise comparisons and cannot be
used for comparisons of multiple samples such as this. "

My experimental set up:
Leaves of a plant were treated with 1
of 9 different compounds, or one compound that served as a control.
Each of these compounds produced a certain amount of Green fluorescent
light when ultraviolet light was shown onto the leaves. This level of
light could be quantitatively measured as photons of light emitted per
second per square centimeter. Each compound was placed onto 10
different leaves and measurements from each of these treatments were
taken. The 10 measurements for each compound was compared to the 10
control measurements to determine if the compound increased
fluorescent light emitted at a statistically significant level as
compared to the control. I thought that to do this a T-test or
possibly an Anova was an appropriate way to make this analysis. Again
I wanted to compare Compound X with Control, Compound Y with control,
Compound Z with control, and so on. I do not care how Compound X
compares with Compound Y.

Am I correct here?

In a word, no, especially if you set this up as a pairwise comparison.
Neither is the reviewer, but I'll give them the benefit of the doubt
because they're being quoted out of context. On the face of it, the
first half of the sentence (ANOVA really for pairwise comparisons) is
incorrect, though I can see what they are trying to get at. There is
some truth to the second half (naive use of ANOVA is not appropriate for
multiple comparisons).

If so how would you respond to the reviewer?

Maybe start by reading about multiple comparisons. (Perhaps someone
else more familiar with literature in your field could suggest a
reference. Otherwise, Google should be your friend.) It's hard to say
exactly how you should be doing your analysis, since some details of
your experimental setup are lacking. Were the control measurements on
the same 10 leaves in each case? Were the leaves all selected from the
same plant? More generally, what scheme was used to assign treatments
to leaves?

I wanted to compare Compound X with Control, Compound Y with control, Compound Z with control, and so on. I do not care how Compound X compares with Compound Y.

Hello,
I am a biologist attempting to publish in a Scientific Journal. I
used t-tests to analyze some of my data and one of the reviewers made
a comment about this being inappropriate. This same reviewer made
other comments that led me to question whether or not he really
understood what was going on, but I wanted to get some input on
whether or not a t-test would be appropriate in this situation. My
statistics background is limited, I did take a class about 4 years ago
in statistics, but since I didn't need the information right away, the
majority of it left my brain almost immediately.
Reviewer's exact quote:
"A T-test or Anova are really for pairwise comparisons and cannot be
used for comparisons of multiple samples such as this. "

My experimental set up:
Leaves of a plant were treated with 1
of 9 different compounds, or one compound that served as a control.
Each of these compounds produced a certain amount of Green fluorescent
light when ultraviolet light was shown onto the leaves. This level of
light could be quantitatively measured as photons of light emitted per
second per square centimeter. Each compound was placed onto 10
different leaves and measurements from each of these treatments were
taken. The 10 measurements for each compound was compared to the 10
control measurements to determine if the compound increased
fluorescent light emitted at a statistically significant level as
compared to the control. I thought that to do this a T-test or
possibly an Anova was an appropriate way to make this analysis. Again
I wanted to compare Compound X with Control, Compound Y with control,
Compound Z with control, and so on. I do not care how Compound X
compares with Compound Y.

Am I correct here?
In a word, no, especially if you set this up as a pairwise comparison.

If so how would you respond to the reviewer?
Maybe start by reading about multiple comparisons. (Perhaps someone

On Feb 21, 11:20 pm, Allen McIntosh <nos...@mouse-potato.com> wrote:





jgpow...@gmail.com wrote:
Hello,
I am a biologist attempting to publish in a Scientific Journal.  I
used t-tests to analyze some of my data and one of the reviewers made
a comment about this being inappropriate.  This same reviewer made
other comments that led me to question whether or not he really
understood what was going on, but I wanted to get some input on
whether or not a t-test would be appropriate in this situation.  My
statistics background is limited, I did take a class about 4 years ago
in statistics, but since I didn't need the information right away, the
majority of it left my brain almost immediately.
Reviewer's exact quote:
"A T-test or Anova are really for pairwise comparisons and cannot be
used for comparisons of multiple samples such as this. "

My experimental set up:
Leaves of a plant were treated with 1
of 9 different compounds, or one compound that served as a control.
Each of these compounds produced a certain amount of Green fluorescent
light when ultraviolet light was shown onto the leaves.  This level of
light could be quantitatively measured as photons of light emitted per
second per square centimeter.  Each compound was placed onto 10
different leaves and measurements from each of these treatments were
taken. The 10 measurements for each compound was compared to the 10
control measurements to determine if the compound increased
fluorescent light emitted at a statistically significant level as
compared to the control.  I thought that to do this a T-test or
possibly an Anova was an appropriate way to make this analysis.  Again
I wanted to compare Compound X with Control, Compound Y with control,
Compound Z with control, and so on.  I do not care how Compound X
compares with Compound Y.

Am I correct here?

In a word, no, especially if you set this up as a pairwise comparison.
Neither is the reviewer, but I'll give them the benefit of the doubt
because they're being quoted out of context.  On the face of it, the
first half of the sentence (ANOVA really for pairwise comparisons) is
incorrect, though I can see what they are trying to get at.  There is
some truth to the second half (naive use of ANOVA is not appropriate for
multiple comparisons).

If so how would you respond to the reviewer?

Maybe start by reading about multiple comparisons.  (Perhaps someone
else more familiar with literature in your field could suggest a
reference.  Otherwise, Google should be your friend.)  It's hard to say
exactly how you should be doing your analysis, since some details of
your experimental setup are lacking.  Were the control measurements on
the same 10 leaves in each case?  Were the leaves all selected from the
same plant?  More generally, what scheme was used to assign treatments
to leaves?

Ouch...not what I was hoping for...regarding the experimental setup.
10 measurements in total were made.  The details for each measurement
are as follows:  Each treatment was done on two leaves of one plant.
So in total 10 leaves from 5 different plants were analyzed.  No
leaves were doubly treated, so each leaf was a "virgin."  While
control leaves and treated leaves were not derived from the same
plant, they were derived from the same batch of plants, i.e. planted
at the same time with the same seeds, watered at the same time,
fertilized at the same time, etc. etc.  As far as "what scheme was
used to assign treatments to leaves?"  I don't know exactly what you
mean, but essentially I just choose similar looking plants to do all
the treatments with, there was no preferential choices going on in
terms of "oh, this plant is healthy looking, lets treat it with
compound y."  Thanks for your help- Hide quoted text -

- Show quoted text -

z wrote:
what they're getting act, i think, is that if a t-test is designed to
be significant or not at the .95 level, that means that 1 time out of
20 it will show a significant difference when there is really no
difference between the treatments, out of random "noise". this is the
generally accepted level of error. however, what you are doing is
basically 10 t-tests simultaneously, so your chance of finding a
significant difference somewhere when in fact there are none at all is
1-(.95)^10, or about .4, if you treat each t-test as though it were
alone and require a .95 confidence limit. this kind of error rate is
obviously a bit high.

The 10 tests aren't independent, so you can't calculate the chance of
finding a significant difference that way.

On Feb 22, 11:14 am, Allen McIntosh <nos...@mouse-potato.com> wrote:

z wrote:
what they're getting act, i think, is that if a t-test is designed to
be significant or not at the .95 level, that means that 1 time out of
20 it will show a significant difference when there is really no
difference between the treatments, out of random "noise". this is the
generally accepted level of error. however, what you are doing is
basically 10 t-tests simultaneously, so your chance of finding a
significant difference somewhere when in fact there are none at all is
1-(.95)^10, or about .4, if you treat each t-test as though it were
alone and require a .95 confidence limit. this kind of error rate is
obviously a bit high.

The 10 tests aren't independent, so you can't calculate the chance of
finding a significant difference that way.

Okay I have done a bit of reading. Based on what I have read I still
feel like T-tests would be alright. Perhaps it would be more powerful
to run an ANOVA followed by a Dunnett's but I think for my purposes a
t-test would be okay. Here are a couple websites that I think support
my thoughts:

http://www.anselm.edu/homepage/jpitocch/biostats/keysmeans.html
Based on this webpage I think I need a 2 independent sample t-test 1 -
direction

http://www.aiaccess.net/e_t.htm
"3) The third form of the t-test, called the "Two Independent Samples
t-test", looks very similar to the previous one. We still have two
sets of measurements, and we are trying to figure out if the averages
of these two sets of measurements are significantly different. But
this time, we assume that there is no relationship whatsoever between
the two sets of measurements, because they were conducted on two, non
intersecting sets of individuals."

I think where everyone gets tripped up is the fact that I have like 9
different treatments all being compared to a control treatment. But
those 9 treatments are all completely different and essentially
unrelated. For all intents and purposes I am really only looking at
two sets of data at any given time, one set of treated leaves vs. the
set of control treatment leaves.

I know you guys are all statistics people so you will probably hate me
when I say that based on my reading an ANOVA/Dunnett would be
statistically more powerful, but being so late in the game with this
paper I am hesitant to change my statistical analysis now.

what they're getting act, i think, is that if a t-test is designed to
be significant or not at the .95 level, that means that 1 time out of
20 it will show a significant difference when there is really no
difference between the treatments, out of random "noise". this is the
generally accepted level of error. however, what you are doing is
basically 10 t-tests simultaneously, so your chance of finding a
significant difference somewhere when in fact there are none at all is
1-(.95)^10, or about .4, if you treat each t-test as though it were
alone and require a .95 confidence limit. this kind of error rate is
obviously a bit high.

Leaves of a plant were treated with 1
of 9 different compounds, or one compound that served as a control.

... Each compound was placed onto 10
different leaves and measurements from each of these treatments were
taken.
regarding the experimental setup.
10 measurements in total were made. The details for each measurement
are as follows: Each treatment was done on two leaves of one plant.
So in total 10 leaves from 5 different plants were analyzed. No
leaves were doubly treated, so each leaf was a "virgin." While
control leaves and treated leaves were not derived from the same
plant, they were derived from the same batch of plants, i.e. planted
at the same time with the same seeds, watered at the same time,
fertilized at the same time, etc. etc.
So was each treatment done 10 times or twice? This isn't completely

I just chose similar looking plants to do all
the treatments with, there was no preferential choices going on in
terms of "oh, this plant is healthy looking, lets treat it with
compound y."

On Feb 22, 11:51 am, jgpow...@gmail.com wrote:
Okay I have done a bit of reading. Based on what I have read I still
feel like T-tests would be alright. Perhaps it would be more powerful
to run an ANOVA followed by a Dunnett's but I think for my purposes a
t-test would be okay.
It's not a question of power (in the statistical sense). It's a

Here are a couple websites that I think support
my thoughts:

http://www.anselm.edu/homepage/jpitocch/biostats/keysmeans.html
Based on this webpage I think I need a 2 independent sample t-test 1 -
direction
Are we looking at the same web page?

http://www.aiaccess.net/e_t.htm
"3) The third form of the t-test, called the "Two Independent Samples
t-test"
This web page covers material that would be taught in a first course to

I think where everyone gets tripped up is the fact that I have like 9
different treatments all being compared to a control treatment. But
those 9 treatments are all completely different and essentially
unrelated.
THey're not. You are analyzing them in the same paper.
For all intents and purposes I am really only looking at
two sets of data at any given time, one set of treated leaves vs. the
set of control treatment leaves.

I know you guys are all statistics people so you will probably hate me
when I say that based on my reading an ANOVA/Dunnett would be
statistically more powerful, but being so late in the game with this
paper I am hesitant to change my statistical analysis now.
If you are going to do that, you should be careful to state that your

Regarding plant selection and randomizing these things I do realize
that there is a level of inherent uncertainty when doing these sorts
of statistical comparisons, and I also realize that it is not ideal to
ignore plant variations, but I think for my purposes it should be
okay. These plants are not like humans, they are all clones of each
other. All grown from the same batch of seeds. All planted in the
same soil, watered at the same time, exposed to the same amount of
light, etc. etc. I realize there will always some differences, even
in clones, but these should be minor.

ignore plant variations, but I think for my purposes it should be
okay.  These plants are not like humans, they are all clones of each
other.  All grown from the same batch of seeds.  All planted in the
same soil, watered at the same time, exposed to the same amount of
light, etc. etc.  I realize there will always some differences, even
in clones, but these should be minor.