 |
|
| Science Forum Index » Space - Consult Forum » ANOVA and measurement repeatability study... |
|
Page 2 of 2 Goto page Previous 1, 2 |
|
| Author |
Message |
| ... |
 Posted: Thu Oct 29, 2009 4:12 pm |
|
|
|
Guest
|
On Oct 30, 2:52 am, Rich Ulrich <rich.ulr... at (no spam) comcast.net> wrote:
[quote]What I "guessed" about the design on Oct 25 was that you
might have 3 factors: rater, location of measurement, and
order of measuring.
[/quote]
I'm not sure how you got this idea.
Please see my response to the previous post...... |
|
|
| Back to top |
|
|
|
| Ray Koopman... |
| Posted: Thu Oct 29, 2009 5:46 pm |
|
|
|
Guest
|
On Oct 29, 7:11 pm, voice_of_rea... at (no spam) australia.edu wrote:
[quote]On Oct 30, 2:25 am, Ray Koopman <koop... at (no spam) sfu.ca> wrote:
If there is only one part, but the program nevertheless gave you a
p-value for the significance of the difference between parts, then
the program did not do the analysis that you think you told it to do.
There is one part that is repeatedly mounted and unmounted from its
measuring fixture. Each mounting is entered into the program as a
"new" part.
Since it is in fact the SAME part, in theory the "between parts"
variation should be insignificant (I can verify the assumption that
mounting and unmounting does not effect the dimensions of the part).
If however there is significant difference, this implies that there
is something going on in the measuring system that is causing this
same part to appear to be different...to be producing significantly
different dimensions.
In other words, repeatedly using this fixture does NOT produce
repeatable results. The system is not repeatable.
[/quote]
All right, now it makes sense -- it was just a labelling problem.
Assuming that the proper error term was used (which depends on the
fixed/random status of the other factors in the design, which have
not been specified), a significant main effect for "Part" (which might
more appropriately be called something like "Trial"), means that the
differences between the means of the 5 trials are bigger than would
usually be expected if there were no differences among the trials.
So either an unusual but accidental event has occurred, or there truly
are trial-to-trial differences among the measurements. However, the
p-value itself establishes only the unusualness of the event and says
nothing about whether the differences are big enough to be concerned
about. |
|
|
| Back to top |
|
|
|
| Rich Ulrich... |
| Posted: Fri Oct 30, 2009 2:01 pm |
|
|
|
Guest
|
On Thu, 29 Oct 2009 19:12:38 -0700 (PDT),
voice_of_reason at (no spam) australia.edu wrote:
[quote]On Oct 30, 2:52 am, Rich Ulrich <rich.ulr... at (no spam) comcast.net> wrote:
What I "guessed" about the design on Oct 25 was that you
might have 3 factors: rater, location of measurement, and
order of measuring.
I'm not sure how you got this idea.
[/quote]
I got it directly from what you posted in the first post.
VOR>
[quote]The experiment involves taking a part from the assembly line, placing
it in its fixture, having two inspectors take measurements a pre-
selected points -- each point measured twice -- in random order.
[/quote]
I point to "two inspectors", I point to "pre-selected points", and
I point to "in random order".
That makes up 3 factors. The conventional analysis would
be a Repeated Measures ANOVA, where "parts" is a term
that is not measured by the ANOVA ... if that denotes a
the identity of the different 'parts' that are each replaced
and tested several times.
- Ray seems to accept that you are taking the "order" effect
and mis-labeling it as Parts. I suspect that he is being too
unsuspicious. I think you are doing something else.
The Repeated Measures with 3 factors can also be performed
as a 4-way ANOVA, one that would obtain an addition Sum of
Squares for Parts, which would *properly* be labeled Parts.
The 4-way is usually avoided because the extra SS terms are
not so generally interesting.
However, having *this* definition of Parts as "significant" is a
good thing for reliability. It says that measures *do* discriminate
between parts. That is the F-test that could be translated or
transformed into a "significant" ICC (intraclass correlation) --
though you usually want to know more than whether a
correlation is significant.
Compare the circumstance to one where you give 6 IQ tests
to a bunch of people -- if there is variation among the people,
then you *should* see a significant difference among "persons"
as a test. If there is no difference, *that* is what says
that you have unreliable tests, or else, not much measureable
difference between persons. And that consequence
would be true whether or not one test is systematically
10 points higher or lower than the others. The variability
of the measures is what matters for "how good is the testing,"
so long as you eventually do pay attention to which mean
is being used.
--
Rich Ulrich |
|
|
| Back to top |
|
|
|
| ... |
| Posted: Sat Oct 31, 2009 9:55 pm |
|
|
|
Guest
|
On Oct 30, 11:46 am, Ray Koopman <koop... at (no spam) sfu.ca> wrote:
[quote]
All right, now it makes sense --
[/quote]
GREAT!!
[quote]...... a significant main effect for "Part" (which might
more appropriately be called something like "Trial"), means that the
differences between the means of the 5 trials are bigger than....
[/quote]
Yes!
[quote]...... the p-value itself establishes only the unusualness of the event and says
nothing about whether the differences are big enough to be concerned
about.
[/quote]
Agreed....but at this point "unusualness" is enough..... |
|
|
| Back to top |
|
|
|
| ... |
| Posted: Sat Oct 31, 2009 9:57 pm |
|
|
|
Guest
|
On Oct 31, 4:01 am, Rich Ulrich <rich.ulr... at (no spam) comcast.net> wrote:
[quote]On Thu, 29 Oct 2009 19:12:38 -0700 (PDT),
voice_of_rea... at (no spam) australia.edu wrote:
On Oct 30, 2:52 am, Rich Ulrich <rich.ulr... at (no spam) comcast.net> wrote:
What I "guessed" about the design on Oct 25 was that you
might have 3 factors: rater, location of measurement, and
order of measuring.
I'm not sure how you got this idea.
I got it directly from what you posted in the first post.
VOR
The experiment involves taking a part from the assembly line, placing
it in its fixture, having two inspectors take measurements a pre-
selected points -- each point measured twice -- in random order.
I point to "two inspectors", I point to "pre-selected points", and
I point to "in random order".
[/quote]
I didn't say anything about "different locations"
[quote]
- Ray seems to accept that you are taking the "order" effect
and mis-labeling it as Parts.
[/quote]
Ray's understanding -- per the above post -- seems to be correct. I
am more confident now that my initial approach is viable. |
|
|
| Back to top |
|
|
|
| Ray Koopman... |
| Posted: Sun Nov 01, 2009 9:04 am |
|
|
|
Guest
|
On Oct 31, 11:57 pm, voice_of_rea... at (no spam) australia.edu wrote:
[quote][...]
Ray's understanding -- per the above post -- seems to be correct.
I am more confident now that my initial approach is viable.
[/quote]
That confidence may be misguided. I do not understand what was done.
My interpretation of the p-value was conditional on the analysis
having been done correctly. It is still not clear what the
experimental design was or how the analysis was done.
After the clarification that there was only one part, the original
post can be interpreted as saying that the data were collected in a
2 (inspectors) x n (points) x 2 (replications) x 5 (trials) factorial
design. On each trial, each inspector measured each point twice.
Some unspecified order was random. Was it which inspector went first?
Was it the order in which the points were measured? Did all this
change from trial to trial, or were the orders the same on every
trial?
And, again, what was the fixed/random status of each factor in the
analysis? |
|
|
| Back to top |
|
|
|
|
|
All times are GMT - 5 Hours
The time now is Sat Nov 28, 2009 8:04 pm
|
|