I need to estimate reliability of a measurement across time.
Normally, a regular correlation or an ICC would do, but in my case I
have:
a. more than two measurements per person and
b. unequal number of measurements per person (1-

.
So, the question is how comparable are ICCs from the fixed design
scenario (assuming two measurements per person and no missing data)
and ICC from a random intercept mixed model (intercept variance /
[total variance or intercept variance + residual unexplained
variance]).
There are two interpretations of the ICC in the case of a mixed
model:
1. Proportion of variance between subjects
2. Correlation between two randomly drawn observations from the
same subject.
Both of these sound acceptable as definitions of measurement
reliability, but I get very different results if I compute ICC from a
simple two-time dataset in fixed vs. random intercept models. Also,
ICC from a mixed model cannot be negative, while ICC from the fixed
model can be negative and I've also seen them greater than and less
than 1.
I don't know much about fixed model ICC as an estimate of
reliability. Any comments? Recommended reading about ICC in the
fixed design world?
Thanks.