Hello, maybe some of you experts can help me out. It is a very simple question and to my surprise my research so far could not answer it conclusively.

I have a finite dataset of around 1300 observations. I am sure that this data cannot be considered as normally distributed.

If I take a moving average of n=30 for this data set (from observation 1 to 30, 2 to 31, 3 to 33 and so forth) can I consider the distribution of the averages as approximately normal? My understanding of the central limit theorem says yes, but I would like the confirmation of someone knowledgeable since I am not a statician.

Thanks for your replies

Hello, maybe some of you experts can help me out. It is a very simple question and to my surprise my research so far could not answer it conclusively.

I have a finite dataset of around 1300 observations. I am sure that this data cannot be considered as normally distributed.

If I take a moving average of n=30 for this data set (from observation 1 to 30, 2 to 31, 3 to 33 and so forth) can I consider the distribution of the averages as approximately normal? My understanding of the central limit theorem says yes, but I would like the confirmation of someone knowledgeable since I am not a statician.

Thanks for your replies

Hello, maybe some of you experts can help me out. It is a very simple question and to my surprise my research so far could not answer it conclusively.

I have a finite dataset of around 1300 observations. I am sure that this data cannot be considered as normally distributed.

If I take a moving average of n=30 for this data set (from observation 1 to 30, 2 to 31, 3 to 33 and so forth) can I consider the distribution of the averages as approximately normal? My understanding of the central limit theorem says yes, but I would like the confirmation of someone knowledgeable since I am not a statician.

Thanks for your replies

On May 6, 11:19 am, Gilberto Reis Filho
bon... at (no spam) gmail.com> wrote:
Hello, maybe some of you experts can help me out.
It is a very simple question and to my surprise my
research so far could not answer it conclusively.

I have a finite dataset of around 1300
observations. I am sure that this data cannot be
considered as normally distributed.

If I take a moving average of n=30 for this data
set (from observation 1 to 30, 2 to 31, 3 to 33 and
so forth) can I consider the distribution of the
averages as approximately normal? My understanding of
the central limit theorem says yes, but I would like
the confirmation of someone knowledgeable since I am
not a statician.

Thanks for your replies


Another thing, why do you care about getting
something that is normally distributed? In my
experience running means are usually used to
filter out high frequency "noise" (realizing
that one man's noise may be another's signal).

Cheers,
Russell

On May 6, 11:19 am, Gilberto Reis Filho
bon... at (no spam) gmail.com> wrote:
Hello, maybe some of you experts can help me out.
It is a very simple question and to my surprise my
research so far could not answer it conclusively.

I have a finite dataset of around 1300
observations. I am sure that this data cannot be
considered as normally distributed.

If I take a moving average of n=30 for this data
set (from observation 1 to 30, 2 to 31, 3 to 33 and
so forth) can I consider the distribution of the
averages as approximately normal? My understanding of
the central limit theorem says yes, but I would like
the confirmation of someone knowledgeable since I am
not a statician.

Thanks for your replies

Another thing, why do you care about getting
something that is normally distributed?  In my
experience running means are usually used to
filter out high frequency "noise" (realizing
that one man's noise may be another's signal).

Cheers,
Russell

Thank you all for the replies so far. I'll try to elaborate a little more and answer all your questions.

The dataset refers to blood glucose level observations of mine. They definively are related to a time series and might not be really independant. On average there are around 3 observations per day but that varies, sometimes more sometimes less.

I want to get something close to a normal distribution to calculate probabilities, let's say answer what is the probability of p<90 or p>180 with some confidence etc. etc.

Another approach I intend to take after sorting this out is computing the means for 1 week, 2 weeks, 3 weeks and 4 weeks and draw conclusions from them, like: 'there is a x% probability that the weekly mean in below 180'. The reason for this is that it can be better understand by other people.

This 'week based analysis' is proving very difficult to me because in this case n varies: there are weeks when I have only 6 observations and weeks when I have 26 observations. I believe that in this case I cannot use the CLT and I am a bit clueless as how to proceed.

The dataset and some analysis I made are available here if anyone is interested (refer to sheets 'Glicemia' and 'Médias'):

http://spreadsheets.google.com/pub?key=paFaAniurNBdgSK9AYzgAMw

Thanks again for all the help.

cheers.- Hide quoted text -

- Show quoted text -

OK, one approach might be to fit a distribution that
one might expect such data to follow to the data and
calculate the probabilities from that. Another
approach could be to transform the data to an
approximately normal distribution. Google Box-Cox
transform for one approach to doing the latter.
Also sometimes just taking the square root or log of
the data will make it approximately normal. Some
people (who I must admit are more highly trained
than I) argue against transforming the data, but it
is done (rightly or wrongly) in practice fairly
frequently.

Cheers,
Russell

OK, one approach might be to fit a distribution
that
one might expect such data to follow to the data
and
calculate the probabilities from that. Another
approach could be to transform the data to an
approximately normal distribution. Google Box-Cox
transform for one approach to doing the latter.
Also sometimes just taking the square root or log
of
the data will make it approximately normal. Some
people (who I must admit are more highly trained
than I) argue against transforming the data, but
it
is done (rightly or wrongly) in practice fairly
frequently.

Cheers,
Russell

Let me see if i follow you.
Fitting the data might not be so easy because I do
not know if it follows any known distribution.
Besides I am not sure I have the knowledge to do
that...

Transforming may be an option. It seems easier. You
mean take the original data (that we know is not
normally distributed) and calculate the log or square
root, what should give me something approximately
normal, right? Then I assume I just need to reverse
the transformation to get readable data again. Sounds
interesting. Is there any way to compute what would
be the expected error in the estimate in this case?

thanks.

OK, one approach might be to fit a distribution that
one might expect such data to follow to the data and
calculate the probabilities from that.  Another
approach could be to transform the data to an
approximately normal distribution.  Google Box-Cox
transform for one approach to doing the latter.
Also sometimes just taking the square root or log of
the data will make it approximately normal.  Some
people (who I must admit are more highly trained
than I) argue against transforming the data, but it
is done (rightly or wrongly) in practice fairly
frequently.

Cheers,
Russell

Let me see if i follow you.
Fitting the data might not be so easy because I do not know if it follows any known distribution. Besides I am not sure I have the knowledge to do that...

Transforming may be an option. It seems easier. You mean take the original data (that we know is not normally distributed) and calculate the log or square root, what should give me something approximately normal, right? Then  I assume I just need to reverse the transformation to get readable data again. Sounds interesting. Is there any way to compute what would be the expected error in the estimate in this case?

thanks.

Hello, maybe some of you experts can help me out. It is a very simple question and to my surprise my research so far could not answer it conclusively.

I have a finite dataset of around 1300 observations. I am sure that this data cannot be considered as normally distributed.

If I take a moving average of n=30 for this data set (from observation 1 to 30, 2 to 31, 3 to 33 and so forth) can I consider the distribution of the averages as approximately normal? My understanding of the central limit theorem says yes, but I would like the confirmation of someone knowledgeable since I am not a statician.

Thanks for your replies

On May 6, 11:19 am, Gilberto Reis Filho
bon... at (no spam) gmail.com> wrote:
Hello, maybe some of you experts can help me out.
It is a very simple question and to my surprise my
research so far could not answer it conclusively.
I have a finite dataset of around 1300
observations. I am sure that this data cannot be
considered as normally distributed.
If I take a moving average of n=30 for this data
set (from observation 1 to 30, 2 to 31, 3 to 33 and
so forth) can I consider the distribution of the
averages as approximately normal? My understanding of
the central limit theorem says yes, but I would like
the confirmation of someone knowledgeable since I am
not a statician.
Thanks for your replies

Another thing, why do you care about getting
something that is normally distributed? In my
experience running means are usually used to
filter out high frequency "noise" (realizing
that one man's noise may be another's signal).

Cheers,
Russell

Thank you all for the replies so far. I'll try to elaborate a little more and answer all your questions.

The dataset refers to blood glucose level observations of mine. They definively are related to a time series and might not be really independant. On average there are around 3 observations per day but that varies, sometimes more sometimes less.

I want to get something close to a normal distribution to calculate probabilities, let's say answer what is the probability of p<90 or p>180 with some confidence etc. etc.

Another approach I intend to take after sorting this out is computing the means for 1 week, 2 weeks, 3 weeks and 4 weeks and draw conclusions from them, like: 'there is a x% probability that the weekly mean in below 180'. The reason for this is that it can be better understand by other people.

This 'week based analysis' is proving very difficult to me because in this case n varies: there are weeks when I have only 6 observations and weeks when I have 26 observations. I believe that in this case I cannot use the CLT and I am a bit clueless as how to proceed.

The dataset and some analysis I made are available here if anyone is interested (refer to sheets 'Glicemia' and 'Médias'):

http://spreadsheets.google.com/pub?key=paFaAniurNBdgSK9AYzgAMw

Thanks again for all the help.

cheers.