| |
 |
|
|
Science Forum Index » Statistics - Math Forum » need a robust estimator for the following situation
Page 1 of 1
|
| Author |
Message |
| bahoo |
 Posted: Thu Jan 25, 2007 11:06 am |
|
|
|
Guest
|
Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are flat
plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1" is an
anomaly.
What kind of robust estimator sould I use to return the location and
scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean and
variance is not a good idea, since it is influenced by the anomaly a
lot. Besides, I know the bump is actually not really a Gaussian.
Thanks |
|
|
| Back to top |
|
| David Jones |
| Posted: Thu Jan 25, 2007 12:33 pm |
|
|
|
Guest
|
bahoo wrote:
Quote: Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are flat
plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1" is an
anomaly.
What kind of robust estimator sould I use to return the location and
scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean
and
variance is not a good idea, since it is influenced by the anomaly a
lot. Besides, I know the bump is actually not really a Gaussian.
Thanks
You could try doing something based on Tukey's ideas of "exploratory
data analysis". In particular, you might try the running median of 3's
idea to remove isolated outliers.
David Jones |
|
|
| Back to top |
|
| Scott Seidman |
| Posted: Thu Jan 25, 2007 12:43 pm |
|
|
|
Guest
|
"bahoo" <b83503104@yahoo.com> wrote in news:1169737604.076324.280120
@k78g2000cwa.googlegroups.com:
Quote: Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are flat
plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1" is an
anomaly.
What kind of robust estimator sould I use to return the location and
scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean and
variance is not a good idea, since it is influenced by the anomaly a
lot. Besides, I know the bump is actually not really a Gaussian.
Thanks
Maybe a running median filter??
--
Scott
Reverse name to reply |
|
|
| Back to top |
|
| da |
| Posted: Thu Jan 25, 2007 3:15 pm |
|
|
|
Guest
|
On Jan 25, 11:43 am, Scott Seidman <namdiestt...@mindspring.com> wrote:
Quote: "bahoo" <b83503...@yahoo.com> wrote in news:1169737604.076324.280120
@k78g2000cwa.googlegroups.com:
Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are flat
plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1" is an
anomaly.
What kind of robust estimator sould I use to return the location and
scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean and
variance is not a good idea, since it is influenced by the anomaly a
lot. Besides, I know the bump is actually not really a Gaussian.
ThanksMaybe a running median filter??
--
Scott
Reverse name to reply
Thanks for the suggestions.
But the problem with the running median filter is, I don't have prior
knowledge of the "width" of the high bump. In other words, I need a
robust estimator for both its location and its scale. |
|
|
| Back to top |
|
| David Winsemius |
| Posted: Thu Jan 25, 2007 6:44 pm |
|
|
|
Guest
|
"da" <dawenliu@gmail.com> wrote in
news:1169752551.681085.160910@m58g2000cwm.googlegroups.com:
Quote: On Jan 25, 11:43 am, Scott Seidman <namdiestt...@mindspring.com
wrote:
"bahoo" <b83503...@yahoo.com> wrote in news:1169737604.076324.280120
@k78g2000cwa.googlegroups.com:
Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are
flat plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1"
is an anomaly.
What kind of robust estimator sould I use to return the location
and scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean
and variance is not a good idea, since it is influenced by the
anomaly a lot. Besides, I know the bump is actually not really a
Gaussian.
Maybe a running median filter??
But the problem with the running median filter is, I don't have prior
knowledge of the "width" of the high bump. In other words, I need a
robust estimator for both its location and its scale.
The usual non-parametric candidates for location and scale would be the
median and the inter-quartile range. Assuming you presented this
distribution as a density, then:
1
2
1
5 <-25%ile
6
4 <-median
6
1
2 <-75%ile
1
6
1
--
David Winsemius |
|
|
| Back to top |
|
| Ray Koopman |
| Posted: Thu Jan 25, 2007 8:37 pm |
|
|
|
Guest
|
On Jan 25, 7:06 am, "bahoo" <b83503...@yahoo.com> wrote:
Quote: Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are flat
plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1" is an
anomaly.
What kind of robust estimator sould I use to return the location and
scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean and
variance is not a good idea, since it is influenced by the anomaly a
lot. Besides, I know the bump is actually not really a Gaussian.
Thanks
The absolute residuals after Tukey's "4253H,twice" smooth are
{ 0, .15, 2.13, .66, 1.22, .71, 2.05, 1.66, .17, .39, 4.83, .17 }.
Point 11 clearly an outlier in that distribution. |
|
|
| Back to top |
|
| Scott Seidman |
| Posted: Fri Jan 26, 2007 9:26 am |
|
|
|
Guest
|
"da" <dawenliu@gmail.com> wrote in news:1169752551.681085.160910
@m58g2000cwm.googlegroups.com:
Quote: On Jan 25, 11:43 am, Scott Seidman <namdiestt...@mindspring.com> wrote:
"bahoo" <b83503...@yahoo.com> wrote in news:1169737604.076324.280120
@k78g2000cwa.googlegroups.com:
Hi,
my 1 dimensional data (simplified) looks like this:
1 2 1 5 6 4 6 1 2 1 6 1
There is a high bump ("5 6 4 6"), and on its left and right are flat
plateu ("1 2 1" and "1 2 1 6 1"). The "6" in the "1 2 1 6 1" is an
anomaly.
What kind of robust estimator sould I use to return the location and
scale of the bump?
Obviously, assuming a Gaussian distribution and computing the mean
and
variance is not a good idea, since it is influenced by the anomaly a
lot. Besides, I know the bump is actually not really a Gaussian.
ThanksMaybe a running median filter??
--
Scott
Reverse name to reply
Thanks for the suggestions.
But the problem with the running median filter is, I don't have prior
knowledge of the "width" of the high bump. In other words, I need a
robust estimator for both its location and its scale.
In this case, perhaps wavelets would be of some use. The problem with
wavelets, in my experience, is knowing which decimation is going to be
useful to you, but they are all there and available.
--
Scott
Reverse name to reply |
|
|
| Back to top |
|
| |
|
Page 1 of 1
All times are GMT - 5 Hours
The time now is Fri Jul 25, 2008 2:49 am
|
|
