| |
 |
|
|
Science Forum Index » Statistics - Math Forum » Howto Initialize Parameter of Gamma Mixture Densities...
Page 1 of 1
|
| Author |
Message |
| gundalav... |
 Posted: Sun Jul 06, 2008 11:36 pm |
|
|
|
Guest
|
Hi all,
Is there any reasonable way to initialize the gamma parameters for
maximum
likelihood estimation using EM.
Any advice, pointer to paper URLs are greatly appreciated.
Some people mention this can be done
by "drawing mixing proportions from Dirichlet distribution".
But that is vague for me on how it is done and why.
Please advice.
-
Gundala Viswanath
Jakarta-Indonesia |
|
|
| Back to top |
|
| ... |
| Posted: Mon Jul 07, 2008 5:13 pm |
|
|
|
Guest
|
On Jul 7, 4:36 am, gundalav <gunda... at (no spam) gmail.com> wrote:
Quote: Hi all,
Is there any reasonable way to initialize the gamma parameters for
maximum
likelihood estimation using EM.
Any advice, pointer to paper URLs are greatly appreciated.
Some people mention this can be done
by "drawing mixing proportions from Dirichlet distribution".
But that is vague for me on how it is done and why.
Please advice.
Drawing from a Dirichlet would seem to be related to drawing some
initial values for the mixture weights of your model. You also should
consider ways of initializing the shape and scale parameters of each
gamma component. Some ideas of the top of my head assuming your
mixture model is written as
K
p( x ) = SUM w_k Gamma( alpha_k, beta_k ) where SUM w_k = 1
k=1
For the mixture weights
1. Sample from a Dirichlet distribution with different alpha values
that represent your belief about the relative strength of the
components. Set all the alphas equal if you believe each
component
to have approximately equal weight. Make alpha very large to
reduce the variance, i.e. Dirichlet([1000,1000,1000]) will
produce
a vector very close to [1/3 1/3 1/3].
2. Set all the mixture weights equal, i.e. for K components w_k = 1/
K
For the gamma parameters
1. Pick random alpha and beta values from some range (probably bad)
2. Fit a single Gamma distribution to the data and jitter the
parameters
3. Fit a Gamma distribution to a random subset of the data. Repeat
K times.\
4. Run a kernel density smoother. Pick the K highest modes. Set
the mode of
the kth Gamma distribution equal to the kth empirical mode. The
variance
is a free parameter.
5. Place a prior over the Gamma parameters and sample K times (like
(1) but
a bit more formal)
Since you're doing EM I assume you've introduced unknown assignment
variables z_ik that indicate if observation i was generated by
component k and that the E-step computes E[z_ik] for all i and k. If
this is the case, you could also randomly initialize E[z_ik] s.t.
SUM_k E[z_ik] = 1 and then begin the EM iterations with the M-step.
-Lucas |
|
|
| Back to top |
|
| Herman Rubin... |
| Posted: Thu Jul 10, 2008 2:04 pm |
|
|
|
Guest
|
In article <3b2a2b21-f4b8-466a-ac4d-61f58059c628 at (no spam) d1g2000hsg.googlegroups.com>,
gundalav <gundalav at (no spam) gmail.com> wrote:
Quote: Hi all,
Is there any reasonable way to initialize the gamma parameters for
maximum
likelihood estimation using EM.
I presume you want a local maximum. The likelihood
function is unbounded.
Quote: Any advice, pointer to paper URLs are greatly appreciated.
One can use moments, including fractional moments,
or other such devices.
Quote: Some people mention this can be done
by "drawing mixing proportions from Dirichlet distribution".
But that is vague for me on how it is done and why.
Please advice.
The Dirichlet distribution is the multivariate analog
of the Beta distribution. It is used (too often) as
a convenient prior distribution.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin at (no spam) stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558 |
|
|
| Back to top |
|
| |
|
Page 1 of 1
All times are GMT - 5 Hours
The time now is Sun Oct 12, 2008 3:16 pm
|
|