| Computers Forum Index » Computer Artificial Intelligence - Neural Nets » worry about "overfitting"?... |
|
Page 1 of 1 |
|
| Author |
Message |
| MBALOVER... |
 Posted: Sun Oct 18, 2009 12:47 am |
|
|
|
Guest
|
Hello,
I fit a 3-order polynomial (with 33 coefficients) to a training data
set that contains 50 samples. I am worried that the resulting model
may be overfitted since the training set is not too large.
So, I try to test the resulting model with another set of 50 samples.
There is no common sample between the training and testing sets. The
testing error still looks good.
Do you think that if I should still be worried about overfitting? Or
is the validation step shows that there is no overfitting problem with
my model?
Thanks |
|
|
| Back to top |
|
|
|
| Greg Heath... |
| Posted: Sun Oct 18, 2009 1:19 am |
|
|
|
Guest
|
On Oct 17, 8:47 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
Quote: Hello,
I fit a 3-order polynomial (with 33 coefficients)
???
A 3rd order polynomial has 4 coefficients.
Hope this helps.
Greg |
|
|
| Back to top |
|
|
|
| amzoti... |
| Posted: Sun Oct 18, 2009 1:22 am |
|
|
|
Guest
|
On Oct 17, 5:47 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
Quote: Hello,
I fit a 3-order polynomial (with 33 coefficients) to a training data
set that contains 50 samples. I am worried that the resulting model
may be overfitted since the training set is not too large.
Thanks
Did you mean with 33 data points here?
Your statement is very confusing otherwise.
~A |
|
|
| Back to top |
|
|
|
| MBALOVER... |
| Posted: Sun Oct 18, 2009 1:50 am |
|
|
|
Guest
|
I am sorry for making confusion. OK, let's forget about the order of
the polynomial.
Let's say that the model has 33 coefficients. I train the model with
50 training samples. I test the model with another 50 samples. (No
common sample between the training and testing sets).
The validation error still looks good.
My question is that if I should still care about overfitting. I hope I
should not since the validation error looks good.
Thanks,
The model is a matrix with
On Oct 17, 8:22 pm, amzoti <amz... at (no spam) gmail.com> wrote:
Quote: On Oct 17, 5:47 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
Hello,
I fit a 3-order polynomial (with 33 coefficients) to a training data
set that contains 50 samples. I am worried that the resulting model
may be overfitted since the training set is not too large.
Thanks
Did you mean with 33 data points here?
Your statement is very confusing otherwise.
~A |
|
|
| Back to top |
|
|
|
| Greg Heath... |
| Posted: Sun Oct 18, 2009 3:14 am |
|
|
|
Guest
|
On Oct 17, 9:50 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
Quote: I am sorry for making confusion. OK, let's forget about the order of
the polynomial.
Let's say that the model has 33 coefficients. I train the model with
50 training samples. I test the model with another 50 samples. (No
common sample between the training and testing sets).
Incorrect terminology.
You have 1 sample with 50 cases (or observations)
for training and another sample of 50 cases for validation.
Quote: The validation error still looks good.
My question is that if I should still care about overfitting.
I hope I should not since the validation error looks good.
You should care. Even though the validation error "looks good",
whatever that is supposed to mean.
A more definitive answer depends on the topology
of the model, the coefficient estimation algorithm
and the probability distribution of the population.
Quote: The model is a matrix with
33 what?
Greg |
|
|
| Back to top |
|
|
|
| Phil Sherrod... |
| Posted: Sun Oct 18, 2009 10:20 pm |
|
|
|
Guest
|
On 17-Oct-2009, Greg Heath <heath at (no spam) alumni.brown.edu> wrote:
Quote: I fit a 3-order polynomial (with 33 coefficients)
???
A 3rd order polynomial has 4 coefficients.
Only if there is only a single independent variable. Maybe he has 11
independen variables and a constant (which would require 34, not 33).
--
Phil Sherrod
http://www.dtreg.com -- Neural networks, SVM, Decision trees |
|
|
| Back to top |
|
|
|
| Greg Heath... |
| Posted: Mon Oct 19, 2009 8:33 am |
|
|
|
Guest
|
On Oct 17, 9:50 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
Quote: I am sorry for making confusion. OK, let's forget about the order of
the polynomial.
Let's say that the model has 33 coefficients. I train the model with
50 training samples. I test the model with another 50 samples. (No
common sample between the training and testing sets).
The validation error still looks good.
My question is that if I should still care about overfitting. I hope I
should not since the validation error looks good.
Thanks,
The model is a matrix with
It would help if you gave more details:
What kind of problem?
How many input variables?
How many output variables?
What kind of model?
What kind of estimation algorithm?
etc
Hope this helps.
Greg |
|
|
| Back to top |
|
|
|
| MBALOVER... |
| Posted: Wed Oct 21, 2009 2:34 am |
|
|
|
Guest
|
Hi Greg and others,
Your answers are helpful.
Greg, if I do K-cross validation and stilll get a good result. Do you
think if overfitting still probably occurs.
Sorry that I could not describe the model here, it is complicated to
describe by text.
Thanks
On Oct 19, 3:33 am, Greg Heath <he... at (no spam) alumni.brown.edu> wrote:
Quote: On Oct 17, 9:50 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
I am sorry for making confusion. OK, let's forget about the order of
the polynomial.
Let's say that the model has 33 coefficients. I train the model with
50 training samples. I test the model with another 50 samples. (No
common sample between the training and testing sets).
The validation error still looks good.
My question is that if I should still care about overfitting. I hope I
should not since the validation error looks good.
Thanks,
The model is a matrix with
It would help if you gave more details:
What kind of problem?
How many input variables?
How many output variables?
What kind of model?
What kind of estimation algorithm?
etc
Hope this helps.
Greg |
|
|
| Back to top |
|
|
|
| Greg Heath... |
| Posted: Wed Oct 21, 2009 4:44 am |
|
|
|
Guest
|
On Oct 20, 10:34 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
Quote: Hi Greg and others,
Your answers are helpful.
Greg, if I do K-cross validation and stilll get a good result. Do you
think if overfitting still probably occurs.
Sorry that I could not describe the model here, it is complicated to
describe by text.
Thanks
On Oct 19, 3:33 am, Greg Heath <he... at (no spam) alumni.brown.edu> wrote:
On Oct 17, 9:50 pm, MBALOVER <mbalov... at (no spam) gmail.com> wrote:
I am sorry for making confusion. OK, let's forget about the order of
the polynomial.
Let's say that the model has 33 coefficients. I train the model with
50 training samples. I test the model with another 50 samples. (No
common sample between the training and testing sets).
The validation error still looks good.
My question is that if I should still care about overfitting. I hope I
should not since the validation error looks good.
Thanks,
The model is a matrix with
It would help if you gave more details:
What kind of problem?
How many input variables?
How many output variables?
What kind of model?
What kind of estimation algorithm?
Please do not top post. It makes it difficult to follow the thread.
Hard to answer your question without more details.
Greg |
|
|
| Back to top |
|
|
|
|
