How do you decide how big your sample should be?

Ok I have k = N/n

What next?

Rob <user@example.com> wrote in news:TEEmj.7232$421.4228@news-
server.bigpond.net.au:

How do you decide how big your sample should be?

Ok I have k = N/n

What next?


You need to let us know what you plan to do with those three numbers.

Rob <user@example.com> wrote in news:TEEmj.7232$421.4228@news-
server.bigpond.net.au:

How do you decide how big your sample should be?

Ok I have k = N/n

OK lets say I was doing an employee survey total population 1000 (N)

Using a random number generator I pick a sample of (n)? employees to
survey.

How do I work out what size my sample I would need to get a
representative sample? What is considered to be significant?

While I have you - if I don't use random sample ie. I invite everybody
in the company to take part in an anonymous online survey and only take
the people that reply. How do I determine if my results are meaningful?

I mean if 1% reply I would say the result do not mean much.

If 100% reply then I could be confident of the response. How do I
determine what size sample is big enough.

Sun, 27 Jan 2008 08:41:02 GMT from Rob <user@example.com>:
Rob <user@example.com> wrote in news:TEEmj.7232$421.4228@news-
server.bigpond.net.au:

How do you decide how big your sample should be?

Ok I have k = N/n
OK lets say I was doing an employee survey total population 1000 (N)

Using a random number generator I pick a sample of (n)? employees to
survey.

Still not sure what k is meant to be.

How do I work out what size my sample I would need to get a
representative sample? What is considered to be significant?

Is this course work?

If so, it looks like right now you're in the
sampling-and-descriptive-statistics portion of the course, and your
question will be answered in the inferential-statistics portion.

Briefly, "representative" and "significant" are matters of degree.
You have to decide up front what an acceptable significance level is.
(5% and 1% and 0.1% are common choices, but not the only ones.) Or if
you're trying to estimate a quantity, you decide up front what margin
of error you can accept and what level of confidence you need. Either
way, once those decisions are made then there's a formula that tells
you how big a sample you need.

(The formulas also require you to know something about your
population -- often this is obtained through a small pilot study.)

While I have you - if I don't use random sample ie. I invite everybody
in the company to take part in an anonymous online survey and only take
the people that reply. How do I determine if my results are meaningful?

That one's easy: they're not. All the statistical techniques you will
learn depend on having a good random sample. If you have a self-
selected sample, there's an excellent chance they are not
representative of the population.

I mean if 1% reply I would say the result do not mean much.

Not necessarily. Presidential polls typically sample about 1000
voters, which is well under 1% of all voters, yet they are accurate
to within ±3%. When testing yes/no data, what matters is how many are
in the sample and not what percent of the population is in the
sample.

If 100% reply then I could be confident of the response. How do I
determine what size sample is big enough.

Tell us more about what you're trying to measure, and what
significance level or confidence level you're shooting for.

Side question here - When dealing in surveys aren't the samples always
to some extent, self selected anyway?

Not every one is going to respond to a survey. You can't force them and
if you could they might give false information just to get rid of you.
[snip, rest

- taking one digression here -

On Mon, 28 Jan 2008 06:00:49 GMT, Rob <user@example.com> wrote:

[snip, much]
Side question here - When dealing in surveys aren't the samples always
to some extent, self selected anyway?

Not every one is going to respond to a survey. You can't force them and
if you could they might give false information just to get rid of you.
[snip, rest

"Self-selected", sure, if you are talking only about the type
of survey that uses one phone call. By the way, I think that
good political pollers do make more than *one* attempt to
reach a number. And they should keep track of which attempt
succeeded, in case "hard to reach" makes a difference in outcome.

Similarly, there are surveys that use mail for contact. Or
follow up phone calls with mail, etc. - If the "quick responders"
look just like the "slow responders", the pollster will be happier.

The U.S. government publishes surveys of employers, and surveys
of markets, though this is the sort of knowledge-seeking that the
Bush Administration has worked to de-fund. (Republican tax and
economic policies sell better with ignorance.)

Sample size calculations tell you how many subjects you need to find a
hypothesized significant difference (or some other quantity) with an
esimated variability. This tells you nothing about a representative
sample. In fact, it is conceivable that the less representative
sample you have, the more likely you are to get a significant
result.

I don't know if there is a "test" that tells you whether your sample
is representative. You can have a perfectly random selection process
but it is the response that can introduce bias.

You have to look at the characteristics of your sample that responded
and compare them to the characteristics of your target population. In
a company, I could think of several important factors to
match......age, experience, education, proportion from each department
or division, general level or rank, salary, etc. It all depends on
what you are looking at. You could use confidence intervals of your
sample and see if they contain the company average or proportion (if
you have access) but then keep in mind, you are penalized for having a
very large sample size.

Hope I am at least in the ballpark of what you are looking for.
Opinion polls and commercial surveys do a nice job of representative
results. But...they have ways of weighting certain responses based
on their target populations. So there is a level of complexity there
that you probably won't get into.

mcap wrote:
Sample size calculations tell you how many subjects you need to find a
hypothesized significant difference (or some other quantity) with an
esimated variability.  This tells you nothing about a representative
sample.  In fact, it is conceivable that the less representative
sample you have, the more likely you are to get a significant
result.

I don't know if there is a "test" that tells you whether your sample
is representative. You can have a perfectly random selection process
but it is the response that can introduce bias.

You have to look at the characteristics of your sample that responded
and compare them to the characteristics of your target population.  In
a company, I could think of several important factors to
match......age, experience, education, proportion from each department
or division, general level or rank, salary, etc.  It all depends on
what you are looking at.  You could use confidence intervals of your
sample and see if they contain the company average or proportion (if
you have access) but then keep in mind, you are penalized for having a
very large sample size.

Hope I am at least in the ballpark of what you are looking for.
Opinion polls and commercial surveys do a nice job of representative
results.   But...they have ways of weighting certain responses based
on their target populations.  So there is a level of complexity there
that you probably won't get into.

Interesting ideas - I will be making another post on Generation
differences (You know Baby boomers, Gen X,  Gen Y etc) I would be
interested on your feed back on that.

In scientific papers it is always the same 95% confidence interval seems
to be pretty well universally accepted.  I was hoping for an answer like
  "In my organization when we do employee surveys we am for a confidence
interval of X"

What are the names of the formulas used for determining the size of
sample so I can search for them?- Hide quoted text -

- Show quoted text -

At work when I am required to participate in a survey that is not
anonymous there are certain questions that I will not answer truthfully.

How can they follow up if they do not know who I am?