Main Page | Report this Page
 
   
Science Forum Index  »  Space - Consult Forum  »  Questionnaire design...
Page 1 of 1    
Author Message
Burger...
Posted: Sat Jun 07, 2008 3:14 pm
Guest
Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?

1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent

or

1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent

Thanks in advance for any ideas on this!
Stan Brown...
Posted: Sat Jun 07, 2008 5:14 pm
Guest
Sat, 7 Jun 2008 22:14:55 +0200 from Burger <burger1965 at (no spam) hotpop.com>:
Quote:
Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?

1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent

or

1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent

I'm not a professional statistician, but my gut reaction is "shorter
phrases are better".

But I don't care for either list that much. In both of them, choices
4 and 5 seem the same to me, and I also wonder why you don't offer
the option of "completely".

--
Stan Brown, Oak Road Systems, Tompkins County, New York, USA
http://OakRoadSystems.com
Shikata ga nai...
Richard Ulrich...
Posted: Sat Jun 07, 2008 7:30 pm
Guest
On Sat, 7 Jun 2008 22:14:55 +0200, "Burger" <burger1965 at (no spam) hotpop.com>
wrote:

Quote:
Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?

1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent

or

1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent


Like Stan says, the gradation is not completely clear
on 4 and 5.

British subjects are apt to read "quite" as either what
Americans mean as "quite" or else as "not quite".

You might find any textbook with the word Psychometrics
in the title. Google-scholar can help.

Also, what you have are items on a 1-5 scale, intended
to specify equal intervals.
I know that the term "Likert" gets used loosely, but the
original Likert scales specified "agreement-disagreement"
using symmetrical terms and where the expected average
was in the middle. I don't think you need to use
"Likert" here unless you at least achieve the symmetry.

--
Rich Ulrich

http://www.pitt.edu/~wpilib/index.html
Stan Devia...
Posted: Mon Jun 09, 2008 6:08 pm
Guest
On Jun 8, 9:30 am, Richard Ulrich <Rich.Ulr... at (no spam) comcast.net> wrote:
Quote:
On Sat, 7 Jun 2008 22:14:55 +0200, "Burger" <burger1... at (no spam) hotpop.com
wrote:



Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?

1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent

or

1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent

Like Stan says, the gradation is not completely clear
on 4 and 5.

British subjects are apt to read "quite" as either what
Americans mean as "quite" or else as "not quite".

You might find any textbook with the word Psychometrics
in the title. Google-scholar can help.

Also, what you have are items on a 1-5 scale, intended
to specify equal intervals.
I know that the term "Likert" gets used loosely, but the
original Likert scales specified "agreement-disagreement"
using symmetrical terms and where the expected average
was in the middle. I don't think you need to use
"Likert" here unless you at least achieve the symmetry.

--
Rich Ulrich

http://www.pitt.edu/~wpilib/index.html

I agree. Most Likert scales I've seen go in order Very Satisfied;
Satisfied; Neither Satisfied or Dissatisfied; Dissatisfied; Very
Dissatisfied.
John Uebersax...
Posted: Tue Jun 10, 2008 2:06 am
Guest
On Jun 10, 6:08 am, Stan Devia <stan.de... at (no spam) gmail.com> wrote:
Quote:

...Very Satisfied;
Satisfied; Neither Satisfied or Dissatisfied; Dissatisfied; Very
Dissatisfied.

I agree that this is better than the original two options. I'd
possibly consider reversing this order, so that responses go from Very
Dissatisfied to Very Satisfied.

I'd also consider including a Likert-type discrete visual analog scale
to accompany the written response options, e.g.:

How satisfied are you with the service you received?

1 2 3 4 5
+---------+---------+---------+---------+
Very Very
Dis. Dis. Neither Satisf. Satisf.

Circle value that best applies.

John Uebersax PhD
Kiran...
Posted: Tue Jun 10, 2008 7:25 am
Guest
You will find a comprehensive white paper on survey best practices at
this link.

http://www.zoomerang.com/whitepapers/customersat.pdf

Thanks,
Kiran
Burger...
Posted: Tue Jun 10, 2008 8:07 am
Guest
Thanks for all the advice! I agree that a symmetrical scale is better as it
approached "equal intervals" more. I also agree that the difference between
the original labels for 4 and 5 is somewhat vague. I got these lists from
http://www.gifted.uconn.edu/siegle/research/instrument%20Reliability%20and%2
0Validity/Likert.html and made some changes to them, which clearly didn't
make the lists better. Thanks again, I will make the necessary changes.

"John Uebersax" wrote ...
On Jun 10, 6:08 am, Stan Devia <stan.de... at (no spam) gmail.com> wrote:
Quote:

...Very Satisfied;
Satisfied; Neither Satisfied or Dissatisfied; Dissatisfied; Very
Dissatisfied.

I agree that this is better than the original two options. I'd
possibly consider reversing this order, so that responses go from Very
Dissatisfied to Very Satisfied.

I'd also consider including a Likert-type discrete visual analog scale
to accompany the written response options, e.g.:

How satisfied are you with the service you received?

1 2 3 4 5
+---------+---------+---------+---------+
Very Very
Dis. Dis. Neither Satisf. Satisf.

Circle value that best applies.

John Uebersax PhD
John Uebersax...
Posted: Wed Jun 11, 2008 10:31 pm
Guest
Interesting paper -- thanks for posting.

They listed the response levels as going from Very satistfied to Very
dissatisfied. Potentially that may create a positive response bias:
seeing the positive levels first (assunming one reads from left to
right), people may be more inclined to check the initial response
levels.

Further, since this layout is, in a sense, mildly counter-intuitive
(i.e., the usual convention, as with a simple number line or x-axis is
to proceed from smaller/negative values on the left to larger/positive
values on the right). Thus the resopondent may see that the
researcher has reversed this convention, and take it as a subtle cue
to make positive responses. (And, of course, such a bias is often
already implicit in the task, as frequently such surveys are used by
organizations to look good, and many people are happy to oblige.)

I'm not agruging strongly for any of these points -- just want to
mention them as things to consider in designing a survey.

John Uebersax PhD


On Jun 10, 7:25 pm, Kiran <kiranga... at (no spam) gmail.com> wrote:
Quote:
You will find a comprehensive white paper on survey best practices at
this link.

http://www.zoomerang.com/whitepapers/customersat.pdf

Thanks,
Kiran
Art Kendall...
Posted: Thu Jun 12, 2008 7:54 am
Guest
In addition, going from (negative, lowest, less) on the left, to
positive, highest, more) on the right is a widespread convention in
Western cultures in many contexts. Using the numeric labels and going
from left to right may help activate the cognitive schema of the number
line so that people implicitly treat the response as closer to interval.

When you have a series of items for a summative scale, it is usually
better to leave the response scale as from less to more and balance the
direction via the wording in the question.

Art Kendall
Social Research Consultants


John Uebersax wrote:
Quote:
Interesting paper -- thanks for posting.

They listed the response levels as going from Very satistfied to Very
dissatisfied. Potentially that may create a positive response bias:
seeing the positive levels first (assunming one reads from left to
right), people may be more inclined to check the initial response
levels.

Further, since this layout is, in a sense, mildly counter-intuitive
(i.e., the usual convention, as with a simple number line or x-axis is
to proceed from smaller/negative values on the left to larger/positive
values on the right). Thus the resopondent may see that the
researcher has reversed this convention, and take it as a subtle cue
to make positive responses. (And, of course, such a bias is often
already implicit in the task, as frequently such surveys are used by
organizations to look good, and many people are happy to oblige.)

I'm not agruging strongly for any of these points -- just want to
mention them as things to consider in designing a survey.

John Uebersax PhD


On Jun 10, 7:25 pm, Kiran <kiranga... at (no spam) gmail.com> wrote:
You will find a comprehensive white paper on survey best practices at
this link.

http://www.zoomerang.com/whitepapers/customersat.pdf

Thanks,
Kiran
ThoML...
Posted: Fri Jun 13, 2008 7:58 pm
Guest
Quote:
How satisfied are you with the service you received?

1 2 3 4 5
+---------+---------+---------+---------+
Very Very
Dis. Dis. Neither Satisf. Satisf.

In my experience, people do not always look as closely at the labels
as a data analysist would want them to and tend to answer such
questions with respect to the outer labels. For this reason, I'd like
to put a minor variation of the above up for discussion:

Were you rather satisfied or dissatisfied ...

1 2 3 4 5
+---------+---------+---------+---------+
Dis. Satisf.

It depends on how you want to analyse and present the data, but IMHO
this often has some advantages.

Regards,
Thomas.
John Uebersax...
Posted: Tue Jun 17, 2008 1:29 am
Guest
On Jun 14, 7:58 am, ThoML <micat... at (no spam) gmail.com> wrote:

Quote:
... people do not always look as closely at the labels
as a data analysist would want them to and tend to answer such

At least some people will look at the labels. And while it's scarcely
infallible, the argument 'from tradition' might apply here: if it's
better to omit the intervening labels, why didn't Likert (and hundreds
of later researchers) discover and implement that simplification?

Quote:
questions with respect to the outer labels. For this reason, I'd like
to put a minor variation of the above up for discussion:

Were you rather satisfied or dissatisfied ...

  1         2         3         4         5
  +---------+---------+---------+---------+
Dis.                                   Satisf.

It depends on how you want to analyse and present the data, but IMHO
this often has some advantages.

I don't see any advantages here. Unless perhaps you have a true
continuous 'thermometer' scale.
Note that with Java code etc. it's getting fairly simple to insert a
continuous 'thermometer' scale item in online surveys.

HTH

John Uebersax PhD
ThoML...
Posted: Tue Jun 17, 2008 2:14 am
Guest
Quote:
if it's better to omit the intervening labels, why didn't Likert
[...]
I don't see any advantages here. Unless perhaps you have a true
continuous 'thermometer' scale.

If it's a potentially acceptable solution, probably depends on how you
conduct the survey (online, telephone, face-to-face, paper/mail ...),
on the research field/discipline (incl. a discipline's "tradition")/
context/culture, and on what is feasible given a project's
constraints.

Regards,
Thomas.
...
Posted: Wed Jun 25, 2008 8:09 am
Guest
As I recall, Likert scales are agree-disagree (well, strongly disagree
to strongly agree) scales (usually in either 5 or 7 gradations). The
items relate to the extent of agreement with a statement per se
(theoretically anyway) rather than the underlying phenomenon. Any
way, it would look like this:

This product satisfies my needs. Strongly disagree disagree
slightly disagree neither agree nor disagree slightly agree
agree strongly agree


If you were to use this for academic research, you would ask
respondents to indicate their level of agreement with several other
statements as well (This product does not meet my needs, I am
satisfied with this product, I would buy this product again, I would
recommend this product to a friend, ...). Then you would assess the
reliability (maybe with Cronbach's alpha) and factor structure (using
factor analysis). I'm not sure about other software, but it's fairly
easy on SPSS (even if you only have the base system).

Then, you use the valid items to create a summated scale (add the
scores, using reverse-coding for items like I"This product does not
meet my needs." Most social scientists would use parametric
procedures (mean, standard deviation, t-test) for the summated scale
even if they wouldn't use them for a single item (after all, the level
of measurement is ordinal, not interval.).
 
Page 1 of 1       All times are GMT - 5 Hours
The time now is Fri Dec 05, 2008 10:37 am