| |
 |
|
|
Science Forum Index » Space - Consult Forum » Questionnaire design...
Page 1 of 1
|
| Author |
Message |
| Burger... |
Posted: Sat Jun 07, 2008 3:14 pm |
|
|
|
Guest
|
Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?
1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent
or
1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent
Thanks in advance for any ideas on this! |
|
|
| Back to top |
|
| Stan Brown... |
Posted: Sat Jun 07, 2008 5:14 pm |
|
|
|
Guest
|
Sat, 7 Jun 2008 22:14:55 +0200 from Burger <burger1965 at (no spam) hotpop.com>:
Quote: Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?
1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent
or
1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent
I'm not a professional statistician, but my gut reaction is "shorter
phrases are better".
But I don't care for either list that much. In both of them, choices
4 and 5 seem the same to me, and I also wonder why you don't offer
the option of "completely".
--
Stan Brown, Oak Road Systems, Tompkins County, New York, USA
http://OakRoadSystems.com
Shikata ga nai... |
|
|
| Back to top |
|
| Richard Ulrich... |
Posted: Sat Jun 07, 2008 7:30 pm |
|
|
|
Guest
|
On Sat, 7 Jun 2008 22:14:55 +0200, "Burger" <burger1965 at (no spam) hotpop.com>
wrote:
Quote: Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?
1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent
or
1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent
Like Stan says, the gradation is not completely clear
on 4 and 5.
British subjects are apt to read "quite" as either what
Americans mean as "quite" or else as "not quite".
You might find any textbook with the word Psychometrics
in the title. Google-scholar can help.
Also, what you have are items on a 1-5 scale, intended
to specify equal intervals.
I know that the term "Likert" gets used loosely, but the
original Likert scales specified "agreement-disagreement"
using symmetrical terms and where the expected average
was in the middle. I don't think you need to use
"Likert" here unless you at least achieve the symmetry.
--
Rich Ulrich
http://www.pitt.edu/~wpilib/index.html |
|
|
| Back to top |
|
| Stan Devia... |
Posted: Mon Jun 09, 2008 6:08 pm |
|
|
|
Guest
|
On Jun 8, 9:30 am, Richard Ulrich <Rich.Ulr... at (no spam) comcast.net> wrote:
Quote: On Sat, 7 Jun 2008 22:14:55 +0200, "Burger" <burger1... at (no spam) hotpop.com
wrote:
Hi everyone! I am not sure if this is the right newsgroup but can't find a
better one. I hope you are willing to take a look at it.
For a Likert type kind of question (something like "does this product
satisfy your needs?"), I have to choose between two lists that I can offer
respondents as potential answers. Both are possible but which would you say
is preferable?
1. Not at all
2. A little
3. Somewhat
4. Quite a lot
5. To a great extent
or
1. Not at all
2. To a small extent
3. To a moderate extent
4. To a large extent
5. To a great extent
Like Stan says, the gradation is not completely clear
on 4 and 5.
British subjects are apt to read "quite" as either what
Americans mean as "quite" or else as "not quite".
You might find any textbook with the word Psychometrics
in the title. Google-scholar can help.
Also, what you have are items on a 1-5 scale, intended
to specify equal intervals.
I know that the term "Likert" gets used loosely, but the
original Likert scales specified "agreement-disagreement"
using symmetrical terms and where the expected average
was in the middle. I don't think you need to use
"Likert" here unless you at least achieve the symmetry.
--
Rich Ulrich
http://www.pitt.edu/~wpilib/index.html
I agree. Most Likert scales I've seen go in order Very Satisfied;
Satisfied; Neither Satisfied or Dissatisfied; Dissatisfied; Very
Dissatisfied. |
|
|
| Back to top |
|
| John Uebersax... |
Posted: Tue Jun 10, 2008 2:06 am |
|
|
|
Guest
|
On Jun 10, 6:08 am, Stan Devia <stan.de... at (no spam) gmail.com> wrote:
Quote:
...Very Satisfied;
Satisfied; Neither Satisfied or Dissatisfied; Dissatisfied; Very
Dissatisfied.
I agree that this is better than the original two options. I'd
possibly consider reversing this order, so that responses go from Very
Dissatisfied to Very Satisfied.
I'd also consider including a Likert-type discrete visual analog scale
to accompany the written response options, e.g.:
How satisfied are you with the service you received?
1 2 3 4 5
+---------+---------+---------+---------+
Very Very
Dis. Dis. Neither Satisf. Satisf.
Circle value that best applies.
John Uebersax PhD |
|
|
| Back to top |
|
| Kiran... |
Posted: Tue Jun 10, 2008 7:25 am |
|
|
|
Guest
|
|
| Back to top |
|
| Burger... |
Posted: Tue Jun 10, 2008 8:07 am |
|
|
|
Guest
|
Thanks for all the advice! I agree that a symmetrical scale is better as it
approached "equal intervals" more. I also agree that the difference between
the original labels for 4 and 5 is somewhat vague. I got these lists from
http://www.gifted.uconn.edu/siegle/research/instrument%20Reliability%20and%2
0Validity/Likert.html and made some changes to them, which clearly didn't
make the lists better. Thanks again, I will make the necessary changes.
"John Uebersax" wrote ...
On Jun 10, 6:08 am, Stan Devia <stan.de... at (no spam) gmail.com> wrote:
Quote:
...Very Satisfied;
Satisfied; Neither Satisfied or Dissatisfied; Dissatisfied; Very
Dissatisfied.
I agree that this is better than the original two options. I'd
possibly consider reversing this order, so that responses go from Very
Dissatisfied to Very Satisfied.
I'd also consider including a Likert-type discrete visual analog scale
to accompany the written response options, e.g.:
How satisfied are you with the service you received?
1 2 3 4 5
+---------+---------+---------+---------+
Very Very
Dis. Dis. Neither Satisf. Satisf.
Circle value that best applies.
John Uebersax PhD |
|
|
| Back to top |
|
| John Uebersax... |
Posted: Wed Jun 11, 2008 10:31 pm |
|
|
|
Guest
|
Interesting paper -- thanks for posting.
They listed the response levels as going from Very satistfied to Very
dissatisfied. Potentially that may create a positive response bias:
seeing the positive levels first (assunming one reads from left to
right), people may be more inclined to check the initial response
levels.
Further, since this layout is, in a sense, mildly counter-intuitive
(i.e., the usual convention, as with a simple number line or x-axis is
to proceed from smaller/negative values on the left to larger/positive
values on the right). Thus the resopondent may see that the
researcher has reversed this convention, and take it as a subtle cue
to make positive responses. (And, of course, such a bias is often
already implicit in the task, as frequently such surveys are used by
organizations to look good, and many people are happy to oblige.)
I'm not agruging strongly for any of these points -- just want to
mention them as things to consider in designing a survey.
John Uebersax PhD
On Jun 10, 7:25 pm, Kiran <kiranga... at (no spam) gmail.com> wrote:
|
|
|
| Back to top |
|
| Art Kendall... |
Posted: Thu Jun 12, 2008 7:54 am |
|
|
|
Guest
|
In addition, going from (negative, lowest, less) on the left, to
positive, highest, more) on the right is a widespread convention in
Western cultures in many contexts. Using the numeric labels and going
from left to right may help activate the cognitive schema of the number
line so that people implicitly treat the response as closer to interval.
When you have a series of items for a summative scale, it is usually
better to leave the response scale as from less to more and balance the
direction via the wording in the question.
Art Kendall
Social Research Consultants
John Uebersax wrote:
Quote: Interesting paper -- thanks for posting.
They listed the response levels as going from Very satistfied to Very
dissatisfied. Potentially that may create a positive response bias:
seeing the positive levels first (assunming one reads from left to
right), people may be more inclined to check the initial response
levels.
Further, since this layout is, in a sense, mildly counter-intuitive
(i.e., the usual convention, as with a simple number line or x-axis is
to proceed from smaller/negative values on the left to larger/positive
values on the right). Thus the resopondent may see that the
researcher has reversed this convention, and take it as a subtle cue
to make positive responses. (And, of course, such a bias is often
already implicit in the task, as frequently such surveys are used by
organizations to look good, and many people are happy to oblige.)
I'm not agruging strongly for any of these points -- just want to
mention them as things to consider in designing a survey.
John Uebersax PhD
On Jun 10, 7:25 pm, Kiran <kiranga... at (no spam) gmail.com> wrote:
You will find a comprehensive white paper on survey best practices at
this link.
http://www.zoomerang.com/whitepapers/customersat.pdf
Thanks,
Kiran
|
|
|
| Back to top |
|
| ThoML... |
Posted: Fri Jun 13, 2008 7:58 pm |
|
|
|
Guest
|
Quote: How satisfied are you with the service you received?
1 2 3 4 5
+---------+---------+---------+---------+
Very Very
Dis. Dis. Neither Satisf. Satisf.
In my experience, people do not always look as closely at the labels
as a data analysist would want them to and tend to answer such
questions with respect to the outer labels. For this reason, I'd like
to put a minor variation of the above up for discussion:
Were you rather satisfied or dissatisfied ...
1 2 3 4 5
+---------+---------+---------+---------+
Dis. Satisf.
It depends on how you want to analyse and present the data, but IMHO
this often has some advantages.
Regards,
Thomas. |
|
|
| Back to top |
|
| John Uebersax... |
Posted: Tue Jun 17, 2008 1:29 am |
|
|
|
Guest
|
On Jun 14, 7:58 am, ThoML <micat... at (no spam) gmail.com> wrote:
Quote: ... people do not always look as closely at the labels
as a data analysist would want them to and tend to answer such
At least some people will look at the labels. And while it's scarcely
infallible, the argument 'from tradition' might apply here: if it's
better to omit the intervening labels, why didn't Likert (and hundreds
of later researchers) discover and implement that simplification?
Quote: questions with respect to the outer labels. For this reason, I'd like
to put a minor variation of the above up for discussion:
Were you rather satisfied or dissatisfied ...
1 2 3 4 5
+---------+---------+---------+---------+
Dis. Satisf.
It depends on how you want to analyse and present the data, but IMHO
this often has some advantages.
I don't see any advantages here. Unless perhaps you have a true
continuous 'thermometer' scale.
Note that with Java code etc. it's getting fairly simple to insert a
continuous 'thermometer' scale item in online surveys.
HTH
John Uebersax PhD |
|
|
| Back to top |
|
| ThoML... |
Posted: Tue Jun 17, 2008 2:14 am |
|
|
|
Guest
|
Quote: if it's better to omit the intervening labels, why didn't Likert
[...]
I don't see any advantages here. Unless perhaps you have a true
continuous 'thermometer' scale.
If it's a potentially acceptable solution, probably depends on how you
conduct the survey (online, telephone, face-to-face, paper/mail ...),
on the research field/discipline (incl. a discipline's "tradition")/
context/culture, and on what is feasible given a project's
constraints.
Regards,
Thomas. |
|
|
| Back to top |
|
| ... |
Posted: Wed Jun 25, 2008 8:09 am |
|
|
|
Guest
|
As I recall, Likert scales are agree-disagree (well, strongly disagree
to strongly agree) scales (usually in either 5 or 7 gradations). The
items relate to the extent of agreement with a statement per se
(theoretically anyway) rather than the underlying phenomenon. Any
way, it would look like this:
This product satisfies my needs. Strongly disagree disagree
slightly disagree neither agree nor disagree slightly agree
agree strongly agree
If you were to use this for academic research, you would ask
respondents to indicate their level of agreement with several other
statements as well (This product does not meet my needs, I am
satisfied with this product, I would buy this product again, I would
recommend this product to a friend, ...). Then you would assess the
reliability (maybe with Cronbach's alpha) and factor structure (using
factor analysis). I'm not sure about other software, but it's fairly
easy on SPSS (even if you only have the base system).
Then, you use the valid items to create a summated scale (add the
scores, using reverse-coding for items like I"This product does not
meet my needs." Most social scientists would use parametric
procedures (mean, standard deviation, t-test) for the summated scale
even if they wouldn't use them for a single item (after all, the level
of measurement is ordinal, not interval.). |
|
|
| Back to top |
|
| |
|
Page 1 of 1
All times are GMT - 5 Hours
The time now is Fri Dec 05, 2008 10:37 am
|
|