A questionnaire is not a list of questions you find interesting. Every item must trace back to an indicator in your theoretical framework, which traces back to a definition, which traces back to a research question. Build it in that order and it will survive validity testing. Build it by instinct and you will discover the problem only after your pilot data comes back.
These seven steps take you from framework to a piloted instrument ready for main data collection.
Step 1: List your constructs and their indicators
Before writing a single item, write out each variable in your study and the indicators or dimensions it breaks into. Those indicators come from the literature you reviewed, not from what seems reasonable to you.
For example, if you are measuring employee performance and your framework defines it through quantity of work, quality of work and timeliness, then you have three indicators and every performance item must belong to one of them.
Expected output: a table with three columns — construct, indicator, source in the literature. Keep this table. It becomes an appendix and answers most instrument questions in the viva before they are asked.
Step 2: Decide whether to adapt or build from scratch
Adapting a published instrument is almost always the better choice for an FYP, and it is not a shortcut. A validated instrument has already been tested on real respondents, and using one lets you compare your results against previous studies.
Rules when adapting:
- Cite the original source properly in your methodology chapter.
- State clearly what you changed and why, whether wording, context or number of items.
- Re-test validity and reliability on your own sample. Results reported for the original version do not automatically transfer to your adapted version or to a different population.
Build from scratch only when nothing suitable exists for your construct, and expect to spend significantly longer on piloting if you do.
Step 3: Write items that measure one thing each
Most items that fail validity testing fail for reasons visible before any data is collected. Check every item against this list:
- No double-barrelled items. “The system is fast and easy to use” asks two questions, and a respondent who finds it fast but confusing cannot answer honestly.
- No leading wording. “How excellent was the service?” presumes the answer. Use neutral phrasing.
- No jargon or technical terms your respondents may not share. Write for the actual population, not for your examiners.
- No negatives inside a positive scale unless you intend reverse-coded items and remember to recode them before analysis. Forgetting to recode is a very common source of nonsensical reliability results.
- Keep items short. Long sentences get skimmed, and skimmed items produce noisy data.
- One time reference. Do not mix “usually” and “last week” across items measuring the same construct.
Because many Malaysian students survey respondents who are answering in a second language, plain wording matters more here than general guidance suggests. If your respondents will read in Malay, translate carefully and have the translation checked by someone fluent in both languages, then state the translation procedure in your methodology chapter.
Step 4: Choose your response scale deliberately
The scale you choose determines what analysis is available to you later, so decide it with your analysis plan in mind rather than by habit.
- Five-point or seven-point agreement scales are the standard for attitude and perception constructs. Seven points give slightly finer discrimination; five are easier for respondents and quicker to complete.
- An even-numbered scale removes the neutral midpoint and forces a direction. Use it deliberately, and justify it, because removing the midpoint from respondents who genuinely have no view introduces its own distortion.
- Frequency scales suit behaviour rather than attitude, but define the anchors concretely. “Often” means different things to different people; “three or more times per week” does not.
- Keep the scale consistent within a construct. Mixing scale types inside one construct breaks the summed score.
Remember the consequence for analysis: a single item on an agreement scale is ordinal data, while a summed multi-item scale is routinely treated as continuous. That distinction decides which tests you can run, as covered in the guide on which statistical test to use for your FYP data.
Step 5: Structure and sequence the questionnaire
Order affects completion rates and data quality.
- Cover statement. Who you are, the purpose, how long it takes, confidentiality, and voluntary participation. Many ethics committees require specific wording here.
- Screening items, if you need to confirm respondents meet your criteria.
- Main construct items, grouped by construct with a short heading for each section.
- Demographic items last. Placing them at the end reduces early drop-off, and respondents who have already invested effort are more likely to complete them.
- Closing thanks and contact details for questions.
Keep the whole thing as short as your framework allows. Every additional item costs completion rate, and a shorter instrument fully completed beats a comprehensive one abandoned halfway.
Step 6: Pilot test before the main collection
The pilot is where design problems surface cheaply. Run it with respondents who resemble your real sample but who are excluded from the main study. A commonly used working figure in Malaysian faculties is around 30 pilot respondents, though this is convention rather than a national rule, so check your own guidelines.
The pilot gives you three things:
- Item validity. Correlate each item score against the total score for its construct and identify items that do not behave like the rest.
- Reliability. Compute internal consistency for each construct, on the retained items only.
- Practical feedback. How long did it take? Which items did people ask about? Ambiguity shows up as questions.
The order matters: test item validity first, remove items that fail, then recompute reliability on what remains. Computing reliability while retaining items you have already declared invalid is a frequent error and an easy one for an examiner to catch.
Step 7: Fix what failed, and know when the problem is bigger
If one or two items fail, remove or rewrite them and note it in your methodology chapter. If more than about a third of items fail, the problem is usually not the respondents. Look for these causes:
- Items that were never properly derived from the construct definition.
- Double-barrelled or ambiguous wording that different respondents read differently.
- Reverse-coded items that were not recoded before analysis.
- A construct that is genuinely two constructs, with items splitting into distinct groups.
Rewrite and pilot again. Letting a construct proceed with only two surviving items will produce a question at the viva that is difficult to answer well.
Writing all of this into your methodology chapter
Everything above needs to appear in the instrument section of your methodology chapter: structure, source of items, scale, translation procedure if any, pilot results, and any items removed. The full chapter structure is set out in the guide on how to write the methodology chapter of a Malaysian FYP or thesis.
How much justification is expected rises with the level of your degree, as explained in the comparison of FYP, thesis and dissertation in Malaysia.
From instrument to a chapter that holds together
A well-built questionnaire still has to be described in prose that stays consistent with your framework, your sampling and your analysis plan. That description is where instrument sections most often come apart, usually because an item was added or removed after the framework was written.
Tesify helps you draft the instrument and methodology sections so that constructs, indicators, items and analysis stay aligned as the document grows, with you remaining the author and responsible for every methodological decision.
Draft your instrument section in Tesify
Frequently asked questions
How many items should each construct have?
Three to five items per construct is a common working range, giving enough coverage to compute reliability while keeping the instrument manageable. Fewer than three makes internal consistency difficult to establish meaningfully.
Can I use an online form instead of paper?
Yes, and it is standard practice. Online collection speeds distribution and removes transcription errors. Check whether your ethics approval covers online data collection and how the platform stores responses, since data protection expectations apply.
What reliability value is acceptable?
A coefficient of 0.70 is the most widely cited lower bound for acceptable internal consistency. It is a scholarly convention rather than an official rule, and some fields apply different thresholds. Cite the source you are following.
Is a very high reliability value good?
Not necessarily. A value close to 1.0 often signals that your items are near-duplicates of each other rather than that your instrument is excellent. It suggests redundancy: several items asking the same thing in slightly different words.
Do I need to translate my questionnaire?
It depends on your respondents. If they are more comfortable in Malay, translation improves data quality. Where you translate, describe the procedure in your methodology chapter and have the translation reviewed by someone fluent in both languages.
Should pilot respondents be included in the main sample?
No. They have already seen the instrument, which may influence how they respond. Exclude them and state that exclusion explicitly in your methodology chapter.
What if my response rate is low?
Distribute more, extend the collection period if your timeline allows, and follow up politely. If you still fall short of your calculated sample size, report the achieved number honestly, state the response rate, and discuss possible non-response bias in your limitations.
Can I add questions after data collection has started?
No. Changing the instrument mid-collection means your respondents did not all answer the same thing, which invalidates comparison across the dataset. If something essential is missing, stop, revise, and restart collection.
How do I handle incomplete responses?
Decide your rule before you look at the data and state it in the chapter, for example excluding any response missing more than a set proportion of items. Report how many were excluded and why. A rule chosen after seeing results invites the suspicion that it was chosen to produce them.
Do I need permission to use a published instrument?
Many published instruments are freely available for academic use, but some require permission from the author or publisher. Check the terms, and where required, request permission by email early. Always cite the original regardless of whether permission was needed.