The FARES framework for evaluating research questions
What is FARES?
FARES is a framework for evaluating research question quality we use in this course. A strong RQ should be:
- Focused
- Answerable
- Relevant
- Empirical
- Significant
The 5 FARES criteria
1/ Focused
Your research question should have clear scope with defined boundaries.
Considerations:
- Can you explain the scope in 2-3 sentences?
- Are all key terms clearly defined?
- What is specifically included and excluded?
- Is the scope manageable within thesis constraints?
Common problems: Questions contain multiple distinct sub-questions; scope could encompass dozens of studies; key terms are vague; no clear boundaries on population or context.
Ex transformation:
Too broad: “How can we improve speech recognition?”
Well-focused: “How does background noise affect keyword spotting accuracy for elderly users of voice assistants?”
2/ Answerable
Your research question should be feasible to investigate with available methods and resources.
Considerations:
- What data would you need to collect?
- Do established methods exist to address this question?
- Do you have realistic access to necessary resources?
- Can this be completed within the thesis timeline?
- Are there insurmountable practical or ethical barriers?
Common problems: Requires impossible-to-obtain data; would take years to complete; depends on non-existent technology; requires unavailable expertise; has unresolvable ethical barriers.
Example transformation:
Unanswerable: “What will speech technology look like in 2050?”
Answerable: “What acoustic features best predict listener-perceived naturalness in current neural text-to-speech systems?”
3/ Relevant
Your research question should address real needs in research or applications.
Considerations:
- Who would care about the answer and why?
- How does this connect to current debates or needs?
- What would change if you answered this question?
- Does this address a problem that users, researchers, or practitioners face?
- Would results inform future research or improve practice?
Common problems: Results only of interest to the researcher; problem is purely hypothetical; question thoroughly answered already; topic disconnected from field priorities; answering wouldn’t advance knowledge or practice.
Example transformation:
Low relevance: “How does voice assistant usage vary between Tuesday and Wednesday?”
High relevance: “How does accent bias in voice assistants affect user trust across different cultural communities?”
4/ Empirical
Your RQ should be investigable through systematic observation or experimentation.
Key considerations:
- What could you observe, measure, or test?
- How would you know if you had successfully answered it?
- What would different possible answers look like?
- Can key concepts be operationalized into measurable variables?
- Is this about facts/observations or values/opinions?
Common problems: Question is purely philosophical or normative; cannot be tested through data; key concepts cannot be measured; relies entirely on speculation.
Example transformation:
Non-empirical: “Should AI have emotions?”
Empirical: “Do listeners perceive emotional expressions differently in synthetic versus natural speech?”
5/ Significant
Your RQ should lead to results that contribute meaningfully to knowledge or practice.
Key considerations:
- What contribution would answering this make to the field?
- How does this advance beyond existing knowledge?
- What new questions would your results create?
- Would the results matter even if they contradicted expectations?
- Could findings inform future research, policy, or practice?
Common problems: Only confirms well-established knowledge; incremental improvement too small to matter; technical change with no clear benefit; answers a question no one is asking; purely descriptive with no implications.
Example transformation:
Low significance: “What happens if we change parameter X from 0.7 to 0.8 in algorithm Y?”
High significance: “How do prosodic features contribute to perceived speaker identity in neural voice cloning, and what implications does this have for ethical voice synthesis?”
Using FARES in practice
Self-evaluation process
Work through each criterion systematically:
- Write out your current RQ
- For each FARES criterion: Rate your question (strong, needs work, weak), list specific concerns or strengths, identify what would need to change to improve
- Prioritize which 1-2 criteria need the most attention
- Revise: Generate alternative framings that address weaknesses
- Re-evaluate: Does your revised question perform better on FARES?
Peer review using FARES
When providing feedback on a colleague’s RQ:
- Clarify first: Make sure you understand what they’re asking
- Apply each criterion: Which does the question satisfy well? Where could it strengthen?
- Be specific: Don’t just say “it’s too broad” - explain what makes it broad and how to narrow it
- Suggest alternatives: Offer concrete reframings
- Stay constructive: Focus on improving the question, and leave the person out of it
Common pitfalls and solutions
- The “everything” question: Tries to answer too many questions at once. Solution: Break into separate focused questions; choose one as primary.
- The “someday” question: Interesting but not feasible within constraints. Solution: Scale down to what you can accomplish, or shift to a related question with available methods.
- The “who cares” question: Technically answerable but lacks clear significance. Solution: Connect to broader debates, practical problems, or theoretical frameworks.
- The “values” question: Asks what should be, where research can only ask what is. Solution: Reframe to investigate empirical aspects of the issue.
- The “been there” question: Already thoroughly investigated with no new angle. Solution: Identify specific gap, new context, or novel approach that distinguishes your work.
Relationship to other frameworks
FINER framework (Hulley et al., 2013)
The FINER framework evaluates RQs as Feasible, Interesting, Novel, Ethical, and Relevant.
Source: Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013). Designing clinical research (4th ed.). Lippincott Williams & Wilkins.
FARES adapts FINER by: replacing “Feasible” with “Answerable” (emphasizing methodological clarity); adding “Focused” (addressing scope definition critical for graduate work); adding “Empirical” (ensuring questions are testable through data); replacing “Novel” and “Interesting” with “Significant” (focusing on contribution over novelty per se); integrating “Ethical” considerations into “Relevant” and “Answerable” criteria.
PICO framework
Commonly used in health sciences: Population, Intervention, Comparison, Outcome. PICO helps structure what you’re asking about. FARES evaluates whether what you’re asking is a good RQ.
Examples across speech technology domains
Automatic speech recognition
Weak: “How can we make ASR better?” - Fails: focused, answerable, significant (too vague)
Strong: “How does code-switching frequency affect recognition accuracy in Spanish-English bilingual speech, and what does this reveal about current ASR training limitations?” - Focused on specific language pair and phenomenon; answerable through corpus analysis; addresses real multilingual user challenges; testable empirically; informs training approaches for multilingual systems.
Speech synthesis
Weak: “Can we make TTS sound natural?” - Fails: focused (what aspect?), answerable (how to measure?), significant (already well-studied)
Strong: “What prosodic features contribute most to perceived speaker identity in neural voice cloning, and how do these perceptions vary across listener demographics?” - Focused on specific features and perceptual dimensions; answerable through perceptual experiments; critical for identity preservation and ethics; empirically testable; informs both technical development and ethical guidelines.
Affective computing
Weak: “Should AI express emotions?” - Fails: empirical (normative question), answerable (requires philosophical argument)
Strong: “Do listeners perceive emotional authenticity differently when they know speech is synthesized versus natural, and how does disclosure timing affect these perceptions?” - Focused on specific perceptual dimension; answerable through experimental manipulation; critical for transparent AI interaction; empirically testable; informs both ethics and design.
Revising your RQ: a worked example
Initial question: “How does machine learning affect speech technology?”
FARES analysis: Focused: Far too broad; Answerable: Impossible to address comprehensively; Relevant: Generally important but unfocused; Empirical: Could be made empirical but needs specification; Significant: Too general to make specific contribution.
Revision attempt 1: “How does deep learning improve ASR?” - Progress: more focused, but still lacks specificity about what aspect of improvement, for whom, compared to what.
Revision attempt 2: “How does end-to-end deep learning compare to traditional hybrid systems for ASR accuracy with accented speech?” - Better: specific approaches, outcome, population; answerable; addresses bias concerns; empirically testable. However, significance could be stronger.
Final revision: “How do end-to-end deep learning systems compare to traditional hybrid approaches in ASR accuracy for non-native English speakers, and what do performance differences reveal about bias in training data?” - Clear scope; established evaluation methods; addresses critical fairness issue; quantitative comparison possible; reveals both performance gaps and underlying causes.
Quick reference checklist
Focused:
- Can explain scope in 2-3 sentences
- Key terms clearly defined
- Boundaries explicit (what’s in/out)
- Scope manageable for thesis
Answerable:
- Know what data needed
- Methods exist to investigate
- Realistic access to resources
- Feasible within timeline
- No insurmountable ethical barriers
Relevant:
- Clear stakeholders would care about results
- Connects to current research or practice needs
- Results would inform future work
- Addresses a real problem or gap
Empirical:
- Key concepts can be observed/measured
- Can be investigated through systematic data collection
- Different possible answers are distinguishable
- Tests facts/observations; values/opinions can’t be tested
Significant:
- Advances knowledge or practice
- Makes meaningful contribution to the field
- Opens new questions or opportunities
- Results would matter regardless of direction
Preparation for Class 2
Before class:
- Draft three different versions of potential RQs
- For each version, briefly self-evaluate using FARES (which criteria does it meet?)
- Identify your strongest version and bring printed copies for peer review
During RQ clinic (Activity 1):
- Present your strongest question and what worries you about it (1 min)
- Receive structured FARES-based feedback (5 min)
- Take notes on suggestions without defending your current version
During question refinement lab (Activity 2):
- Synthesize feedback received
- Systematically evaluate your question against all five FARES criteria
- Write three revised versions addressing weaknesses (scope, significance, method)
- Select your strongest revision with clear justification
After class: Continue refining your question based on peer feedback, your FARES self-evaluation, and connections you discover during literature review. Remember: RQs evolve. The question you have today will likely be refined further as you engage more deeply with the literature and methodology planning. That’s expected and healthy.
Additional resources
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications.
Hulley, S. B., et al. (2013). Designing clinical research (4th ed.). Lippincott Williams & Wilkins. [Source of FINER framework]
Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review. SAGE Publications.
Prepared for Research Design (MSc Speech Technology), University of Groningen