Research Design

Content › Frameworks

The FARES framework for evaluating research questions

What is FARES?

FARES is a framework for evaluating research question quality we use in this course. A strong RQ should be:

The 5 FARES criteria

1/ Focused

Your research question should have clear scope with defined boundaries.

Considerations:

Common problems: Questions contain multiple distinct sub-questions; scope could encompass dozens of studies; key terms are vague; no clear boundaries on population or context.

Ex transformation:

Too broad: “How can we improve speech recognition?”

Well-focused: “How does background noise affect keyword spotting accuracy for elderly users of voice assistants?”

2/ Answerable

Your research question should be feasible to investigate with available methods and resources.

Considerations:

Common problems: Requires impossible-to-obtain data; would take years to complete; depends on non-existent technology; requires unavailable expertise; has unresolvable ethical barriers.

Example transformation:

Unanswerable: “What will speech technology look like in 2050?”

Answerable: “What acoustic features best predict listener-perceived naturalness in current neural text-to-speech systems?”

3/ Relevant

Your research question should address real needs in research or applications.

Considerations:

Common problems: Results only of interest to the researcher; problem is purely hypothetical; question thoroughly answered already; topic disconnected from field priorities; answering wouldn’t advance knowledge or practice.

Example transformation:

Low relevance: “How does voice assistant usage vary between Tuesday and Wednesday?”

High relevance: “How does accent bias in voice assistants affect user trust across different cultural communities?”

4/ Empirical

Your RQ should be investigable through systematic observation or experimentation.

Key considerations:

Common problems: Question is purely philosophical or normative; cannot be tested through data; key concepts cannot be measured; relies entirely on speculation.

Example transformation:

Non-empirical: “Should AI have emotions?”

Empirical: “Do listeners perceive emotional expressions differently in synthetic versus natural speech?”

5/ Significant

Your RQ should lead to results that contribute meaningfully to knowledge or practice.

Key considerations:

Common problems: Only confirms well-established knowledge; incremental improvement too small to matter; technical change with no clear benefit; answers a question no one is asking; purely descriptive with no implications.

Example transformation:

Low significance: “What happens if we change parameter X from 0.7 to 0.8 in algorithm Y?”

High significance: “How do prosodic features contribute to perceived speaker identity in neural voice cloning, and what implications does this have for ethical voice synthesis?”

Using FARES in practice

Self-evaluation process

Work through each criterion systematically:

  1. Write out your current RQ
  2. For each FARES criterion: Rate your question (strong, needs work, weak), list specific concerns or strengths, identify what would need to change to improve
  3. Prioritize which 1-2 criteria need the most attention
  4. Revise: Generate alternative framings that address weaknesses
  5. Re-evaluate: Does your revised question perform better on FARES?

Peer review using FARES

When providing feedback on a colleague’s RQ:

Common pitfalls and solutions

Relationship to other frameworks

FINER framework (Hulley et al., 2013)

The FINER framework evaluates RQs as Feasible, Interesting, Novel, Ethical, and Relevant.

Source: Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013). Designing clinical research (4th ed.). Lippincott Williams & Wilkins.

FARES adapts FINER by: replacing “Feasible” with “Answerable” (emphasizing methodological clarity); adding “Focused” (addressing scope definition critical for graduate work); adding “Empirical” (ensuring questions are testable through data); replacing “Novel” and “Interesting” with “Significant” (focusing on contribution over novelty per se); integrating “Ethical” considerations into “Relevant” and “Answerable” criteria.

PICO framework

Commonly used in health sciences: Population, Intervention, Comparison, Outcome. PICO helps structure what you’re asking about. FARES evaluates whether what you’re asking is a good RQ.

Examples across speech technology domains

Automatic speech recognition

Weak: “How can we make ASR better?” - Fails: focused, answerable, significant (too vague)

Strong: “How does code-switching frequency affect recognition accuracy in Spanish-English bilingual speech, and what does this reveal about current ASR training limitations?” - Focused on specific language pair and phenomenon; answerable through corpus analysis; addresses real multilingual user challenges; testable empirically; informs training approaches for multilingual systems.

Speech synthesis

Weak: “Can we make TTS sound natural?” - Fails: focused (what aspect?), answerable (how to measure?), significant (already well-studied)

Strong: “What prosodic features contribute most to perceived speaker identity in neural voice cloning, and how do these perceptions vary across listener demographics?” - Focused on specific features and perceptual dimensions; answerable through perceptual experiments; critical for identity preservation and ethics; empirically testable; informs both technical development and ethical guidelines.

Affective computing

Weak: “Should AI express emotions?” - Fails: empirical (normative question), answerable (requires philosophical argument)

Strong: “Do listeners perceive emotional authenticity differently when they know speech is synthesized versus natural, and how does disclosure timing affect these perceptions?” - Focused on specific perceptual dimension; answerable through experimental manipulation; critical for transparent AI interaction; empirically testable; informs both ethics and design.

Revising your RQ: a worked example

Initial question: “How does machine learning affect speech technology?”

FARES analysis: Focused: Far too broad; Answerable: Impossible to address comprehensively; Relevant: Generally important but unfocused; Empirical: Could be made empirical but needs specification; Significant: Too general to make specific contribution.

Revision attempt 1: “How does deep learning improve ASR?” - Progress: more focused, but still lacks specificity about what aspect of improvement, for whom, compared to what.

Revision attempt 2: “How does end-to-end deep learning compare to traditional hybrid systems for ASR accuracy with accented speech?” - Better: specific approaches, outcome, population; answerable; addresses bias concerns; empirically testable. However, significance could be stronger.

Final revision: “How do end-to-end deep learning systems compare to traditional hybrid approaches in ASR accuracy for non-native English speakers, and what do performance differences reveal about bias in training data?” - Clear scope; established evaluation methods; addresses critical fairness issue; quantitative comparison possible; reveals both performance gaps and underlying causes.

Quick reference checklist

Focused:

Answerable:

Relevant:

Empirical:

Significant:

Preparation for Class 2

Before class:

During RQ clinic (Activity 1):

During question refinement lab (Activity 2):

After class: Continue refining your question based on peer feedback, your FARES self-evaluation, and connections you discover during literature review. Remember: RQs evolve. The question you have today will likely be refined further as you engage more deeply with the literature and methodology planning. That’s expected and healthy.

Additional resources

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications.

Hulley, S. B., et al. (2013). Designing clinical research (4th ed.). Lippincott Williams & Wilkins. [Source of FINER framework]

Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review. SAGE Publications.

Prepared for Research Design (MSc Speech Technology), University of Groningen