UX Researcher

Ryan Mahoney

Ryan Mahoney

Director of Product, FirstWho

“The hard part isn't finding a researcher or an engineer. It is finding someone who can handle both responsibilities without dropping one when pressure mounts. You need a candidate who builds evaluation systems that work with real user data while sticking to strict privacy rules. Most people fail because they treat human feedback like just another dataset instead of considering the person behind it. The right candidate can keep product teams on the same page without losing accuracy and will tell you when the data is too messy to trust.”

Skip the setup

Use as-is, or remix to fit your team.

Start hiring now

Competency Questions

1 of 16

UX Research & Operations

Encompasses the end-to-end research lifecycle from qualitative and quantitative execution to operational governance, compliance, and emerging technology evaluation. At mid-level, emphasis shifts to independent ownership of evaluation streams, pipeline optimization, and cross-functional collaboration with ML teams.

Analysis & Synthesis

Synthesizes findings into coherent narratives and identifies root causes, bridging qualitative insight with scalable data pipelines for actionable product recommendations.

Interview round: Hiring Manager Portfolio Review

You have a large dataset of interview transcripts but limited time. What is your process?

Positive indicators

  • Mentions affinity mapping or tagging
  • Describes sampling strategy
  • Sets clear timeboxes for analysis

Negative indicators

  • Attempts to read every word linearly
  • No clear system for organizing data
  • Misses deadline due to perfectionism

Attitude Questions

1 of 13

Active Listening

The intentional cognitive and behavioral process of fully concentrating on, understanding, and responding to a speaker, characterized by withholding immediate judgment, validating spoken and unspoken cues, and accurately reflecting content to ensure alignment between user realities, stakeholder constraints, and technical possibilities.

Interview round: Recruiter Screen

Walk me through a research session where a participant said something unexpected. How did you handle it?

Positive indicators

  • Describes adapting questions in real-time
  • Mentions noticing participant discomfort or enthusiasm
  • Connects unexpected input to insights

Negative indicators

  • Stick rigidly to script despite participant cues
  • Dismisses unexpected input as off-topic
  • Cannot recall specific unexpected moments

Progression Framework

This table shows how competencies evolve across experience levels. Each cell shows competency at that level.

UX Research & Operations

7 competencies

CompetencyJuniorMidSeniorPrincipal
Analysis & Synthesis

Tags data and identifies basic themes using provided frameworks, processing raw data to surface initial patterns.

Synthesizes findings into coherent narratives and identifies root causes, bridging qualitative insight with scalable data pipelines for actionable product recommendations.

Connects insights across multiple studies to inform product strategy.

Develops new synthesis frameworks and drives industry-level insight generation, creating novel evaluation paradigms for AGI systems.

Data Collection & Method Execution

Moderates simple sessions and collects data using established scripts, ensuring consistency and quality in data gathering.

Moderates complex sessions and adapts scripts in real-time based on participant responses, ensuring data quality across qualitative and quantitative collection methods.

Oversees data collection quality across multiple studies and trains junior researchers.

Establishes best practices for data collection integrity across the organization, ensuring research quality standards for high-stakes AI safety studies.

Domain Adaptation & Emerging Technology

Learns domain-specific terminology and assists in specialized studies, building contextual knowledge for domain-specific research execution.

Executes research in specialized domains using adapted methodologies, applying research methods to AI/ML contexts and evaluating emerging technology impact on UX.

Leads research for emerging technology products and defines domain-specific best practices.

Anticipates industry shifts and defines research strategy for new technology categories, particularly human-AI interaction paradigms.

Ethics, Privacy & Compliance

Follows consent protocols and anonymizes data as instructed, ensuring research activities meet legal and ethical standards.

Reviews study plans for compliance risks and manages consent forms, ensuring research activities meet legal, ethical, and privacy standards for human evaluation frameworks.

Advises product teams on ethical implications of research and data usage.

Sets organizational ethics standards and liaises with legal/compliance teams, providing existential safety and ethics leadership for human-AI research.

Operational Governance

Organizes files and maintains research repositories according to team standards, supporting workflow efficiency and auditability.

Optimizes workflows and manages tool subscriptions for the team, driving pipeline latency reduction and research velocity improvement through automation and repository management.

Designs operational systems to scale research output and reduce redundancy.

Defines organizational research ops strategy and budget allocation, enabling scalable research infrastructure for AGI safety programs.

Reporting & Stakeholder Communication

Creates summary reports and presents findings to immediate team, communicating research outcomes through documented artifacts.

Tailors communication styles for different stakeholders and facilitates workshops, enabling metric adoption by engineering teams and research velocity improvement.

Influences product roadmap through persuasive storytelling and executive presentations.

Establishes communication standards and advocates for research impact at the board level, influencing industry-wide adoption of alignment metrics.

Research Planning & Fundamentals

Assists in drafting research plans and screening participants under guidance, supporting study logistics aligned with product goals.

Independently creates research plans and selects methods for standard product questions, aligning evaluation protocols with product goals and ML team requirements.

Designs complex longitudinal studies and aligns research strategy with business objectives.

Defines organizational research frameworks and mentors others on strategic planning, establishing long-term research direction aligned with AGI alignment goals.