Performance / NTD Analyst

Ryan Mahoney

Why this role is hard · Ryan Mahoney

The real challenge is finding someone who can stick to strict federal reporting rules while explaining them clearly to the operations team. At this level, they need to figure out how to combine messy vehicle tracking numbers with financial records on their own, without breaking compliance guidelines. You will see how they handle this when they describe dealing with conflicting departmental requests instead of just agreeing to everything. Too many candidates either hide behind their spreadsheets or promise faster results than they can actually deliver. You want the person who can justify a statistical adjustment and still hand over a clean audit trail.

Core Evaluation

Critical questions for this role

The competency and attitude questions below are where the hiring decision is made. They run in the live interview rounds and are calibrated to the level selected above.

17 Competency Questions

1 of 17

  1. Discipline

    Transit Data Governance & Compliance Analytics

  2. Job requirement

    Data Privacy & Audit Compliance

    Conducts privacy impact assessments, maintains audit trails, and implements data retention policies.

  3. Expected at Mid

    Important for secure data handling, but typically supported by dedicated compliance functions at this level, making it a strong growth competency.

Interview round: Hiring Manager Technical Deep Dive

Walk me through how you managed privacy reviews and maintained audit trails for a dataset containing sensitive operational or rider information.

Positive indicators

  • Details PII identification and handling methods.
  • Explains audit trail structure clearly.
  • References alignment with retention policies.
  • Mentions access control documentation.
  • Ensures logs are retrievable and secure.

Negative indicators

  • Stores sensitive data without protection.
  • Lacks access logging or audit trails.
  • Ignores retention policy requirements.
  • Fails to document privacy assessments.
  • Uses ad-hoc security measures.

10 Attitude Questions

1 of 10

Active Listening

The deliberate, focused practice of fully receiving, processing, and comprehending verbal and non-verbal information from stakeholders, with the explicit goal of accurately translating qualitative operational realities, frontline feedback, and conflicting narratives into precise quantitative metrics, validation baselines, and unified reporting frameworks.

Interview round: Recruiter Screening

If finance and operations provide conflicting explanations for a sudden drop in revenue-hours on a specific route, how would you approach gathering the necessary details to resolve the discrepancy?

Positive indicators

  • Requests specific logs from both departments
  • Schedules cross-departmental alignment sessions
  • Maps narratives to timestamped system data
  • Documents resolution methodology clearly
  • Prioritizes empirical evidence over opinions

Negative indicators

  • Accepts the first department's explanation without verification
  • Escalates immediately without independent investigation
  • Ignores documentation requests from stakeholders
  • Uses vague reconciliation methods
  • Fails to document the decision rationale

Supporting Evaluation

How candidates earn the selection conversation

The goal is to reduce effort for everyone by collecting more useful signal before adding more interviews. Lightweight application prompts and structured screens help the panel focus live time on the candidates most likely to succeed.

Stage 1 · Application

Filter at the door

Runs the moment a candidate hits Submit. Disqualifying answers end the application; everything else is captured for review.

Knock-out Questions

1 of 2

Application Screen: Knock-out

Have you directly prepared, validated, or audited FTA National Transit Database (NTD) reports or federal transit compliance submissions within the past 5 years?

Yes
Qualifies
No
Auto-decline

Video-Response Questions

1 of 3

Application Screen: Video Response

You discover that revised revenue-mile calculations conflict with dispatch logs, causing a 15% variance in monthly reporting. How would you explain this discrepancy and propose a reconciliation timeline to the scheduling team, who are unfamiliar with NTD compliance requirements?

Candidate experience

REC
0:42 / 2:00
1Record
2Review
3Submit

Response time

2 min

Format

Recorded video

Stage 2 · Resume Screening

Read the resume against fixed criteria

Reviewers score every application that clears the door against the same criteria. Stronger reviews advance to live interviews; weaker ones are archived without further screening.

Resume Review Criteria

8 criteria
Evidence of independently applying validated sampling ratios, calculating confidence intervals, and merging datasets across fixed-route, paratransit, or on-demand modes.
Evidence of maintaining technical records, conducting mock audits, and documenting data transformations to withstand regulatory scrutiny.
Evidence of investigating systemic data anomalies, defending metric adjustments, and aligning operational narratives with financial or scheduling records.
Evidence of building SQL/ETL workflows to reduce manual reconciliation, integrating telematics or charging data, and delivering actionable performance narratives.

Is the resume complete, well-organized, and free from formatting, spelling, and grammar mistakes?

Does the cover letter or personal statement convey clear relevance and familiarity with the job?

Does the resume show relevant prior work experience?

Does the resume indicate required academic credentials, relevant certifications, or necessary training?

Stage 3 · During Interviews

Where the hire is decided

Interview rounds use the competency and attitude questions outlined above, then add tests, work simulations, and presentations that reveal deeper evidence about how the candidate thinks and works.

Presentation Prompt

Walk us through a past project where you designed or implemented a sampling plan for passenger load or ridership estimation. Discuss your sampling methodology, how you reconciled conflicting operational narratives, and how you defended your confidence intervals to non-technical stakeholders.

Format

deck-and-walkthrough · 20 min · ~2 hr prep

Audience

Analytics Manager and Compliance Lead

What to prepare

  • 3-5 slides summarizing the project context, methodological choices, and outcomes
  • Notes on stakeholder pushback and how you adapted your communication

Deliverables

  • A 20-minute deck-led walkthrough with 5 minutes reserved for Q&A

Ground rules

  • Use only work you are permitted to share. If past work is confidential, anonymize agency names and redact sensitive metrics while preserving the analytical structure.
  • Do not build new models or datasets; focus on explaining your existing approach.

Scoring anchors

Exceeds
Presents a robust, well-defended sampling methodology, seamlessly translates statistical concepts for cross-functional alignment, and demonstrates mature stakeholder management under tight deadlines.
Meets
Clearly explains the sampling approach, identifies key tradeoffs, and communicates confidence intervals in an accessible way with minor prompting.
Below
Struggles to justify methodological choices, uses opaque statistical language, or cannot articulate how operational feedback was integrated into the model.

Response time

20 min

Positive indicators

  • Articulates the rationale behind stratification or sample size calculations clearly
  • Translates statistical uncertainty into actionable guidance for operational teams
  • Demonstrates how they balanced methodological rigor with practical field constraints
  • Sets clear boundaries on data validation timelines when facing stakeholder pressure

Negative indicators

  • Relies heavily on technical jargon without explaining its relevance to business outcomes
  • Jumps directly to results without explaining the methodological tradeoffs or sampling design
  • Fails to address how conflicting operational data was reconciled into the final estimate
  • Defends methodology rigidly without acknowledging real-world sampling limitations

Work Simulation Scenario

Scenario. You are calculating annual unlinked passenger trips for a newly launched Bus Rapid Transit (BRT) line. Initial sampling ratios show high variance, and dispatch reports that APC sensors were frequently disabled during peak hours due to 'system glitches.' You must design an approach to reconcile the sampling data, adjust for the missing counts, and defend your methodology to operations leadership.

Problem to solve. Develop a statistically sound reconciliation and adjustment strategy that maintains NTD compliance while accounting for known hardware failures.

Format

discovery-interview · 35 min · ~2 hr prep

Success criteria

  • Probe for APC failure patterns and calibration logs to quantify missing data.
  • Propose a defensible statistical adjustment or imputation method.
  • Clearly communicate uncertainty and margin of error to non-technical stakeholders.

What to review beforehand

  • NTD sampling and unlinked passenger trip calculation guidelines.
  • Basic principles of statistical imputation and confidence intervals.
  • Company data validation protocols for hardware failures.

Ground rules

  • Ask for any operational logs, maintenance records, or historical sampling data you need.
  • Focus on structuring your analytical approach and defending your methodology.
  • Treat this as a live discovery and planning conversation.

Roles in scenario

Operations Data Manager (informed_partner, played by cross_functional)

Motivation. Ensure the reconciliation method is statistically rigorous, accounts for real-world hardware constraints, and can withstand audit scrutiny without requiring costly hardware replacements mid-cycle.

Constraints

  • Cannot authorize budget for new APC hardware until next fiscal year.
  • Will only answer direct questions about dispatch logs and maintenance schedules.
  • Operations leadership expects a finalized trip count within 10 business days.

Tensions to introduce

  • Dispatch insists the 'glitches' were temporary and don't warrant sample exclusion.
  • Finance is concerned that adjusting the sample downward will impact federal funding eligibility.
  • Previous attempts to use proxy data were rejected during mock audits.

In-character guidance

  • Provide honest answers about maintenance logs and sensor calibration history.
  • Push back gently if the candidate's statistical assumptions ignore operational realities.
  • Wait for the candidate to ask for specific failure timestamps or calibration reports.

Do not

  • Do not volunteer the exact APC failure rates or calibration schedules.
  • Do not suggest a specific statistical model or adjustment factor.
  • Do not solve the reconciliation problem or coach the candidate on NTD formulas.

Scoring anchors

Exceeds
Designs a robust, transparent statistical adjustment backed by hardware logs, clearly communicates uncertainty, and aligns methodology with both operational realities and audit standards.
Meets
Requests relevant operational data, proposes a reasonable statistical adjustment, and explains the methodology and uncertainty to stakeholders.
Below
Relies on unvalidated assumptions, fails to account for hardware failure patterns, or cannot clearly communicate statistical limitations.

Response time

35 min

Positive indicators

  • Asks for calibration logs, failure timestamps, and historical maintenance records before modeling.
  • Proposes a transparent imputation or stratified adjustment method with clear confidence intervals.
  • Translates statistical uncertainty into clear operational and funding implications for leadership.
  • Validates assumptions with dispatch before finalizing the methodology.

Negative indicators

  • Guesses adjustment factors without grounding them in maintenance or calibration data.
  • Uses opaque statistical methods and fails to communicate margin of error.
  • Ignores the operational constraints or funding implications of the adjustment.
  • Freezes when confronted with conflicting dispatch and finance priorities.

Progression Framework

This table shows how competencies evolve across experience levels. Each cell shows competency at that level.

Transit Data Governance & Compliance Analytics

4 competencies

CompetencyJuniorMidSeniorPrincipal
Data Privacy & Audit Compliance

Follows established protocols for data anonymization and assists in routine compliance audits.

Conducts privacy impact assessments, maintains audit trails, and implements data retention policies.

Develops comprehensive privacy frameworks, leads audit remediation efforts, and integrates compliance controls into data engineering pipelines.

Defines organizational data ethics and privacy strategy, navigates complex regulatory landscapes, and establishes industry-leading compliance architectures.

Public Data Standards & Open Data Publishing

Prepares datasets for publication by applying formatting guidelines and basic metadata tagging.

Manages open data catalog updates, ensures schema compliance, and coordinates with developers on API data feeds.

Designs data publication workflows, establishes quality gates for public releases, and advocates for interoperability standards across agencies.

Shapes open data policy, leads cross-agency standardization initiatives, and evaluates emerging data-sharing paradigms for public impact.

Regulatory Reporting & Data Validation

Extracts and formats transit data according to established reporting templates and performs basic validation checks under supervision.

Independently manages end-to-end reporting cycles, identifies data anomalies, and implements corrective validation rules across multiple datasets.

Designs automated validation pipelines, mentors junior analysts, and aligns reporting frameworks with evolving regulatory requirements.

Architects enterprise-wide data governance strategies, influences regional/national transit data standards, and ensures long-term compliance scalability.

Statistical Estimation & Performance Metrics

Applies standard statistical methods to calculate ridership and performance metrics using provided analytical tools.

Selects appropriate estimation techniques for complex scenarios, troubleshoots model assumptions, and documents analytical methodologies.

Develops advanced statistical models for predictive performance, integrates disparate data sources, and validates model accuracy against operational outcomes.

Pioneers novel analytical frameworks for system-wide performance benchmarking and drives strategic decision-making through advanced econometric modeling.

Transit Operations & Financial Systems Analytics

4 competencies

CompetencyJuniorMidSeniorPrincipal
Financial Modeling & Cost-Performance Analytics

Compiles financial data and assists in calculating cost-per-mile and revenue recovery metrics.

Develops budget variance analyses, models farebox impacts, and evaluates cost-efficiency of service changes.

Constructs multi-year financial forecasts, performs scenario modeling for funding allocations, and advises on fiscal sustainability.

Drives enterprise financial strategy through advanced ROI modeling, optimizes capital vs. operational trade-offs, and influences regional funding policy.

Fleet Sustainability & Asset Lifecycle Analytics

Tracks maintenance logs and calculates basic fleet utilization and fuel consumption rates.

Analyzes asset condition trends, supports preventive maintenance scheduling, and evaluates electrification transition costs.

Develops lifecycle cost models, optimizes fleet renewal strategies, and integrates telematics data for predictive maintenance.

Sets long-term fleet modernization roadmaps, evaluates emerging propulsion technologies, and aligns asset strategy with sustainability mandates.

Multimodal Integration & Ecosystem Analytics

Aggregates ridership and service data from multiple transit modes into consolidated reports.

Maps transfer patterns, evaluates first/last-mile connectivity, and assesses partnership performance metrics.

Designs integrated mobility dashboards, models network synergy effects, and negotiates data-sharing agreements with microtransit providers.

Defines regional multimodal integration strategy, architects unified mobility data platforms, and leads public-private ecosystem development.

Real-time Operations Monitoring & Dispatch Analytics

Monitors real-time dashboards, logs operational incidents, and generates basic performance reports.

Analyzes real-time telemetry to identify bottlenecks, optimizes dispatch parameters, and supports schedule adjustments.

Builds predictive models for service disruptions, designs real-time alerting systems, and leads operational review sessions.

Architects real-time analytics ecosystems, integrates AI-driven dispatch optimization, and aligns operational tech strategy with capital planning.