TikTok

Requirements

• Define the vision, strategy, and roadmap for AI-driven ACE governance
• Partner with Engineering, Data Science, and Operations to deliver scalable systems end-to-end
• Establish robust offline + online evaluation and experimentation to drive iteration
• Own key metrics (e.g. Risk leak rate, false positives, user negative feedback rate etc.)
• Drive cross-functional alignment across ads platforms, integrity, legal/compliance (as needed), and global stakeholders

Benefits

• 401k Matching
• Health Insurance
• Paid Time Off
• Flexible Hours
• Remote Work
• Stock Options

Skills

Requirements

• Proficiency in frameworks for auditing models
• Conduct audits on the model lifecycle from training through deployment
• Identify model vulnerabilities including bias, fairness violations, harmful hallucinations, security risks
• Recommend remediation strategies
• Develop and maintain collaborative working relationships with stakeholders

Benefits

• 401k Matching
• Health Insurance
• Paid Time Off
• Remote Work
• Stock Options

Skills

Requirements

• - Proficiency in frameworks for auditing models, including criteria like robustness, fairness, interpretability, alignment, and compliance.
• - Model Evaluation & Audit Frameworks: conduct audits on the model lifecycle from training through deployment and monitoring, ensuring compliance with quality, performance, fairness, and risk-management standards.
• - Risk Identification & Mitigation: Identify model vulnerabilities including bias, fairness violations, harmful hallucinations, security risks, and recommend remediation strategies.
• - Measurement Metrics & Statistical Validation: Define and assess model performance metrics (accuracy, precision/recall, F1, calibration, robustness, fairness metrics), measurement of hallucination rates in LLMs, bias/fairness quantification, confidence scoring, and stability analyses.
• - Communication & Collaboration: Develop and maintain collaborative working relationships with stakeholders, including data partners and owners across different business verticals.

Benefits

• 401k Matching
• Health Insurance
• Paid Time Off
• Remote Work

Skills

Requirements

• Proficiency in frameworks for auditing models
• Conduct audits on the model lifecycle from training through deployment
• Identify model vulnerabilities including bias, fairness violations, harmful hallucinations, security risks
• Recommend remediation strategies
• Develop and maintain collaborative working relationships with stakeholders

Benefits

• 401k Matching
• Health Insurance
• Paid Time Off
• Remote Work

Skills

Requirements

• Own and evolve the GenAI / LLM safety evaluation framework
• Design and scale online safety evaluation systems
• Build and maintain safety benchmarks
• Establish and continuously improve root cause analysis processes
• Partner cross-functionally with Model, Algorithm, Product, Data, and Safety teams

Skills