AI Evaluation Engineer — Agentic Systems (H/F)

Skills

Benefits

• Equity
• Parental Leave

Skills

Skills

Requirements

• Operationalizing and scaling Phaidra's solutions for enterprise data center customers
• Working closely with Technical Program Managers and Data Scientists
• Using Phaidra's platform and tools to put AI into production
• Creating a model of the facility for ingestion by our product lineup
• Commissioning and tuning our products according to facility and customer requirements
• Configuring our products according to facility and customer requirements
• Understanding the facility and configuring our products according to facility and customer requirements
• Defining what the AI agents should do and configuring them without writing code

Benefits

• 401k Matching
• Certification Support
• Flexible Hours
• Health Insurance
• Home Office Budget
• Learning Budget
• Remote Work
• Stock Options

Skills

Requirements

• Perform and support fintech and embedded banking security assessments
• Maintain audit-ready workpapers and control documentation
• Assess information security, technology, third-party, and AI-related risks
• Support audits, regulatory examinations, and applicable governance reporting
• Serve as an Information Security liaison for the Bank's fintech and embedded banking initiatives
• Provide security and control-assurance recommendations
• Bring banking or financial services experience in auditing, information technology, information security, cybersecurity, technology risk, fintech, or GRC
• Demonstrate an AI-first working style

Benefits

• 401k Matching
• Health Insurance
• Paid Time Off
• Performance Bonus
• Remote Work
• Retirement Plan
• Vision Insurance

Skills

Requirements

• Develop AI tools alongside Citizen Developers and Technology
• Determine the right tool to solve the problem
• Run scenario simulations and red-team exercises
• Establish best practices and standards for the review and deployment of AI tools
• Assist with coordination of the AI Controls Council
• Produce quarterly AI Controls Intelligence Report for senior leadership
• Co-sign every tool submission before it is approved for deployment to colleagues
• Identify risks sooner and work with business to solve problems

Benefits

• 401k Matching
• Certification Support
• Dental Insurance
• Disability Insurance
• Employee Assistance Program
• Employee Discounts
• Flexible Hours
• Gym Membership

Skills

Requirements

• Experience in risk or control management within financial services, with a focus on compliance and operational risk mitigation, equivalent to seven or more years of experience.
• Technical AI fluency — a genuine understanding of how AI models work, including their assumptions, limitations, and failures.
• Determine the right tool to solve the problem from a cost-effective mindset across AI and general automation tools.
• Run scenario simulations and red-team exercises; establish best practices and standards for the review and deployment of AI tools across the AI Controls Enablement team.
• Assist with coordination of the AI Controls Council, producing quarterly AI Controls Intelligence Report for senior leadership.
• Co-sign every tool submission before it is approved for deployment to colleagues.
• Develop AI tools alongside Citizen Developers and Technology, optimizing control frameworks, identifying risks sooner and working with business to solve problems.

Benefits

• 401k Matching
• Certification Support
• Dental Insurance
• Disability Insurance
• Employee Assistance Program
• Employee Discounts
• Flexible Hours
• Gym Membership

Skills

Requirements

• Support the operation, optimisation, and continuous improvement of the organisation’s Data & AI Governance landscape.
• Maintain and evolve governance processes supporting AI use cases, AI models, AI agents, data products, and associated business processes.
• Maintain and monitor the connections between AI use cases, AI models, AI agents, data, data processing activities, AI applications, and business processes.
• Design and support platform integrations across governance, architecture, and compliance ecosystems.
• Contribute to the management and integration of platforms such as Collibra, LeanIX, OneTrust, Atlan, and related governance tools.
• Define and document enterprise architecture, data architecture, and integration patterns.
• Support metadata management, data lineage, data quality, and associated business processes.

Benefits

• Flexible Hours
• Health Insurance
• Home Office Budget
• Learning Budget
• Paid Time Off
• Remote Work
• Remote Work Stipend
• Training Budget

Skills

Requirements

• Leading a large scale cross-organisational change program in a tech or preferably data field
• Demonstrated experience in leading a cultural shift at an enterprise level
• Technical knowledge in Data & AI Governance Standards & Frameworks
• Data Ownership & Stewardship, Risk Management, Regulatory Change, Business Operations and various Data Governance capabilities
• Ability to bring structure to complex transformations, embed data literacy across the company, and act as a trusted partner for (S)VP-level stakeholders
• Ability to support to embed change into the team's processes, and are the change expert for the Business Unit

Benefits

• Flexible Hours
• Gym Membership
• Health Insurance
• Home Office Budget
• Learning Budget
• Paid Time Off
• Remote Work
• Wellness Programs

Skills

Requirements

• Master's Degree or PhD in a quantitative discipline
• 5–7 years of relevant experience in model validation, model development, or quantitative risk management
• Strong understanding of model risk management principles and applicable regulatory guidance
• Possess a strong analytical background with a solid theoretical foundation
• Experience of working with PD / LGD / EAD models
• Complex quantitative modelling, numerical analysis, and computational methods
• Programming languages (such as Python, R, C++, Java, MATLAB, SAS)
• Mathematical/statistical software packages

Skills

Requirements

• PhD in Computer Science, Machine Learning, Artificial Intelligence, Mathematics, Physics, or a related STEM discipline, or equivalent practical experience
• 1 year of experience initiating, scoping, and executing research in AI safety, alignment, or mechanistic interpretability
• Experience evaluating, steering, red teaming, or implementing empirical safeguards on Large Language Models (LLMs) or frontier systems
• Experience in Python or C++, deep learning frameworks (e.g., JAX, PyTorch), and agentic workflows or tools
• First-author publications at machine learning conferences (e.g., NeurIPS, ICML, ICLR, or COLM)
• Experience in interdisciplinary research bridging cognitive science, computer science, machine learning, and technological ethics to address systemic AI risks
• Deep expertise in post-training alignment, reinforcement learning, reasoning models, and agentic or mechanistic interpretability to diagnose machine behaviors
• Demonstrated track record of translating theoretical alignment and safety findings into scalable, production-grade mitigations on live frontier models

Benefits

• 401k Matching
• Health Insurance
• Flexible Hours
• Paid Time Off
• Stock Options
• Training Budget

Skills

Requirements

• Support organisations to operationalise trustworthy AI principles across the AI lifecycle.
• Help design accountability structures, policies, standards and control frameworks that enable the safe development, deployment and use of AI systems.
• Help organisations understand whether their AI systems are performing as intended and operating within acceptable risk thresholds.
• Support the design and execution of evaluation, testing and validation activities covering model performance, bias, explainability, robustness and other key trust considerations.
• Help organisations identify and manage AI-related risks while enabling innovation and business value.
• Support clients to build practical governance frameworks, operating models, accountability structures and decision-making processes that promote responsible AI adoption.
• Work with clients to assess AI use cases, model types and assess AI-related risks.

Skills

Skills

Skills

Skills

Skills

Skills

Skills

Skills

Skills

Benefits

• Parental Leave

Skills

Skills

Benefits

• Equity

Skills

Skills

Benefits

• Equity

Skills

Benefits

• Life Insurance

Skills

Skills

Benefits

• Disability Insurance
• Equity
• Health Insurance
• Paid Time Off
• Parental Leave

Skills

Benefits

• Unlimited PTO
• Vision Insurance

Skills

Skills

Benefits

• Equity
• Parental Leave

Skills

Skills

Skills

Skills

Skills

Skills

Skills

Skills

Benefits

• Life Insurance
• Paid Time Off

Skills

Benefits

• Equity

Skills