Remote Opportunity

Applied AI Evaluation Engineer

Join BMO as a senior professional working remotely from Canada. Explore the role, benefits, and apply in one place.

Full Time
CAD 82.8k - CAD 154.8k
3 weeks ago
Canada
Worldwide
AI Governance & Programs
Senior
Python
Java
PyTorch
+5 more

Job Description

Application Deadline:

03/30/2026

Address:

100 King Street West

Job Family Group:

Data Analytics & Reporting

Hybrid role

About the Team

BMO’s Applied AI team sets the standard for safe, high-performing AI systems across the bank, building BMO’s leading-edge AI products and capabilities. Our work enables BMO’s strategic outcomes, bridging business needs with scalable and responsible AI. If you thrive on technical challenges and want to make a strategic impact, this role is for you.

 

About the Role

As an Engineering & Data Science Specialist in Applied AI Evaluations, you will design, implement, and optimize evaluation frameworks and pipelines that measure the quality, safety, and performance of advanced AI systems at BMO. You’ll work closely with the Product Owner and collaborate with engineering, research, and business teams to translate evaluation standards into robust, scalable solutions.

Your work will directly impact model development, deployment, and continuous improvement cycles—ensuring that evaluation signals are deeply integrated into experimental and production systems. You’ll contribute to the development of reusable tools, agents, and harnesses that support responsible AI practices and regulatory compliance.

 

 

Key Responsibilities

  • Develop, implement, and maintain evaluation pipelines and harnesses for AI models, ensuring reliability, reproducibility, and scalability for banking applications.
  • Collaborate with the Product Owner to translate evaluation standards and business requirements into technical solutions.
  • Prototype and validate evaluation methods using real banking workflows, integrating feedback into model training and deployment.
  • Integrate evaluation metrics and signals into research, training, and production systems to drive measurable improvements in model performance and customer outcomes.
  • Partner with engineering, research, and product teams to shape model interaction paradigms and deployment strategies.
  • Build reusable tools and systems that enable contributions from across BMO and raise the quality bar for AI solutions.
  • Ensure all evaluation frameworks adhere to regulatory, compliance, and Responsible AI requirements.
  • Document processes, share knowledge, and contribute to a culture of continuous learning and responsible innovation.

 

Qualifications

  • 7+ years of experience in engineering or data science roles, preferably most recent 3 years in AI, machine learning, or data-driven product environments.
  • Strong programming skills (e.g., Python, Java, or similar) and experience with ML/AI frameworks (e.g., PyTorch, TensorFlow).
  • Experience building and maintaining evaluation pipelines for large language models (LLMs) and/or other AI systems, including multi-agent workflows, tool use, or long-context scenarios.
  • Solid understanding of deep learning concepts, model training, and deployment practices.
  • Familiarity with MLOps, CI/CD, and cloud-based AI infrastructure is a plus.
  • Experience driving initiatives as a leader
  • Strong analytical and problem-solving skills, with the ability to communicate technical concepts to both technical and non-technical audiences.
  • Motivated by high-impact collaboration and able to thrive in fast-paced, ambiguous environments.
  • Commitment to BMO’s values of inclusion, integrity, and responsible innovation.

 

Salary:

$82,800.00 - $154,800.00

Pay Type:

Salaried

The above represents BMO Financial Group’s pay range and type.

Salaries will vary based on factors such as location, skills, experience, education, and qualifications for the role, and may include a commission structure. Salaries for part-time roles will be pro-rated based on number of hours regularly worked. For commission roles, the salary listed above represents BMO Financial Group’s expected target for the first year in this position.

BMO Financial Group’s total compensation package will vary based on the pay type of the position and may include performance-based incentives, discretionary bonuses, as well as other perks and rewards. BMO also offers health insurance, tuition reimbursement, accident and life insurance, and retirement savings plans. To view more details of our benefits, please visit: https://jobs.bmo.com/global/en/Total-Rewards

About Us

At BMO we are driven by a shared Purpose: Boldly Grow the Good in business and life. It calls on us to create lasting, positive change for our customers, our communities and our people. By working together, innovating and pushing boundaries, we transform lives and businesses, and power economic growth around the world.

As a member of the BMO team you are valued, respected and heard, and you have more ways to grow and make an impact. We strive to help you make an impact from day one – for yourself and our customers. We’ll support you with the tools and resources you need to reach new milestones, as you help our customers reach theirs. From in-depth training and coaching, to manager support and network-building opportunities, we’ll help you gain valuable experience, and broaden your skillset.

To find out more visit us at https://jobs.bmo.com/ca/en.

BMO is committed to an inclusive, equitable and accessible workplace. By learning from each other’s differences, we gain strength through our people and our perspectives. Accommodations are available on request for candidates taking part in all aspects of the selection process. To request accommodation, please contact your recruiter.

Note to Recruiters: BMO does not accept unsolicited resumes from any source other than directly from a candidate. Any unsolicited resumes sent to BMO, directly or indirectly, will be considered BMO property. BMO will not pay a fee for any placement resulting from the receipt of an unsolicited resume. A recruiting agency must first have a valid, written and fully executed agency agreement contract for service to submit resumes.

Requirements

  • 7+ years of experience in engineering or data science roles
  • Strong programming skills (e.g., Python, Java, or similar)
  • Experience building and maintaining evaluation pipelines for large language models (LLMs) and/or other AI systems
  • Solid understanding of deep learning concepts, model training, and deployment practices
  • Familiarity with MLOps, CI/CD, and cloud-based AI infrastructure is a plus
  • Experience driving initiatives and collaborating with cross-functional teams

Benefits

  • 401k Matching
  • Certification Support
  • Flexible Hours
  • Health Insurance
  • Home Office Budget
  • Learning Budget
  • Paid Time Off
  • Remote Work

Skills

Python
Java
PyTorch
TensorFlow
MLOps
CI/CD
ML/AI frameworks
Cloud-based AI infrastructure

Ready to Apply?

Join BMO today

Salary Range
CAD 82.8k - CAD 154.8k
Posted 3 weeks ago

More AI Governance & Programs roles you might like

Discover similar opportunities from companies that are also hiring remotely.

Full Time
$9149.346k - $1372.402k
10 hours ago
Worldwide
AI Governance & Programs
Senior
Python
Microsoft Entra ID
Azure AD
+5 more
Full Time
$225k - $280k
14 hours ago
Worldwide
AI Governance & Programs
Senior
Python
Data Science
Machine Learning
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Generative AI
+5 more

Explore more remote openings

Browse fresh listings from our global community of remote-friendly teams.

Full Time
$120,000 - $180,000*
4 hours ago
United Kingdom
Worldwide
Model Risk Management & Validation
Senior
Solvency II
Internal Model
Model Risk
+4 more
Full Time
$120,000 - $180,000*
14 hours ago
Worldwide
AI Security & Privacy
Senior
AI/ML
Security Architecture
Cloud Security
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Canada
Worldwide
Model Risk Management & Validation
Senior
Python
SAS
SQL
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Governance & Programs
Senior
Copilot Studio
Power Automate
Power Apps
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Ireland
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Data Science
+4 more
Full Time
$120,000 - $180,000*
1 day ago
United States
Worldwide
AI Governance & Programs
Senior
Model Risk Management
AI Governance
Compliance
+4 more
Full Time
CAD 94.6k - CAD 176k
1 day ago
Canada
Worldwide
AI Risk & Controls
Senior
Finance
Model Validation
Risk Management
+3 more
Full Time
$106.23k - $145k
1 day ago
Worldwide
AI Governance & Programs
Senior
AI/ML
Cloud Security
Data Science
+5 more
Full Time
$148k - $274.2k
1 day ago
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Data Science
+3 more
Full Time
$228.911k - $471.286k
1 day ago
Worldwide
AI Compliance & Legal
Senior
AI
Machine Learning
Data
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Security & Privacy
Senior
Python
Adversarial Machine Learning
Enterprise Security Architecture
+5 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Security & Privacy
Senior
Python
Adversarial Machine Learning
Enterprise Security Architecture
+5 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Governance & Programs
Senior
Python
Node.JS
Go
+3 more
Full Time
$147k - $211k
1 day ago
Worldwide
AI Governance & Programs
Senior
C++
API design
Stubby
+3 more
Full Time
$80,000 - $140,000*
1 day ago
Worldwide
AI Risk & Controls
Mid
Data Analysis
Risk Assessment
Automation
+2 more
Full Time
$120,000 - $200,000*
1 day ago
Worldwide
AI Risk & Controls
Senior
Python
Django
Kubernetes
+5 more
Full Time
$98.16k - $159.27k
2 days ago
United States
AI Security & Privacy
Senior
Azure Security Engineer
Microsoft Cybersecurity Architect
CISSP
+4 more
Full Time
$108.75k - $200k
2 days ago
United States
Model Risk Management & Validation
Senior
Quantitative models
Risk Management
Model Risk Management
+4 more
Full Time
$108k - $185k
2 days ago
United States
Worldwide
AI Compliance & Legal
Senior
Artificial Intelligence
Generative AI
Machine Learning
+3 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Governance & Programs
Senior
Generative AI
LLMs
prompt engineering
+5 more
Full Time
$229.9k - $262.4k
2 days ago
United States
Worldwide
AI Governance & Programs
Senior
Python
Machine Learning
Cloud Computing
+3 more
Full Time
$120,000 - $180,000*
2 days ago
Canada
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Risk Management
+3 more
Full Time
$80,000 - $140,000*
2 days ago
Germany
Worldwide
Model Risk Management & Validation
Mid
Model Validation
Risk Management
Model Development
+5 more
Full Time
$100,000 - $150,000*
2 days ago
Worldwide
AI Governance & Programs
Senior
AI
Data governance
Regulatory Compliance
+5 more
Full Time
$120,000 - $180,000*
2 days ago
United Kingdom
Worldwide
AI Governance & Programs
Senior
Cloud
AI
API
+3 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Governance & Programs
Senior
Python
Machine Learning
Data Analysis
+4 more
Full Time
$108k - $185k
2 days ago
United States
Worldwide
AI Compliance & Legal
Senior
Artificial Intelligence
Generative AI
Machine Learning
+3 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Governance & Programs
Senior
Data security & privacy
Access control and permissions
API integrations
+5 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Governance & Programs
Senior
AWS
Azure
GCP
+5 more
Contract
$120,000 - $180,000*
4 days ago
Worldwide
AI Governance & Programs
Senior
ISO/IEC 42001
AI Lifecycle Governance
Data governance
+4 more
Contract
$120,000 - $180,000*
4 days ago
Worldwide
AI Governance & Programs
Senior
ISO/IEC 42001
Audit Methodology
Artificial Intelligence Management Systems
+1 more
Full Time
$102.8k - $210.2k
4 days ago
Worldwide
AI Compliance & Legal
Senior
Data Analysis
Business Analysis
Data Management
+5 more
Full Time
$197.4k - $246.75k
4 days ago
Worldwide
AI Governance & Programs
Senior
Machine Learning
Generative AI
Python
+5 more
Full Time
$149.084k - $218.657k
4 days ago
Worldwide
AI Security & Privacy
Senior
AI/ML systems
Governance
Model Risk Management
+5 more
Full Time
$80,000 - $160,000*
4 days ago
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Data Science
+5 more
Full Time
$80,000 - $160,000*
4 days ago
Worldwide
AI Governance & Programs
Senior
Pharmacy
Retail Operations
Customer Service
+5 more