Principal Engineer – AI Security Developer

Overview

Connecting clients to markets – and talent to opportunity.With 5,400+ employees and over 80,000 institutional, commercial, and payments clients, we operate from more than 80 offices spread across six continents. As a Fortune 100, Nasdaq-listed provider, we connect clients to the global markets – focusing on innovation, human connection, and providing world-class products and services to all types of investors.Whether you want to forge a career connecting our retail clients to potential trading opportunities, or ingrain yourself in the world of institutional investing, StoneX Group is made up of four business segments that offer endless potential for progression and growth.

Business Segment Overview

Corporate: Engage in a deep variety of business-critical activities that keep our company running efficiently. From strategic marketing and financial management to human resources and operational oversight, you’ll have the opportunity to optimize processes and implement game-changing policies.

Position Purpose: At StoneX, defending a global financial services organization increasingly means building capabilities ourselves rather than waiting for the market to provide them. As a Principal Engineer – AI Security Developer, you’ll serve as the senior technical authority for how StoneX applies AI to cybersecurity, setting the technical vision, architecture, and engineering standards for our AI security platform while personally building the hardest and highest-leverage parts of it. 

Your mission is to define how AI becomes a secure, scalable, and trusted capability across the security organization. You’ll establish the reference architecture for GenAI applications, AI agents, retrieval-augmented generation (RAG) platforms, advanced analytics, shared services, and the guardrails and engineering patterns that allow multiple teams to build consistently and safely. 

This is a senior individual contributor role with enterprise-wide influence, not a role removed from the technology. You’ll remain a hands-on engineer while shaping technical direction across multiple security and engineering teams. You’ll make the difficult architectural decisions, build shared platform capabilities, prototype new approaches, solve the hardest technical problems, and raise the engineering bar for how AI capabilities are designed and delivered across StoneX. 

From Day 1, you’ll begin shaping the technical direction of StoneX’s AI security capabilities while getting hands-on with the platform and the problems our teams are trying to solve. You’ll work across security, engineering, infrastructure, data, risk, and other partners to identify where AI creates meaningful value, establish the architecture and standards teams can build upon, and help turn that direction into trusted production capabilities at scale.

Responsibilities

Primary duties will include: 

• AI Security Strategy: Define and own the multi-year technical strategy and roadmap for AI-powered security capabilities, translating business and risk priorities into a sequenced engineering direction. 

• Reference Architecture: Design and govern the target architecture for StoneX’s AI security platform, including LLM services and orchestration, agent frameworks, RAG and knowledge pipelines, evaluation infrastructure, data access patterns, and integrations across the security environment. 

• Platform Engineering: Architect and build shared platform capabilities including secure LLM services, agent runtimes, tool and connector frameworks, retrieval services, prompt and model management, and observability that allow teams to deliver AI solutions quickly, consistently, and securely. 

• Hands-On Engineering: Remain deeply technical by personally designing and building complex and critical components, prototyping new approaches, and helping teams solve difficult engineering problems across the full stack. 

• Technical Decision-Making: Lead evaluation of models, frameworks, vendors, and open-source technologies and make recommendations based on security, performance, cost, maintainability, and long-term strategic value. 

• Portfolio Prioritization: Partner with security leadership to identify, size, and prioritize high-impact AI opportunities across detection, response, vulnerability management, identity, cloud, application security, and other security domains while challenging work that does not demonstrate meaningful value. 

• Secure Integration: Establish patterns for integrating AI services and agentic workflows with security technologies including source-code repositories, threat models, cloud platforms, SIEM, SOAR, IAM, CMDB, and vulnerability management systems while enforcing appropriate least privilege, auditability, and human oversight. 

• Production Excellence: Establish expectations for reliability, scalability, latency, monitoring, operational readiness, and cost management across production AI workloads. 

• AI Security & Assurance: Serve as a technical authority on securing AI systems against prompt injection, data leakage, model and supply-chain risk, agent tool misuse, adversarial manipulation, and other emerging threats. 

• Evaluation & Quality: Establish how StoneX evaluates LLM and agent performance through benchmark datasets, automated evaluation pipelines, red teaming, regression testing, and measures of accuracy, hallucination, reliability, and analyst trust. 

• Responsible AI: Partner with Risk, Compliance, Legal, Data Governance, and Architecture to ensure AI capabilities meet regulatory, model risk management, privacy, and data residency expectations. 

• Engineering Standards: Set and champion standards for AI engineering, including architecture, code quality, design review, testing, CI/CD, documentation, and production readiness. 

• Technical Leadership: Influence technical direction across security, infrastructure, data, and application engineering teams, building consensus and resolving complex cross-team technical challenges without relying on direct authority. 

• Developing Engineers: Mentor senior and staff-level engineers, strengthen AI engineering capability across the security organization, and help shape technical hiring standards and career development. 

• Executive Advisory: Serve as a trusted technical advisor to the CISO, security leadership, and executive stakeholders by clearly communicating technical strategy, trade-offs, risks, and measurable outcomes. 

• Industry Leadership: Represent StoneX within the broader security and AI community through industry engagement, working groups, conferences, publications, or open-source contributions and bring relevant external thinking back into the organization. 

A typical week might include the following: 

• Spending at least four days in the office engaging directly with engineers, security teams, architects, and other stakeholders. 

• Designing and building critical components of StoneX’s AI security platform, particularly where the technical complexity or organizational leverage is highest. 

• Working with security leadership and engineering teams to identify and prioritize AI use cases based on measurable security value, feasibility, and strategic importance. 

• Leading architecture and design reviews for AI capabilities that span teams, security domains, or enterprise technologies. 

• Evaluating emerging models, agent frameworks, development approaches, vendors, and open-source technologies against StoneX’s security, performance, cost, and maintainability requirements. 

• Prototyping new AI capabilities to validate difficult technical concepts before establishing reusable patterns that other engineers can build upon. 

• Working with engineering teams to solve difficult integration, architecture, scalability, reliability, or security challenges. 

• Reviewing production AI performance, evaluations, model behavior, observability, reliability, and cost to identify opportunities for improvement. 

• Threat modeling AI systems and ensuring appropriate protections exist around data access, agent actions, tool execution, model behavior, and other AI-specific risks. 

• Mentoring engineers through architecture discussions, design reviews, code reviews, and complex technical challenges. 

• Advising security and executive leadership on AI strategy, technical trade-offs, emerging risks, and the practical implications of changes in AI technology. 

This job might be for you if: 

• You’re a technical leader who still builds – you enjoy setting architecture and direction, but you still want to write production code and personally tackle the problems where your involvement creates the greatest leverage. 

• You think beyond individual applications – you naturally look for the shared platforms, patterns, services, and standards that allow many teams to build better and faster. 

• You’re practical about AI – you can separate meaningful technological change from hype and focus teams on where AI can produce measurable security outcomes. 

• You’re comfortable making hard technical decisions – you can evaluate competing technologies and approaches, make a recommendation, explain the trade-offs, and stand behind the decision. 

• You’re security-minded by design – you treat AI as both a defensive capability and an attack surface and consider adversarial behavior from the beginning. 

• You’re a multiplier – you want the engineers and teams around you to become stronger because of the architecture, standards, mentorship, and technical guidance you provide. 

• You lead through influence – you can establish technical direction and build alignment across teams and stakeholders even when they don’t report to you. 

• You’re willing to challenge the premise – if a use case is vague, a vendor claim doesn’t hold up, or AI isn’t the right solution, you’ll say so and redirect attention toward measurable value. 

• You’re continuously learning – you follow AI and security research closely, evaluate emerging technologies critically, and bring back what can genuinely improve StoneX’s capabilities. 

• You’re a versatile communicator – you can explain complex AI concepts and strategic trade-offs to executive audiences while still being able to go deeply technical with engineers and researchers. 

Qualifications

To land this role you will need:

• 12+ years of hands-on software engineering experience building and operating production systems, including 5+ years focused on AI/ML, security engineering, or both, with experience across tools and frameworks such as DSPy, LiteLLM, Datadog, and Azure. 

• Proven experience operating as a Principal, Staff, or equivalent senior technical leader with responsibility for architecture and technical direction spanning multiple teams or an organization. 

• Demonstrated experience architecting, deploying, and operating GenAI applications, RAG solutions, or AI agents in production at scale, including evaluation, monitoring, and cost management. 

• Deep expertise with large language models, prompt and context engineering, embeddings and vector databases, agent frameworks and tool orchestration, and model evaluation techniques. 

• Expert proficiency in Python and strong experience with at least one additional production language such as TypeScript/JavaScript, Go, Java, or C#. 

• Full-stack technical credibility spanning back-end services and APIs, front-end interfaces, integrations, and data pipelines. 

• Strong knowledge of distributed systems and cloud-native architecture across AWS, Azure, or GCP, including experience scaling and containerizing workloads in Azure, and infrastructure as code. 

• Deep understanding of cybersecurity principles, technologies, and workflows including SIEM, SOAR, vulnerability management, IAM, cloud security, and application security. 

• Strong working knowledge of AI-specific security risks and relevant frameworks such as the OWASP Top 10 for LLM Applications, MITRE ATT&CK, and NIST AI Risk Management Framework. 

• Demonstrated ability to influence senior technical stakeholders and executives and drive technical change across organizational boundaries. 

• Strong written and verbal communication skills with the ability to communicate technical strategy, architecture, trade-offs, and risk across both deeply technical and executive audiences. 

What makes you stand out:  

• Experience within financial services or another highly regulated industry. 

• Experience with model risk management, responsible AI, AI governance, or similar regulatory and governance requirements. 

• Industry contributions such as publications, conference presentations, patents, standards participation, or open-source leadership. 

• Bachelor’s or Master’s degree in Computer Science, Cybersecurity, Engineering, or a related field. Equivalent practical experience and non-traditional paths are welcomed.

 

Working environment:

• 4 days' work from office.