Senior AI Infrastructure Engineer

4 days ago

New York, New York, United States UMATR Full time

Senior AI Infrastructure Engineer – Platform & Reliability

We are looking for a
Senior AI Infrastructure Engineer
to join the Platform & Reliability team and help design, operate, and secure the core systems that power our AI products. You will focus on
high-concurrency inference, data pipelines, and automation tooling
, ensuring infrastructure reliability under heavy load.

Responsibilities

  • System Design & Architecture:
    Build scalable, reliable infrastructure for AI workflows and high-volume data pipelines
  • Queue & Job Scheduling:
    Transition from traditional multiprocessing and database-backed queues to
    Kubernetes-native orchestration
  • Managed Data Pipelines:
    Tune partitioning and throughput, implement safe retries, dead-letter queues, and idempotent sinks
  • Autoscaling & Resilience:
    Scale based on queue lag, request volume, and latency; add burst capacity and safe drains
  • Tool Reliability:
    Harden AI toolchains with circuit breaking, sandboxing, timeouts, and auditing
  • Progressive Delivery:
    Implement canary and blue/green deployments, pre-warm models, and graceful termination
  • Observability:
    Build dashboards and distributed traces across infrastructure and AI workflows
  • Infrastructure as Code:
    Manage multi-environment deployments with IaC tools, secrets management, and policy-as-code

Requirements

  • 3+ years of experience operating high-concurrency backends or distributed systems
  • Experience with
    Kafka or other message streaming platforms
    , including fan-in/out and at-least-once processing
  • Production experience in
    Python and Rust
  • Experience with incident response, chaos testing, and capacity planning
  • Familiarity with
    AWS, Kubernetes, Terraform/Helm/Kustomize
  • Strong debugging skills across runtime, networking, and authentication layers
  • Security-focused mindset: least privilege, default-deny, audibility, policy enforcement

Nice to Have:
GPU workloads, inference servers, token streaming, cross-region active/active deployments, service mesh, analytics databases

  • Senior AI Engineer

    4 days ago

    New York, New York, United States Information Technology Senior Management Forum Full time

    Posted Date11/03/2025DescriptionSenior AI Engineer (AI Foundations, LLM Core and Agentic AI)Overview:At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized customer experiences. Our investments in technology...

  • Senior AI Engineer

    2 weeks ago

    New York, New York, United States Information Technology Senior Management Forum Full time $158,600 - $197,400 per year

    Posted Date10/31/2025DescriptionSenior AI Engineer (AI Foundations, LLM Core, Agentic AI)At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized customer experiences. Our investments in technology...

  • AI Engineer

    2 days ago

    New York, New York, United States People In AI Full time

    AI/Machine Learning Engineer, LLMs & Infra$325,000 base + bonus + packageNYC HybridJoin a pioneering VC firm building its AI future.This is a rare opportunity to become the first dedicated ML Engineer at a top-tier investment firm: one that sees AI not as an add-on, but as core to its evolution. Backed by decades of success and billions in managed capital,...

  • Senior AI Engineer

    6 days ago

    New York, New York, United States Finster AI Full time

    Compensation: $120,000-$200,000 + Equity About Finster AI We're a Series A stage firm, redefining the future of finance with our AI-native research and task automation platform, backed by leading, global venture investors. Founded by a team of experts from Google DeepMind, Meta AI, and J.P. Morgan, Finster AI provides cutting-edge solutions to help...

  • Senior AI Engineer, Prior Authorization

    4 days ago

    New York, New York, United States Arbiter AI Full time $180,000 - $240,000

    Arbiter is reimagining how healthcare works - not by adding more point solutions, but by building the infrastructure that runs the system. We're designing the intelligent operating spine that unifies data, automates workflows, and aligns incentives across providers, payers, and patients. Our platform embeds AI into real-world care and revenue cycle...

  • Senior Lead AI Engineer

    2 weeks ago

    New York, New York, United States Information Technology Senior Management Forum Full time $225,400 - $280,600 per year

    Posted Date11/03/2025DescriptionSenior Lead AI Engineer (AI Foundations, LLM Core and Agentic AI)Overview:At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized customer experiences. Our investments in...

  • Senior AI Engineer

    2 weeks ago

    New York, New York, United States Capital One Full time $158,600 - $197,400 per year

    Senior AI Engineer (AI Foundations, LLM Core, Agentic AI)At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized customer experiences. Our investments in technology infrastructure and world-class talent —...

  • Senior AI Engineer

    5 days ago

    New York, New York, United States Largeton Group Full time

    Title: Senior AI EngineerTerm: Full TimeLocation: Remote (New York)Role Overview:This role is focused on building and optimizing advanced AI-driven systems that bring agentic workflows to life. You'll design and implement LLM-powered pipelines with continuous evaluation and feedback loops, develop organization-specific customization layers, and create...

  • Senior Infrastructure Software Engineer

    2 weeks ago

    New York, New York, United States Nexus Full time $150,000 - $250,000 per year

    About NexusNexus is innovating at the intersection of artificial intelligence, blockchain, and zero-knowledge cryptography to build a Layer 1 for the AI era. Our team of world-leading experts is developing the Nexus Layer 1 blockchain, Nexus zkVM, and other breakthrough products with the goal of creating a verifiable financial world.Nexus has raised $25M in...

  • Senior Cloud Infrastructure Engineer

    5 days ago

    New York, New York, United States TekleadsLLC Full time

    Strictly W2 No C2CJob Title: Senior Cloud Infrastructure Engineer (Azure/AWS)Locations: New York, NY | Pittsburgh, PA | Lake Mary, FL (Hybrid Onsite - 2-3 days per week)Please Note: We are only considering local candidates who can work onsite in one of the locations listed above.About the Role:We are seeking a highly skilled and experienced Cloud...