Principal Test System Architect

7 hours ago

Colorado Springs, Colorado, United States Sanmina Full-time $145,000 - $190,000 Contract
Description Sanmina Corporation (Nasdaq: SANM) is a global powerhouse in integrated manufacturing solutions serving the fastest growing segments of the global Electronics Manufacturing Services (EMS) market. We are a global engine providing the end-to-end manufacturing backbone for the world's most essential and innovative brands. Whether it's autonomous vehicles, life-saving medical tech, next-gen aerospace, or AI/cloud infrastructure
- our integrated solutions don’t just meet standards—they redefine them. Summary Viking Enterprise Solutions (VES) has a rich history of leadership in providing storage and storage server platforms to hyperscale and enterprise data centers worldwide. The division leads the market with ultra-high capacity and high-performance systems. Now VES provides advanced software solutions supporting ML/AI applications for generative and inferential AI in the data center and at the Edge. Software is at the heart of what makes VES a solutions provider. Cloud native systems are designed and built to simplify the use of tools and hardware so users get the job done quickly and efficiently. VES also designs and builds NVMe flash based server appliances, JBOD storage systems and related products for a variety of data center and storage applications. Viking Enterprise Solutions products are tailored for integration into OEM, System Integrator and data center products. Viking Enterprise Solutions is proud to be part of the Sanmina family. Sanmina is recognized as a technology leader, providing end-to-end design, manufacturing and logistics solutions, maintaining a global footprint of design and manufacturing operations in 23 countries on six continents. What You’ll Do In this newly created role, you will act as Test System Architect to design cloud-native orchestration system capable of validating hundreds of complex Storage Servers, Compute servers, JBODs, and GPU enclosures in parallel. You will operate at the intersection of bare-metal hardware, asynchronous microservices, AI-drive data forensics, and real-time UI visualization to drive comprehensive System Validation, rigorous Firmware Verification across multiple platforms. To succeed in this role, you must bring relentless dedication, an extensive foundation in test engineering, deep linux HW expertise, and uncompromising commitment to software engineering excellence. You will share architectural ownership over advanced platforms that sit at the core of our operations, partnering closely with cross-site engineering teams to define and standardized our global test orchestration framework. Sanmina/ZT system is seeking a team player who enjoys technical challenges within a fast-paced environment, applies sound judgment in successful task planning and execution, meets deadlines, and communicates the highest standards of professionalism in his/her work habits.
• Serve as a strategic bridge between firmware, hardware, and software engineering teams to align testing frameworks with global business and manufacturing objectives.
• Architect modular, stateful Python automation scripts and Hardware-in-the-Loop (HIL) test execution frameworks. Develop intelligent, data-driven test suites capable of autonomous system-level validation across diverse hardware configurations.
• Architect flexible test execution pipelines catering to different product lifecycles — ranging from rapid factory-floor Production Reliability testing, to comprehensive Systems Validation for long-term burn-in, to rigorous Design Verification for early-stage firmware.
• Lead the design and implementation of highly scalable testing frameworks utilizing Temporal Durable Execution, NATS JetStream, and FastAPI to orchestrate hundreds of parallel hardware testing workflows.
• Architect and develop strict asynchronous execution engines using uvloop, asyncssh, and asyncpg to eliminate GIL contention, maximize CPU utilization, and manage thousands of concurrent Device Under Test (DUT) communication sessions.
• Architect bare-metal provisioning pipelines using iPXE and Ansible for automated OS deployment, dependency injection, and large-scale lab infrastructure deployments.
• Architect dynamic telemetry pipelines across multiple hardware generations. Implement Redfish API for modern out-of-band management (Gen5/Gen6), while leveraging deep in-band Linux telemetry architectures (SMBIOS/DMI extraction, SCSI Enclosure Services, BMC/IPMI interfaces, and NVMe health diagnostics) for robust legacy support (Gen4) and advanced storage diagnostics.
• Design and automate enterprise-grade qualifications, performance tuning workloads, and fault-injection scenarios (Chaos Engineering) to validate the operational boundaries and recovery capabilities of advanced storage and computer architectures.
• Embed modern data pipelines and Vector/RAG technologies directly into test frameworks to automate root cause analysis (RCA), parse complex logs, and dynamically map failure profiles for RMA triage.
• Architect seamless integrations between our testing frameworks and enterprise tracking tools (Jira, Confluence) via REST APIs to automate defect tracking, enforce ISO 9000 Quality Management System (QMS) workflows, and generate dynamic compliance reporting.
• Rapidly architect and deploy extensions for our testing frameworks during New Product Introduction (NPI), partnering across engineering and manufacturing disciplines to ensure the orchestration platforms fully support the testing of next-generation hardware architectures.
• Lead and mentor a team of software engineers in the development of enterprise testing frameworks. Architect automated CI/CD pipelines, enforce rigorous version control strategies (GitFlow, Trunk-based), and implement Infrastructure as Code (IaC) deployment paradigms. What You’ll Bring
• Engineering Bachelor’s degree in related field of work.
• Minimum of 15 years of related experience.
• Responsible for architecting and driving multiple test projects and leading a small team of test Automation and AI engineering team
• Advanced proficiency in modern Python (3.12+) and strict asynchronous programming paradigms (asyncio, uvloop) to maximize high-speed hardware I/O.
• Proven experience architecting high-performance RESTful microservices and data models using FastAPI, Pydantic, and SQLAlchemy.
• Extensive experience with Docker, Kubernetes, object storage (MinIO), and architecting high-performance relational and time-series databases (PostgreSQL, QuestDB, SQLAlchemy).
• Proven experience building fault-tolerant state machines using Temporal.io and high-throughput data streams (e.g., NATS JetStream).
• Deep understanding of modern web development using React and TypeScript, with experience in state management (Zustand) and real-time data streaming (WebSocket/SSE).
• Demonstrated experience developing CI/CD pipelines (GitHub Actions, Jenkins, GitLab), architecting custom integrations with Atlassian toolchains (Jira, Confluence REST APIs), and enforcing ISO 9000 document control workflows.
• Extensive understanding of Linux OS performance tuning, kernel panic triage, memory management, and PCIe topology mapping.
• Mastery of industry-standard benchmarking and stress-testing methodologies across every hardware commodity. This includes synthetic block-storage benchmarking, deep JBOD/enclosure telemetry mapping, memory bandwidth and latency stressors, pure floating-point compute burn-in, and RDMA/high-speed fabric network testing.
• Advanced knowledge of storage protocols (NVMe, SAS), JBOD architectures, and compute node topologies.
• Architecting and developing Zero-Touch Provisioning (ZTP) pipelines, custom iPXE boot configurations, and headless configuration management (Ansible) ecosystems.
• Proven experience with hardware bring-up, debugging complex integration issues, and influencing system-level Design for Testability (DFT).
• Experienced architecting RAG (Retrieval-Augmented Generation) applications and integrating LLMs for automated data forensics and log analysis. Why Join Us? At Sanmina, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $145,000
- $190,000 and your base pay will depend on your skills, qualifications, experience, and location. Sanmina employees also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, and for formal education related to advancing your career at Sanmina, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Sanmina is an Equal Opportunity Employer and prohibits discrimination and harassment. Sanmina provides equal employment opportunities to employees and applicants for employment and prohibits discrimination and harassment without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or other characteristic protected by applicable federal, state, or local laws. Please be aware that this position may require access to technology, software, or technical data that is subject to U.
S. export control laws. All offers of employment are contingent on the applicant either 1) being a "U.
S. Person," defined as a (i) U.
S. citizen or national, (ii) U.
S. lawful permanent resident, or (iii) other protected individual, such as a refugee under 8 U.
S.
C.
• 1157 or asylee under 8 U.
S.
C.
• 1158; or 2) being eligible to obtain a timely export control license or license exception from the applicable U.
S. federal agency.