Quality Assurance Automation Engineer
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
21X | Remote | Full-Time
21X is building the next generation of conversational AI infrastructure.
Our platform powers intelligent AI agents that communicate with customers across voice, SMS, RCS, email, and web. We help enterprises automate complex customer interactions, execute transactions, and deliver exceptional customer experiences at scale.
We’re growing across mortgage, aviation, sports, and other industries, with a strong focus on enterprise reliability, performance, and scalability.
We’re looking for an exceptional Senior QA Automation Engineer to build and own our quality assurance function.
This isn’t a traditional QA position.
We’re looking for a highly technical, automation-first engineer who can design, implement, and lead our entire testing ecosystem.
You’ll work directly with engineering and product leadership to create a scalable QA infrastructure that enables rapid development without compromising quality.
Our ambition is to automate nearly everything that can reasonably be automated.
You’ll establish the frameworks, infrastructure, processes, and standards that make that possible.
As the company grows, you’ll also lead and manage manual QA engineers responsible for exploratory testing, complex scenarios, and validating AI behavior.
1. Build Our QA Automation Infrastructure
- Architect and implement an automated testing framework from scratch.
- Develop comprehensive end-to-end, integration, API, and regression testing.
- Integrate automated testing into our CI/CD pipelines.
- Establish automated testing across development, staging, and production environments.
- Create reusable test frameworks that scale with our engineering organization.
- Implement intelligent test execution, reporting, and failure analysis.
2. Automate Testing Across Our AI Platform
Our technology presents unique testing challenges. You’ll develop automation to validate:
- Voice AI: Call flows, agent responses, interruptions, transfers, latency, and conversation outcomes.
- SMS & RCS: Message delivery, conversational workflows, rich messaging, and carrier integrations.
- AI Agents: Tool execution, prompt adherence, knowledge retrieval, conversational accuracy, and agent behavior.
- CRM Integrations: Salesforce and other enterprise integrations, data synchronization, and API reliability.
- Enterprise Workflows: Multi-step automation, appointment scheduling, transaction execution, and complex business logic.
You’ll also build automated regression suites for AI agent behavior to identify changes in conversational performance as models, prompts, and underlying infrastructure evolve.
3. Implement AI-Powered QA
We want someone who thinks beyond traditional test automation.
- Leverage LLMs and AI-assisted testing to generate and maintain test cases.
- Build automated conversation simulations across multiple channels.
- Develop evaluation frameworks for conversational AI quality.
- Implement automated scoring for agent accuracy, compliance, and task completion.
- Create systems that detect regressions in AI performance.
- Explore autonomous testing agents that can identify defects and validate new functionality.
4. Lead Manual QA Engineers
- Establish QA standards, test plans, and release validation processes.
- Lead and mentor manual QA engineers as the team grows.
- Delegate exploratory testing, edge-case validation, and complex user scenarios.
- Create clear defect reporting, severity classifications, and escalation procedures.
- Ensure