DevOps Data Ingestion Engineer

2 weeks ago


San Francisco, United States SherlockTalent Full time

Job Title: AI DevOps Data Ingestion Engineer

Location: Bay Area, CA

Job Type: Founding Level SWE, Full Time

Salary: Founders-level equity and $200K+

Job ID: 7330

SherlockTalent is Hiring: AI DevOps Data Ingestion Engineer

Are you passionate about designing scalable data pipelines and ensuring robust infrastructure for seamless data flow?

SherlockTalent is seeking an AI DevOps Data Ingestion Engineer to bridge the gap between infrastructure and data engineering, building and maintaining data ingestion pipelines while automating deployment processes.

Key Responsibilities:
  1. Design, develop, and maintain scalable and efficient data ingestion pipelines to collect and process data from diverse sources such as CRMs, marketing automation platforms, and sales systems.
  2. Automate infrastructure provisioning and pipeline deployments using Infrastructure as Code (IaC) tools like Terraform, Ansible, or CloudFormation.
  3. Implement and manage CI/CD pipelines to support the deployment and monitoring of data systems.
  4. Collaborate with data scientists and analysts to ensure data integrity, normalization, and transformation meet analytics needs.
  5. Optimize data ingestion workflows for performance, scalability, and reliability in cloud environments like AWS, GCP, or Azure.
  6. Monitor data pipelines, troubleshoot issues, and maintain high availability and fault tolerance across systems.
  7. Ensure security and compliance for data systems by implementing best practices for access control, encryption, and monitoring.
  8. Provide support to cross-functional teams by ensuring data infrastructure aligns with organizational goals.
Required Skills:
  1. Strong experience with cloud platforms (AWS, GCP, Azure) and associated data services (e.g., S3, BigQuery, Redshift, or similar).
  2. Expertise in building data ingestion pipelines using tools like Apache Kafka, Apache NiFi, or AWS Glue.
  3. Hands-on experience with CI/CD tools such as Jenkins, GitLab CI/CD, or GitHub Actions.
  4. Proficiency in scripting and programming languages such as Python, Bash, or Go for automation.
  5. Familiarity with containerization and orchestration tools like Docker and Kubernetes.
  6. Strong knowledge of database systems (SQL and NoSQL) and data transformation techniques.
  7. Solid understanding of monitoring and logging tools such as Prometheus, Grafana, or ELK stack.
Preferred Qualifications:
  1. Experience with data lake or data warehouse architectures.
  2. Knowledge of big data technologies such as Spark, Hadoop, or Flink.
  3. Background in implementing data security best practices in cloud environments.

Why Join Us? As an AI DevOps Data Ingestion Engineer at SherlockTalent, you’ll have the opportunity to work on complex and impactful projects, ensuring that our systems are scalable, secure, and optimized for data-driven success. Join a team that thrives on innovation, collaboration, and delivering excellence.

Apply Now: Be part of a dynamic environment where your expertise in DevOps and data engineering will shape the future of our data infrastructure. Join SherlockTalent today

**SherlockTalent loves to share $1,000 bonuses for successful referrals.

#J-18808-Ljbffr

  • San Francisco, United States SherlockTalent Full time

    Job Title: AIDevOps Data Ingestion EngineerLocation: Bay Area, CAJob Type: Founding Level SWE, Full TimeSalary: Founders-level equity and $200K+Job ID: 7330SherlockTalent is Hiring: AI DevOps Data Ingestion EngineerAre you passionate about designing scalable data pipelines and ensuring robust infrastructure for seamless data flow?SherlockTalent is seeking an...


  • San Jose, California, United States McAfee Full time

    Job Overview:We are seeking a seasoned analytical leader to join our Enterprise Data Platform team as the Capability Lead, Data Ingestion. As a key leader of McAfee's enterprise data strategy, you will be responsible for leading a team that manages ingestion on the Databricks platform. This role requires a deep understanding of data lake ingestion processes...


  • San Jose, California, United States McAfee Full time

    About the Role:This position offers an opportunity to lead a team of data ingestion engineers in managing and maintaining ETL/ELT pipelines on the Databricks platform. The ideal candidate will have 5+ years of experience in data engineering, ingestion pipelining, and ETL/ELT, with hands-on experience in Spark/Scala, SQL, Python/PySpark or similar programming...


  • San Francisco, California, United States OpenAI Full time

    Job SummaryWe are seeking an experienced Software Engineer to lead the development of our data acquisition and ingestion systems at OpenAI. The successful candidate will have a strong background in software development, data systems, and scalability.Key ResponsibilitiesLeading the development of data acquisition and ingestion systems.Designing and...


  • San Jose, California, United States Adobe Inc. Full time

    About the RoleWe are seeking a highly skilled Senior Software Engineer to join our team at Adobe Inc. in this key role as a Data Ingestion Expert. This is an exciting opportunity for an experienced software engineer to leverage their expertise in building high-performance data ingestion pipelines and data stores to serve complex use cases.Job...

  • AI Infrastructure

    2 weeks ago


    San Francisco, United States SherlockTalent Full time

    Job Title: AI Infrastructure & Data EngineerLocation: Bay Area, CAJob Type: Founding Level SWE, Full TimeSalary: Founders-level equity and $200K+Job ID: 7332SherlockTalent is Hiring: AI Infrastructure & Data EngineerAre you ready to build the backbone of innovative systems that empower cutting-edge applications and data pipelines? SherlockTalent is seeking...


  • San Francisco, California, United States SherlockTalent Full time

    At SherlockTalent, we are seeking an experienced AI DevOps Data Ingestion Engineer to design and develop scalable data ingestion pipelines that collect and process data from diverse sources. Key ResponsibilitiesWe are looking for a talented professional who can automate infrastructure provisioning and pipeline deployments using Infrastructure as Code (IaC)...

  • DevOps Specialist

    7 days ago


    San Diego, California, United States Diverse Lynx Full time

    About the Opportunity:We are excited to offer a unique opportunity for a skilled AWS SRE Engineer to join our team at Diverse Lynx LLC. As an AWS SRE Engineer, you will be responsible for designing, deploying, and maintaining scalable and highly available systems on the Amazon Web Services (AWS) cloud platform.The ideal candidate will have 7+ years of...


  • San Francisco, California, United States SherlockTalent Full time

    About the RoleWe are seeking an experienced Cloud Engineer to join our team as an AI Infrastructure and Data Engineer. This is a fantastic opportunity to work with innovative technologies and contribute to the development of scalable, reliable, and secure infrastructure.In this role, you will be responsible for designing and implementing cloud-based...

  • DevOps Engineer

    3 months ago


    San Francisco, United States Sigmaways Inc Full time

    Looking for a DevOps engineer to craft comprehensive system monitoring strategies specifically tailored for data center environments, covering both physical and virtual infrastructure.Responsibilities:Design and implement robust cloud architectures (AWS, GCP) and on-premises data center monitoring and automation solutions, ensuring they align with business...

  • Data Architect

    4 weeks ago


    San Antonio, Texas, United States Syntricate Technologies Full time

    Data Scientist (Client Infrastructure) Job InformationAt Syntricate Technologies, we are seeking a talented Data Scientist to design and develop Client infrastructure, including data ingestion, storage, and management systems. The ideal candidate will have a strong background in data science and machine learning, as well as experience with tools such as...

  • Software Architect

    2 days ago


    San Francisco, California, United States Hyperspell Full time

    Job OverviewHyperspell is pioneering a revolutionary data access platform for AI applications, ensuring secure and seamless connections to user data. As a Founding Python Generalist Engineer, you'll play a pivotal role in designing and implementing our core infrastructure from scratch.Key ResponsibilitiesDesign and Implement Core Infrastructure: Collaborate...


  • San Francisco, California, United States Together AI Full time

    Job SummaryWe are seeking a highly skilled Senior DevOps Engineer to join our cloud engineering organization. As a key member of our team, you will be responsible for developing and maintaining the infrastructure for our AI workloads, ensuring scalability, reliability, and high performance. Key Responsibilities- Design and implement automated infrastructure...


  • San Francisco, California, United States CyberTec Full time

    Job Summary:CyberTec is seeking an experienced Cloud Data Engineering Specialist to join our team. In this role, you will be responsible for designing and developing cloud-based data pipelines using Kafka, DynamoDB, and related AWS technologies.The ideal candidate will have more than 7 years of experience in owning the design and development activities of...

  • DevOps Engineer

    1 month ago


    San Francisco, United States Sigmaways Inc Full time

    QualificationsExperience in GitLab or any other DevOps technologies is also required as you will be engaged in the DevOps process (code review, code merge, code deployment, executing CI/CD pipelines, dependency, vulnerability scanning, etc)Bachelor’s degree in Computer Science, Information Systems, or other related technical field or equivalent...

  • Software Engineer

    2 weeks ago


    San Francisco, United States Gigascale Capital Full time

    Terradot is hiring a Software Engineer (Data). This role will be pivotal in developing and scaling our data infrastructure and integrations to support the scaling of our deployments and removing carbon at the speed & scale our planet requires. You will build data infrastructure that leverage bits to move atoms effectively!This role is a key part of...


  • San Francisco, California, United States Grow Full time

    About RockerboxRockerbox empowers marketing executives to make informed decisions with confidence. We help companies like Tula, Figs, and Burton drive growth through strategic decision-making.Our Integrations team plays a crucial role in our success, managing and scaling data pipelines that ingest marketing data from third-party APIs. This data is validated,...

  • DevOps Engineer

    4 weeks ago


    San Jose, California, United States Diverse Lynx Full time

    Job Title: DevOps Engineer - Data Pipeline SpecialistSalary: $120,000 - $180,000 per year.About Diverse Lynx LLCDiverse Lynx LLC is an Equal Employment Opportunity employer. We promote and support a diverse workforce across all levels in the company.Job Description:We are seeking a skilled DevOps Engineer to join our team. The successful candidate will be...

  • DevOps Engineer

    4 days ago


    San Francisco, United States Houston Texans Full time

    Company OverviewSwish Analytics is a sports analytics, betting and fantasy startup building the next generation of predictive sports analytics data products. We believe that oddsmaking is a challenge rooted in engineering, mathematics, and sports betting expertise; not intuition. We're looking for team-oriented individuals with an authentic passion for...

  • DevOps Engineer

    10 hours ago


    San Francisco, California, United States Stackwatch Full time

    Job Title: DevOps EngineerWe are seeking a skilled DevOps Engineer to join our team at Stackwatch. As a key member of our DevOps team, you will be responsible for designing, implementing, and maintaining continuous integration and continuous deployment (CI/CD) pipelines using industry-leading tools such as Jenkins, Travis CI, and GitLab CI/CD.Key...