Data Engineer

3 weeks ago

Austin, United States ZipRecruiter Full time

Job DescriptionJob Description

Are you passionate about building data infrastructure that powers advanced analytics and machine learning? Do you thrive on transforming raw data into well-organized, accessible, and reliable datasets that fuel data-driven decision-making? If you’re excited about working with cutting-edge data technologies and architecting scalable pipelines, then our client has an exciting opportunity for you. We’re looking for a Data Engineer (aka The Data Pipeline Architect) to design, develop, and optimize the data systems that form the backbone of our products.

As a Data Engineer at our client, you’ll be responsible for constructing efficient, scalable data pipelines, ensuring data is accessible and usable for analysts, data scientists, and business stakeholders. You’ll work with large datasets, implement ETL processes, and build the infrastructure that powers analytics and AI-driven insights.

Key Responsibilities:

  1. Design and Develop Data Pipelines:
  • Build and maintain robust, scalable, and efficient data pipelines to ingest, process, and store data from a variety of sources. You’ll design ETL (Extract, Transform, Load) processes to move and transform data, ensuring data integrity and accuracy.
Data Warehouse Management:
  • Architect and maintain data warehouses or data lakes using cloud platforms (e.g., AWS Redshift, Google BigQuery, or Snowflake) to organize and store large-scale datasets. You’ll ensure the infrastructure is optimized for fast querying and scalability.
Collaborate with Data Scientists and Analysts:
  • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver datasets that meet business needs. You’ll provide clean, well-structured data to enable advanced analytics and machine learning projects.
Data Quality and Governance:
  • Implement data quality checks and monitoring systems to ensure the accuracy, completeness, and consistency of data across the pipeline. You’ll help establish data governance standards and policies to ensure compliance and security.
Performance Optimization:
  • Optimize the performance of data systems, ensuring fast and reliable data access. You’ll tune queries, design efficient storage architectures, and implement best practices for data retrieval and processing.
Automation and Monitoring:
  • Automate data workflows, pipeline deployments, and data quality checks to minimize manual intervention. You’ll set up monitoring and alerting systems to detect issues early and ensure smooth operation of data pipelines.
Data Security and Compliance:
  • Implement security protocols to protect sensitive data, ensuring compliance with relevant regulations such as GDPR, HIPAA, or SOC2. You’ll work with security teams to enforce access controls, encryption, and data privacy best practices.

Requirements

Required Skills:

  • Data Engineering Expertise: Strong experience building and maintaining data pipelines, ETL processes, and data warehouses using cloud platforms (AWS, GCP, Azure). You’re skilled at handling large, complex datasets efficiently.
  • Programming and Scripting: Proficiency in such as Python, SQL, or Scala, and experience with data engineering tools like Apache Spark, Airflow, or Kafka. You can write efficient code to process and transform large datasets.
  • Data Warehousing and Storage: Expertise in managing and optimizing data warehouses or data lakes (e.g., Redshift, BigQuery, Snowflake). You understand partitioning, indexing, and storage optimization techniques.
  • Database and Query Optimization: Strong knowledge of database design and query optimization for performance. You can fine-tune SQL queries and structure databases for fast, reliable access to large volumes of data.
  • Data Governance and Security: Solid understanding of data governance practices, security protocols, and compliance regulations. You can enforce data privacy and implement measures to safeguard sensitive information.

Educational Requirements:

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related field. Equivalent experience in data engineering is also highly valued.
  • Certifications in cloud platforms (AWS, GCP, Azure) or data engineering technologies (e.g., Apache Hadoop, Apache Spark) are a plus.

Experience Requirements:

  • 3+ years of experience in data engineering, with hands-on experience building and managing data pipelines, data warehouses, and cloud-based storage solutions.
  • Proven experience working with big data technologies and distributed systems, optimizing data flows and processes to handle large datasets.
  • Familiarity with data quality, data governance, and data security best practices is highly desirable.

Benefits

  • Health and Wellness: Comprehensive medical, dental, and vision insurance plans with low co-pays and premiums.
  • Paid Time Off: Competitive vacation, sick leave, and 20 paid holidays per year.
  • Work-Life Balance: Flexible work schedules and telecommuting options.
  • Professional Development: Opportunities for training, certification reimbursement, and career advancement programs.
  • Wellness Programs: Access to wellness programs, including gym memberships, health screenings, and mental health resources.
  • Life and Insurance: Life insurance and short-term/long-term coverage.
  • Employee Assistance Program (EAP): Confidential counseling and support services for personal and professional challenges.
  • Tuition Reimbursement: Financial assistance for continuing education and professional development.
  • Community Engagement: Opportunities to participate in community service and volunteer activities.
  • Recognition Programs: Employee recognition programs to celebrate achievements and milestones.

  • Data Engineer

    3 weeks ago

    Austin, United States TalentBurst Full time

    Title: Data Engineer II Duration: 12 months Location: Austin, TX 78758 (5 days onsite) Number of Interviews: 3 interviewsJob Description: Client is looking for a Senior Data Engineer to join a team of highly skilled individuals working on our initiatives in resource and application discovery within Client security. Our product, Veritas, is a foundational...

  • Data Engineer

    1 month ago

    Austin, United States eStaff Full time

    We seek a Data Engineer to work on our W2 from anywhere in Texas for a long-term assignment. Please emailhbarnett@estaffllc.com, your resume if you are interested. We cannot work with C2Cs. JOB DESCRIPTION: We are looking for Data engineers who currently live in Texas only. The Data Engineer will be pivotal in designing, developing, and maintaining...

  • Data Engineer

    2 weeks ago

    austin, United States TalentBurst Full time

    Title: Data Engineer II Duration: 12 months Location: Austin, TX 78758 (5 days onsite) Number of Interviews: 3 interviews Job Description: Client is looking for a Senior Data Engineer to join a team of highly skilled individuals working on our initiatives in resource and application discovery within Client security. Our product, Veritas, is a foundational...

  • Data Engineer

    1 week ago

    Austin, United States eStaffLLC Full time

    We are seeking a Data Engineer work on our W2 from anywhere in Texas for a long-term assignment. We cannot work with C2C candidates or provide sponsorship. Must currently live in Texas. JOB DESCRIPTION: The Data Engineer will be pivotal in designing, developing, and maintaining robust data architectures to support data science initiatives. This individual...

  • Data Engineer

    17 hours ago

    Austin, United States Estaffllc Full time

    We are seeking a Data Engineer work on our W2 from anywhere in Texas for a long-term assignment. We cannot work with C2C candidates or provide sponsorship. Must currently live in Texas.JOB DESCRIPTION: The Data Engineer will be pivotal in designing, developing, and maintaining robust data architectures to support data science initiatives. This individual...

  • Data Engineer

    2 weeks ago

    Austin, United States Droisys Full time

    Droisys is an innovation technology company focused on helping companies accelerate their digital initiatives from strategy and planning through execution. We leverage deep technical expertise, Agile methodologies, and data-driven intelligence to modernize systems of engagement and simplify human/tech interaction.Amazing things happen when we work in...

  • Data Engineer

    2 weeks ago

    Austin, United States Droisys Full time

    Droisys is an innovation technology company focused on helping companies accelerate their digital initiatives from strategy and planning through execution. We leverage deep technical expertise, Agile methodologies, and data-driven intelligence to modernize systems of engagement and simplify human/tech interaction.Amazing things happen when we work in...

  • TalentBurst | Data Engineer

    2 weeks ago

    austin, United States TalentBurst Full time

    Title: Data Engineer II Duration: 12 months Location: Austin, TX 78758 (5 days onsite) Number of Interviews: 3 interviews Job Description: Client is looking for a Senior Data Engineer to join a team of highly skilled individuals working on our initiatives in resource and application discovery within Client security. Our product, Veritas, is a foundational...

  • Data Engineer

    2 days ago

    Austin, United States Atlassian Full time

    Atlassian is looking for a Data Engineer to join our Data Engineering team and build world-class data solutions and applications that power crucial decisions throughout the organisation. We are looking for an open-minded, structured thinker who is passionate about building systems at scale. You will enable a world-class data engineering practice, drive the...

  • Data Engineer

    1 month ago

    Austin, United States Diverse Lynx Full time

    Job Title: Data Engineer (Python, Pyspark, Java) Location: Sunnyvale, CA & Austin, TX REMOTE (PST Hours) Job Type: Full timeJOB DESCRIPTION Looking for a Data Engineer who has a passion for building scalable and high performing data pipelines and data services. Responsibilities Design, implement, optimize, and maintain data pipelines and reports using...

  • Data Engineer

    7 months ago

    Austin, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    3 weeks ago

    Austin, Texas, United States realtor Full time

    Job Description\We are seeking a highly skilled Data Engineer to join our team. In this role, you will be responsible for designing, building, and maintaining large-scale data processing systems.The ideal candidate will have a strong background in computer science, with expertise in programming languages such as Java, Python, or Scala. You should also have...

  • Data Engineer

    4 days ago

    Austin, United States Applied Materials Full time

    Data Engineer At Applied Materials, we are building the next generation fab automation solutions using Artificial Intelligence and Machine Learning. Our AI/ML team is looking for a Data Engineer who will be responsible for expanding and optimizing Quality Process data and data pipeline architecture, as well as optimizing data flow and collection for cross...

  • Senior Data engineer

    1 month ago

    Austin, United States Futran Tech Solutions Pvt. Ltd. Full time

    Title - Senior Data engineer Duration - 9+ Months Location - Austin, TX (Day 1 onsite) Locals Only Responsibilities : We are looking for a Senior Data Engineer to join our team and help us build and maintain our data infrastructure and pipelines. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing scalable and...

  • Data Engineer

    1 month ago

    Austin, United States Pedernales Electric Full time

    Data Engineer Austin, TX, US, 78701 PHYSICAL OFFICE LOCATION: PEC Technology Center - 1506 N US Highway 281, Austin, TX 78701 Hybrid Work Options Available $100,000.00 Minimum Starting Base Pay* + KPI Bonuses + Competitive Benefits *Qualifications may warrant greater starting base pay within the full pay range. Individual base pay is determined by...

  • Data Engineering Lead

    4 weeks ago

    Austin, Texas, United States Apple Full time

    Our TeamWe're a diverse group of talented engineers, data scientists, and product managers who share a passion for innovation and excellence.Salary: $200,000/yearWe're looking for a Data Engineering Lead to lead our team in building and maintaining scalable and reliable data platforms.ResponsibilitiesThe ideal candidate will have strong leadership skills,...

  • Data Engineer

    7 months ago

    Austin, United States Virtual Full time

    Responsibilities: ? Assembling mixed sets of data that meet non-functional and functional business requirements. ? Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes. ? Building required infrastructure for optimal...

  • Data Engineering Lead

    3 weeks ago

    Austin, Texas, United States Diverse Lynx Full time

    Data Engineer Lead Job DescriptionWe are seeking a highly experienced data engineer lead to join our team at Diverse Lynx LLC. The successful candidate will be responsible for leading a team of data engineers, overseeing the development and maintenance of big data platforms, and ensuring the delivery of high-quality projects.Key Responsibilities:Lead a team...

  • Principal Data Engineer

    1 month ago

    Austin, United States University of Texas at Austin Full time

    Technical LeadershipDesign, plan, develop and deploy production quality enterprise level data pipelines using Cloud technologies with a team of data engineers using new and emerging industry standard tools, frameworks, and best practices.Conduct and participate in code reviews for a high-quality code.Contribute to Data Engineering, SDLC, process improvements...

  • Senior Data Engineer

    3 months ago

    Austin, United States DauTech Full time

    Senior Data Engineer/Software Engineer for Data Platform Engineering Exclusive Senior Engineering Opportunity with a Pre-IPO SaaS Cloud Security Company Offering Long-Term Growth Potential. As a Senior Data Engineer, you will play a pivotal role in driving the delivery of key features in and around our Application Data Platform (ADP), empowering both our...