Big Data Engineer

2 weeks ago


Austin, United States ShiftCode Analytics Full time

Interview : Video

Visa : All apart from H1b , cpt and TN

This is hybrid from day-1.

Description :

Requirements:

  • 5+ years' experience in data engineering work
  • Previous experience as a Big Data Engineer
  • In-depth knowledge of Hadoop, Spark, Airflow
  • Technical expertise with data models, data mining, and segmentation techniques
  • Hands-on experience with SQL database, Redis
  • Familiarity with Kubernetes and Docker tools
  • Familiarity with S3, Apache Parquet and Apache Iceberg
  • Understand how to extract data
  • Experience with Graph databases
  • Knowledge of programming languages (e.g. Python)
  • Experience with Streaming - Kafka or RMQ
  • Experience with Gitlab based CI/CD
  • Experience with Data catalog

Added advantage:
  • AI/Client Engineer
  • Client Pipelines

Responsibilities:
  • Analyze and organize raw data
  • Build data systems and pipelines
  • Interpret trends and patterns in enterprise data
  • Conduct complex data analysis and report on results
  • Prepare data for prescriptive and predictive modeling
  • Combine raw information from different sources
  • Explore ways to enhance data quality and reliability
  • Identify opportunities for data acquisition
  • Collaborate with data scientists and architects on several projects
  • Work independently with different data owners and design data flows

  • Systems Engineer

    2 days ago


    Austin, United States Hudson River Trading Full time

    Hudson River Trading (HRT) is looking for Systems Engineers to join our growing Research & Development team. This team builds and maintains an exceptionally large and growing distributed compute cluster, many-PB-scale storage systems, operating systems, automation software, and development tools. We are seeking a highly skilled engineer with expertise in...


  • Austin, United States Diverse Lynx Full time

    Title: Big Data Engineer with cloud experience Location: Sunnyvale, CA/Austin, TX(Onsite) Experience: 10+ Years Technical/Functional Skills: Big Data, Tableau, Kafka, Iceberg, Spark Roles & Responsibilities: Having 10 years of professional experience fields of software Analysis, Design, Development, Deployment and Maintenance of software and Big Data...


  • Austin, United States Diverse Lynx Full time

    Role: Big Data Platform Administration Location: Austin, TX/ Dallas, TX (Onsite) Contract Skills - Platform: HPE/MapR Hadoop (Big Data), Programming: Python, UNIX Scripting CICD: Ansible, Platform Support, ITSM tools knowledge, ITIL 2 12 Years of IT experience in Big Data, Hadoop platform management & min 2 years experience in Big Data platform...

  • Cloud Data Engineer

    3 days ago


    Austin, Texas, United States Apple Full time

    SummaryPosted: May 24, 2024Role Number: The people here at Apple don't just create products - they create the kind of wonder that's revolutionized entire industries. It's the diversity of those people and their ideas that inspires the innovation that runs through everything we do, from amazing technology to industry-leading environmental efforts. Apple is...

  • Data Engineer

    2 weeks ago


    Austin, United States VMC Soft Technologies, Inc Full time

    -----------------------------------------Title: Data Engineer (Spark and Scala)Location: Austin, TX / Sunnyvale, CA(Hybrid-3 days in a week)W2 Contract-----------------------------------------Required Qualifications:Bachelor’s degree or foreign equivalent required from an accredited institution.NO C2CAt least 8 years of experience in Information...

  • Data Engineer

    2 weeks ago


    Austin, United States VMC Soft Technologies, Inc Full time

    -----------------------------------------Title: Data Engineer (Spark and Scala)Location: Austin, TX / Sunnyvale, CA(Hybrid-3 days in a week)W2 Contract-----------------------------------------Required Qualifications:Bachelor’s degree or foreign equivalent required from an accredited institution.NO C2CAt least 8 years of experience in Information...

  • Data Engineer

    2 weeks ago


    Austin, United States Loxo Full time

    Job DescriptionJob DescriptionAs an early hire to our engineering team, you will be responsible for managing Loxos data integration function. You will be primarily responsible for migrating new clients legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience...

  • Azure Data Engineer

    2 weeks ago


    Austin, United States Epicor Inc Full time

    Epicor is seeking a highly skilled Azure Data Engineer to join our dynamic team. As Data Engineer, you will be responsible for overseeing all aspects of our Azure data processing pipelines, including development, implementation, and testing. In addition, you will work closely with our business functions and BI team to identify opportunities to optimize their...

  • Senior Data Engineer

    1 month ago


    Austin, United States 2K Full time

    Who We Are 2K is headquartered in Novato, California and is a wholly owned label of Take-Two Interactive Software, Inc. (NASDAQ: TTWO). Founded in 2005, 2K Games is a global video game company, publishing titles developed by some of the most influential game development studios in the world. Our studios responsible for developing 2K’s portfolio of...


  • Austin, United States Civitas Learning Full time

    Civitas Learning is bringing together the best of new technology, design thinking, and data science in our mission to help one million more students graduate each year. We are building a data platform and cloud-based applications to deliver insights and action analytics to the frontlines of education. We are looking for insanely talented people who want to...


  • Austin, United States Civitas Learning Full time

    Civitas Learning is bringing together the best of new technology, design thinking, and data science in our mission to help one million more students graduate each year. We are building a data platform and cloud-based applications to deliver insights and action analytics to the frontlines of education. We are looking for insanely talented people who want to...

  • Data Engineer

    5 days ago


    Austin, United States XeoMatrix Full time

    Data Engineer We are currently seeking an experienced data engineer with 5 - 7 years with hands-on data engineering experience. This candidate must possess technical, business analysis, and communication skills. This position offers an opportunity to work directly with clients to design strategic data solutions that help them visualize complex data in a...

  • Data Engineer

    4 days ago


    Austin, United States XeoMatrix Full time

    Data Engineer We are currently seeking an experienced data engineer with 5 - 7 years with hands-on data engineering experience. This candidate must possess technical, business analysis, and communication skills. This position offers an opportunity to work directly with clients to design strategic data solutions that help them visualize complex data in a...

  • Data Engineer

    4 days ago


    Austin, Texas, United States Apple Full time

    SummaryPosted: Apr 24, 2024Weekly Hours: 40Role Number: At Apple, we work every day to create products that enrich people's lives Our Advertising Platforms group makes it possible for people around the world to easily access informative and imaginative content on their devices while helping publishers and developers promote and monetize their work. Today,...

  • Data Engineer

    3 weeks ago


    Austin, United States X-Bow Systems Full time

    Job DescriptionJob DescriptionCompany Overview:X-Bow Launch Systems, Inc. is a cutting-edge aerospace technology company at the forefront of revolutionizing space launch and transportation. With a commitment to innovation and a mission to advance space exploration, we are always on the lookout for talented individuals to join our dynamic team.Location: This...

  • Data Engineer

    1 week ago


    Austin, United States Russell Tobin Full time

    Job DescriptionJob DescriptionLocation: Remote/Prefers someone to work in the central time zonePay Range: $55.00 - 67.00/hour W2Contract Duration: 12+ Potential to extend Summary:Client Commerce Engineering Operations is building next-generation business analytics and decision support capabilities to enable modern, data-driven engineering management and...

  • Data Engineer

    2 weeks ago


    Austin, United States Vaco Full time

    Responsibilities:Assembling mixed sets of data that meet non-functional and functional business requirements.Identifying, designing, and implementing internal process improvements including re-designinginfrastructure for greater scalability, optimizing data delivery, and automating manualprocesses.Building required infrastructure for optimal extraction,...

  • Data Engineer

    2 weeks ago


    Austin, United States Vaco Full time

    Responsibilities:Assembling mixed sets of data that meet non-functional and functional business requirements.Identifying, designing, and implementing internal process improvements including re-designinginfrastructure for greater scalability, optimizing data delivery, and automating manualprocesses.Building required infrastructure for optimal extraction,...

  • Data Engineer

    2 weeks ago


    Austin, United States Vaco Full time

    Responsibilities:Assembling mixed sets of data that meet non-functional and functional business requirements.Identifying, designing, and implementing internal process improvements including re-designinginfrastructure for greater scalability, optimizing data delivery, and automating manualprocesses.Building required infrastructure for optimal extraction,...

  • Data Engineer

    3 days ago


    Austin, United States Dealerware Full time

    About Dealerware: Dealerware transforms the automotive retailers of today into the mobility network of tomorrow. Launched in 2016, Dealerware manages tens of thousands of vehicles at dealerships in North America across every major manufacturer brand. Dealerware is committed to being a people-first culture and a great place to work for all, with a...