Big Data Engineer

1 month ago


Austin, United States ShiftCode Analytics Full time

Interview : Video

Visa : All apart from H1b , cpt and TN

This is hybrid from day-1.

Description :

Requirements:

  • 5+ years' experience in data engineering work
  • Previous experience as a Big Data Engineer
  • In-depth knowledge of Hadoop, Spark, Airflow
  • Technical expertise with data models, data mining, and segmentation techniques
  • Hands-on experience with SQL database, Redis
  • Familiarity with Kubernetes and Docker tools
  • Familiarity with S3, Apache Parquet and Apache Iceberg
  • Understand how to extract data
  • Experience with Graph databases
  • Knowledge of programming languages (e.g. Python)
  • Experience with Streaming - Kafka or RMQ
  • Experience with Gitlab based CI/CD
  • Experience with Data catalog

Added advantage:
  • AI/Client Engineer
  • Client Pipelines

Responsibilities:
  • Analyze and organize raw data
  • Build data systems and pipelines
  • Interpret trends and patterns in enterprise data
  • Conduct complex data analysis and report on results
  • Prepare data for prescriptive and predictive modeling
  • Combine raw information from different sources
  • Explore ways to enhance data quality and reliability
  • Identify opportunities for data acquisition
  • Collaborate with data scientists and architects on several projects
  • Work independently with different data owners and design data flows


  • Austin, United States Grid Dynamics Full time

    Description Position at Grid Dynamics Our customer is an American multinational technology company headquartered in Cupertino, California. Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world. On this project, we are working with bleeding-edge big data technologies to develop a...


  • Austin, United States Grid Dynamics Full time

    Description Position at Grid Dynamics Our customer is an American multinational technology company headquartered in Cupertino, California. Our customer is one of the world's largest technology companies based in Silicon Valley with operations all over the world. On this project, we are working with bleeding-edge big data technologies to develop a...

  • Systems Engineer

    2 weeks ago


    Austin, United States Hudson River Trading Full time

    Hudson River Trading (HRT) is looking for Systems Engineers to join our growing Research & Development team. This team builds and maintains an exceptionally large and growing distributed compute cluster, many-PB-scale storage systems, operating systems, automation software, and development tools. We are seeking a highly skilled engineer with expertise in...

  • Big Data Engineer

    4 weeks ago


    Austin, United States ShiftCode Analytics Full time

    Interview : Video Visa : All apart from H1b , cpt and TN This is hybrid from day-1. Description : Requirements: 5+ years' experience in data engineering work Previous experience as a Big Data Engineer In-depth knowledge of Hadoop, Spark, Airflow Technical expertise with data models, data mining, and segmentation techniques Hands-on experience with SQL...

  • Big Data Engineer

    1 week ago


    Austin, United States ShiftCode Analytics Full time

    Interview : Video Visa : All apart from H1b , cpt and TN This is hybrid from day-1. Description : Requirements: 5+ years' experience in data engineering work Previous experience as a Big Data Engineer In-depth knowledge of Hadoop, Spark, Airflow Technical expertise with data models, data mining, and segmentation techniques Hands-on experience with SQL...

  • Big Data Engineer

    2 days ago


    Austin, United States Saxon Global Full time

    This person is a key member of the TXDOT Enterprise Data Architecture team, and this resource will develop the requisite data pipelines to load the TxDOT data warehouse and other analytical repositories to support advanced analytics by the business teams. They will use a combination of technologies and methods to (1) build a replication framework, (2) curate...


  • Austin, United States Diverse Lynx Full time

    Title: Big Data Engineer with cloud experience Location: Sunnyvale, CA/Austin, TX(Onsite) Experience: 10+ Years Technical/Functional Skills: Big Data, Tableau, Kafka, Iceberg, Spark Roles & Responsibilities: Having 10 years of professional experience fields of software Analysis, Design, Development, Deployment and Maintenance of software and Big Data...


  • Austin, United States Diverse Lynx Full time

    Title: Big Data Engineer with cloud experience Location: Sunnyvale, CA/Austin, TX(Onsite) Experience: 10+ Years Technical/Functional Skills: Big Data, Tableau, Kafka, Iceberg, Spark Roles & Responsibilities: Having 10 years of professional experience fields of software Analysis, Design, Development, Deployment and Maintenance of software and Big Data...


  • Austin, United States Diverse Lynx Full time

    Title: Big Data Engineer with cloud experience Location: Sunnyvale, CA/Austin, TX(Onsite) Experience: 10+ Years Technical/Functional Skills: Big Data, Tableau, Kafka, Iceberg, Spark Roles & Responsibilities: Having 10 years of professional experience fields of software Analysis, Design, Development, Deployment and Maintenance of software and Big Data...


  • Austin, United States Infosys Full time

    Infosys is seeking a Big Data Lead Developer. This position requires designing and developing highly usable and scalable new application capabilities and enhancing existing capabilities to meet the needs of our customers in a fast-paced agile development environment. You will have the opportunity to collaborate with some of the best talent in the industry to...

  • Data Engineer

    2 days ago


    Austin, United States Diverse Lynx Full time

    Role : Data Engineer Location: Austin/ Sunnyvale, CA (Onsite) Contract Key skills: Python, Kube, Big Data, AlgoCD and AirFlow * Experience in monitoring and troubleshooting large scale data platforms, data pipelines, complex backend services * Able to effectively communicate incidents, coordinate incident responses, document/manage runbooks, set up...


  • Austin, United States Diverse Lynx Full time

    Role: Big Data Platform Administration Location: Austin, TX/ Dallas, TX (Onsite) Contract Skills - Platform: HPE/MapR Hadoop (Big Data), Programming: Python, UNIX Scripting CICD: Ansible, Platform Support, ITSM tools knowledge, ITIL 2 12 Years of IT experience in Big Data, Hadoop platform management & min 2 years experience in Big Data platform...


  • Austin, United States Diverse Lynx Full time

    Role: Big Data Platform Administration Location: Austin, TX/ Dallas, TX (Onsite) Contract Skills - Platform: HPE/MapR Hadoop (Big Data), Programming: Python, UNIX Scripting CICD: Ansible, Platform Support, ITSM tools knowledge, ITIL 2 12 Years of IT experience in Big Data, Hadoop platform management & min 2 years experience in Big Data platform...


  • Austin, United States Diverse Lynx Full time

    Role: Big Data Platform Administration Location: Austin, TX/ Dallas, TX (Onsite) Contract Skills - Platform: HPE/MapR Hadoop (Big Data), Programming: Python, UNIX Scripting CICD: Ansible, Platform Support, ITSM tools knowledge, ITIL 2 12 Years of IT experience in Big Data, Hadoop platform management & min 2 years experience in Big Data platform...

  • Cloud Data Engineer

    2 weeks ago


    Austin, Texas, United States Apple Full time

    SummaryPosted: May 24, 2024Role Number: The people here at Apple don't just create products - they create the kind of wonder that's revolutionized entire industries. It's the diversity of those people and their ideas that inspires the innovation that runs through everything we do, from amazing technology to industry-leading environmental efforts. Apple is...

  • Data Engineer

    4 hours ago


    Austin, United States Avance Consulting Full time

    Job DescriptionJob DescriptionData Engineer (Day 1 onsite)Austin, TXMust to have skillsPythonPysparkSQLData EngineeringBig DataJob DescriptionWe're seeking a Data Engineer to take the lead in implementing and scaling datacollection, storage, processing, and filtering for fine-tuning large language models (LLMs) withinConversational Engineering. These...

  • Data Engineer

    2 hours ago


    Austin, United States Tech M USAAvance Consulting Full time

    Data Engineer (Day 1 onsite) Austin, TX Must to have skills Python Pyspark SQL Data Engineering Big Data Job Description We're seeking a Data Engineer to take the lead in implementing and scaling data collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering. These data pipelines are...

  • Data Engineer

    1 week ago


    Austin, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    2 days ago


    Austin, United States Loxo Full time

    As an early hire to our engineering team, you will be responsible for managing Loxo's data integration function. You will be primarily responsible for migrating new clients' legacy data from their previous recruitment platform to Loxo and will work closely with the Customer Success team to deliver a positive onboarding experience for all new Loxo users. You...

  • Data Engineer

    1 week ago


    Austin, United States RevOpsforce Full time

    Job DescriptionJob DescriptionAbout RevOpsforce:RevOpsforce is a revenue operations workforce management firm supporting clients to unlock revenue potential through solutions that better align your people, processes, data, and technology.We empower organizations with cutting-edge revenue operations management systems, seamlessly aligning sales, marketing,...