Big Data Engineer

6 months ago


Austin, United States ShiftCode Analytics Full time

Interview : Video

Visa : All apart from H1b , cpt and TN

This is hybrid from day-1.

Description :

Requirements:

  • 5+ years' experience in data engineering work
  • Previous experience as a Big Data Engineer
  • In-depth knowledge of Hadoop, Spark, Airflow
  • Technical expertise with data models, data mining, and segmentation techniques
  • Hands-on experience with SQL database, Redis
  • Familiarity with Kubernetes and Docker tools
  • Familiarity with S3, Apache Parquet and Apache Iceberg
  • Understand how to extract data
  • Experience with Graph databases
  • Knowledge of programming languages (e.g. Python)
  • Experience with Streaming - Kafka or RMQ
  • Experience with Gitlab based CI/CD
  • Experience with Data catalog

Added advantage:
  • AI/Client Engineer
  • Client Pipelines

Responsibilities:
  • Analyze and organize raw data
  • Build data systems and pipelines
  • Interpret trends and patterns in enterprise data
  • Conduct complex data analysis and report on results
  • Prepare data for prescriptive and predictive modeling
  • Combine raw information from different sources
  • Explore ways to enhance data quality and reliability
  • Identify opportunities for data acquisition
  • Collaborate with data scientists and architects on several projects
  • Work independently with different data owners and design data flows


  • Austin, United States Futran Tech Solutions Pvt. Ltd. Full time

    Role :- Big Data with Java Data Engineer - Hadoop with Java Python, Hdfs, Hadoop Python, Hdfs, Hadoop Nice To Have Skills (optional) AWS, Airflow, Oozie Role Production support Enhancements & bug fixes New Feature development

  • Big data Engineer

    4 weeks ago


    Austin, United States Omega Solutions Inc Full time

    Interview Process: 3 Rounds of Zoom Video Interview Required: • 8 years experience in BIG data development such as Hadoop/spark/hive • 6+ years of hands on experience on AWS cloud • 6+ years of experience in SNOWFLAKE • 8+ years of experience in Python and data science libraries on data validation

  • Big Data Tester

    2 months ago


    Austin, United States United Software Group Full time

    Role: Big Data Tester Location: - Austin, TX / Sunnyvale, CA ( Day 1 Onsite ) Duration: Fulltime Role Job Description - Big Data Testing Requirement API Testing/ JAVA Backend exp. good to have Skill Requirements: Technical Experts in Big data , Spark, Hive, Java 6-8 years of industry experience Responsibilities Communicate project status and relevant...


  • Austin, United States United Software Group Full time

    Architect - Big Data & Python Austin, TX - Day one on site Must have skills: • Big Data, Spark, and Python • Kafka • Development experience in cloud technologies Job Description: In-depth knowledge of integration patterns, integration technologies and integration platforms Good programming skills in Python Experience with Queuing related...

  • Data Engineer

    4 months ago


    Austin, United States Diverse Lynx Full time

    Key Skills: Snowflake, Big data processing, Python, pyspark, ETL. Job Description: Engineers they need is more for Data Engineering work (90% Data Engg and 10% Client model training). From Data engineering, we need folks with experience in Big data processing, Python, pyspark, ETL, feature engineering etc . Diverse Lynx LLC is an Equal Employment...


  • Austin, United States Maintec Technologies Full time

    ROLE: - Sr. Big Data Developer LOCATION: - Austin/Remote DURATION: - Long term contract MUST SKILLS: - Python programming, Pandas, NumPy, PySpark, ETL testing, AWS glue, ec2 and lambda, SQL queries Job Description: 10+ years of hands-on and strong experience in python programming 10+ years of experience in developing test automation utilities for data...

  • Big Data Developer

    4 weeks ago


    Austin, United States Futran Tech Solutions Pvt. Ltd. Full time

    Big Data Developer Austin, TX (Day 1 onsite) - Full-time Primary Skillset: Spark, Scala, AWS Secondary Skillset : Python, Kafka Job Description: Common data archetypes, writing and coding functions, algorithms, logic development, control flow, object-oriented programming languages, external libraries and how to collect data from different sources. This...


  • Austin, United States Info Way Solutions Full time

    Role: Core Java with Big data Location: Austin, TX (Onsite) Type: Contract Job Description: 10+ years of experience Good Knowledge on Java Working on Bigdata, Kafka, Mongo DB. Good coding skills Springboot Sql queries


  • Austin, United States Expedia Partner Solutions Full time

    If you need assistance during the recruiting process due to a disability, please reach out to our Recruiting Accommodations Team through the Accommodation Request form. This form is used only by individuals with disabilities who require assistance or adjustments in applying and interviewing for a job. This form is not for inquiring about a position or the...


  • Austin, United States Tekgence Inc Full time

    Data Engineering ManagerAustin, TX – HybridFulltimeShould not require sponsorship to work in the US, now, or in the future. Minimum 15+ years of experience. Hands on Manager.Technology stack:Strong Cloud experience required but open to which cloud provider – AWS, Azure, GCPPower BI Experience writing complex queries and DAXExperience with large data sets...


  • austin, United States Tekgence Inc Full time

    Data Engineering ManagerAustin, TX – HybridFulltimeShould not require sponsorship to work in the US, now, or in the future. Minimum 15+ years of experience. Hands on Manager.Technology stack:Strong Cloud experience required but open to which cloud provider – AWS, Azure, GCPPower BI Experience writing complex queries and DAXExperience with large data sets...

  • Data Engineer

    2 months ago


    Austin, United States Visa Full time

    Company Description Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure...


  • Austin, United States Charles Schwab Full time

    Position Type: RegularYour opportunity At Schwab, you’re empowered to make an impact on your career. Here, innovative thought meets creative problem solving, helping us “challenge the status quo” and transform the finance industry together. Duties: Develop, maintain, and support large-scale and complex ETL and distributed Big Data applications....

  • Senior Data Engineer

    2 weeks ago


    Austin, TX, United States Harnham Full time

    Title: Senior Data EngineerLocation: US-based - Remote - Company based in Austin, TXOverview of Role:As Senior Data Engineer, you will be responsible for driving the design, development, implementation, and enhancement of our platforms. We are looking for someone who has good tenure, experience with sensitive data, and EXTENSIVE SQL experience.Role...

  • Cloud Data Engineer

    4 weeks ago


    Austin, United States Unreal Gigs Full time

    Are you passionate about building and optimizing data pipelines that drive data-driven decision-making in cloud environments? Do you have the technical expertise to design robust data architectures that manage large-scale data processing and ensure seamless data flow? If you're ready to harness the power of cloud technology to create scalable data solutions,...

  • Senior Data Engineer

    3 weeks ago


    Austin, United States MNTN Full time

    Job DescriptionJob DescriptionAt MNTN, we've built a culture based on quality, trust, ambition, and accountability – but most importantly, we really enjoy working here. We pride ourselves on our self-service platform and are constantly seeking to improve the user experience for our customers and scale for efficiency. Our startup spirit powers our...

  • Data Quality Engineer

    4 weeks ago


    Austin, United States MVP Group Inc Full time

    Job DescriptionJob DescriptionAbout MVP:Headquartered in the heart of Austin, Texas, MVP is the best-in-class SaaS measurement platform that provides omnichannel media and sponsorship valuation. We offer a built-to-scale software for our clients--Fortune 1,000 brands, professional sports, global agencies and media--to accurately quantify the value of their...

  • Senior Data Engineer

    6 months ago


    Austin, United States 2K Full time

    Who We Are 2K is headquartered in Novato, California and is a wholly owned label of Take-Two Interactive Software, Inc. (NASDAQ: TTWO). Founded in 2005, 2K Games is a global video game company, publishing titles developed by some of the most influential game development studios in the world. Our studios responsible for developing 2K’s portfolio of...


  • Austin, Texas, United States Apple Full time

    Cloud Data EngineerAt Apple, we're revolutionizing industries with innovative technology and environmental efforts. We're seeking an experienced Cloud Data Engineer to join our worldwide business development and strategy team. This role will play a crucial part in defining and delivering cloud-based analytical and big data solutions that impact our...

  • Data Engineer

    1 month ago


    austin, United States Intelliswift Software Full time

    Austin TX - Hybrid 3 days in office Must HaveAWS Cloud servicesData EngineeringPython or ScalaREQUIRED SKILLSLooking for Sr. Data engineer Very comfortable with Data modeling, data warehousing ETL, and Scripting. Orchestration, Data Pipeline, Taxonomy Python, Scala or Spark, SQL Looking for someone who is already AWS cloud services like S3, lambda, glue,...