Big Data Engineer

2 weeks ago

phoenix, United States Zensar Technologies Full time

Job Summary:

We are seeking a skilled and motivated Software Developer with a focus on Big Data Engineering to join our dynamic team. In this role, you will work with cutting-edge technologies in the Hadoop Ecosystem to develop, maintain, and optimize large-scale data processing systems. As part of our engineering team, you will be responsible for building robust data pipelines, ensuring data quality, and integrating big data solutions into our systems.

Key Responsibilities:

  • Data Pipeline Development: Design, develop, and maintain scalable data pipelines using Hadoop, Spark, and related technologies to process and analyze large datasets.
  • Big Data Systems: Work with Hadoop Ecosystem tools such as HDFS, Hive, HBase, and Pig to manage and optimize data storage, retrieval, and processing.
  • ETL Design and Implementation: Build and optimize Extract, Transform, Load (ETL) workflows for high-volume data integration from various sources.
  • Performance Optimization: Analyze and tune the performance of large-scale data processing workflows and jobs in a distributed computing environment.
  • Collaboration: Collaborate with cross-functional teams, including Data Scientists, Analysts, and Product Managers, to gather requirements and deliver optimal data solutions.
  • Data Quality & Security: Ensure data integrity, quality, and security by implementing robust validation checks, monitoring, and logging mechanisms.
  • Innovation: Stay updated with the latest trends in Big Data technologies and continuously improve processes and systems with innovative solutions.

Required Skills & Qualifications:

  • Experience: experience working with Big Data technologies, particularly within the Hadoop Ecosystem (HDFS, Hive, Spark, HBase, Pig, Flume, etc.).
  • Programming Skills: Strong programming experience in Java, Scala, or Python for data processing.
  • Big Data Tools: Hands-on experience with Apache Spark, Hive, HBase, YARN, MapReduce, Kafka, and Flume.
  • ETL Frameworks: Experience in building ETL pipelines and working with tools like Apache Nifi, Talend, or similar.
  • Data Modeling & Warehousing: Knowledge of data warehousing, data modeling, and SQL (experience with NoSQL databases is a plus).
  • Cloud Experience: Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform and big data services like EMR, DataProc, etc.
  • Data Integration: Experience integrating data from multiple sources and working with structured and unstructured data.
  • Version Control & CI/CD: Familiarity with version control tools like Git and continuous integration/continuous deployment practices.
  • Problem-Solving & Analytical Skills: Strong troubleshooting skills and ability to work on complex data problems in a fast-paced environment.

Preferred Skills & Qualifications:

  • Familiarity with containerization technologies like Docker and orchestration tools like Kubernetes.
  • Experience working with Apache Kafka for real-time data streaming.
  • Understanding of machine learning concepts and integrating them into data pipelines (preferred but not required).

Education:

  • Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. Master’s degree is a plus.

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States iSoftTek Solutions Inc Full time

    Job Title: Big Data Engineer Job Location: Phoenix, AZ Job Type: Contract Job Description: iSoftTek Solutions Inc is seeking a talented Big Data Engineer to join our team. In this role, you will be responsible for developing and maintaining big data solutions and frameworks. You will work closely with cross-functional teams to design, implement, and optimize...

  • Big Data Engineer

    1 month ago

    Phoenix, Arizona, United States Group Nine LLC Full time

    Job Description:As a Big Data Developer at Group Nine LLC, you will be responsible for designing and developing data engineering solutions on big data ecosystems. You will work closely with the SRE team to provide post-implementation and ongoing support. Your key responsibilities will include developing and designing software applications, assessing and...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Syntricate Technologies. The ideal candidate will have extensive experience working with large-scale application development using the Big Data ecosystem, including Hadoop, Hive, Kafka, Spark, Scala, and Snowflake utilities.Key...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Syntricate Technologies. The ideal candidate will have extensive experience working with large-scale application development using the Big Data ecosystem, including Hadoop, Hive, and Kafka.Key Responsibilities:Design and develop scalable data...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Syntricate Technologies. The ideal candidate will have expertise in MapReduce, Spark, Hive, and SQL, with a strong background in data warehousing and analytics.Key Responsibilities:Design and develop big data applications using Spark, Hive, and...

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States Zensar Technologies Full time

    Job Summary:We are seeking a skilled and motivated Software Developer with a focus on Big Data Engineering to join our dynamic team. In this role, you will work with cutting-edge technologies in the Hadoop Ecosystem to develop, maintain, and optimize large-scale data processing systems. As part of our engineering team, you will be responsible for building...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Diverse Lynx Full time

    Job Description:At Diverse Lynx LLC, we are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have expertise in Big Data and Hadoop ecosystems, with a strong focus on MapR. Additionally, they will have hands-on experience with programming languages such as Python and Java, as well as application design and software...

  • Big Data engineer

    2 weeks ago

    Phoenix, United States Diverse Lynx Full time

    Role: Big Data engineer Location: Phoenix,AZ Experience: 8+ Year Duration: 12+ Months Essential Skills: Desirable Skills: Must have good experience working in large scale application development using Big Data ecosystem - Hadoop (HDFS, MapReduce, Yarn), Hive, Kafka. -- Should have hands on experience using Spark, Sacal & Spark SQL.-- Hands-on experience...

  • Big data Engineer

    2 weeks ago

    Phoenix, United States Syntricate Technologies Full time

    Required Skills: Big Data Engineer with mapreduce , spark , hive , sql skillset Expert in SQL and Data warehousing concepts. Hands-on experience with public cloud data warehouse (GCP, Azure, AWS). GCP certification will be very good to have. MapR experience is must Strong Hands on experience with one or more programming languages ( Python or Java). ...

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States Omega Hires Full time

    Job Title: Big Data Engineer Location: Phoenix AZ/ New York City, NY Duration: Long Term Contract Job Description: - 8+ years of experience in Big Data, Java, Spark, Hive, and MapReduce. - Strong experience PL/SQL with Unix/Linux and shell scripting. - Familiarity with distributed systems and large-scale data processing. - Strong problem-solving and...

  • Big Data Engineer

    5 days ago

    Phoenix, United States Omega Hires Full time

    Job Title: Big Data Engineer Location: Phoenix AZ/ New York City, NY Duration: Long Term ContractJob Description: - 8+ years of experience in Big Data, Java, Spark, Hive, and MapReduce. - Strong experience PL/SQL with Unix/Linux and shell scripting. - Familiarity with distributed systems and large-scale data processing. - Strong problem-solving and debugging...

  • Big Data Engineer

    5 months ago

    Phoenix, United States Lorven Technologies Full time

    Hi, Our client is looking Big Data Engineer  for Long term  project in Multi Location (USA)  below is the detailed requirements. Job Title: Big Data Engineer Location: Multi Location (USA) Duration: Contract / Full-Time Primary Skill: AWS Cloud Python programming and AWS services (EC2, RDS, S3, Lambda, . Airflow DAG Python...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Description:We are seeking a highly skilled Big Data Developer to join our team at Syntricate Technologies. The ideal candidate will have a strong background in handling database operations in Cassandra and Postgres, as well as experience with Pyspark and Python. Additionally, they should have hands-on experience with scheduling tools like Event Engine...

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States Diverse Lynx Full time

    Role name: Developer Role Description: Big Data Engineer with mapreduce , spark , hive , sql skillset"Expert in SQL and Data warehousing concepts.Hands-on experience with public cloud data warehouse (GCP, Azure, AWS). GCP certification will be very good to have. MapR experience is mustStrong Hands on experience with one or more programming languages (...

  • Junior Big Data Engineer

    2 weeks ago

    Phoenix, United States Saxon Global Full time

    We will not consider any relocation options! We are looking for a candidate that is already local to the Phoenix, AZ area. Data Engineer role Looking for candidates with Google Cloud Platform experience and Java experience. Spark and scala experience are preferred. SQL experience is preferred. Looking for a Junior Software Engineer with strong analytical...

  • Big Data Engineer with Cloud Experience

    4 weeks ago

    Phoenix, Arizona, United States BS Technologies Inc. Full time

    Job Requirements:We are seeking a skilled Big Data Engineer with experience in Google Cloud Platform (GCP) to join our team at BS Technologies Inc.Key Responsibilities:Design and develop scalable big data solutions using GCP services.Collaborate with cross-functional teams to integrate big data solutions with existing systems.Ensure data quality, security,...

  • Engineer - Big Data - LUMI

    4 months ago

    Phoenix, United States American Express Full time

    You Lead the Way. We’ve Got Your Back. With the right backing, people and businesses have the power to progress in incredible ways. When you join Team Amex, you become part of a global and diverse community of colleagues with an unwavering commitment to back our customers, communities and each other. Here, you’ll learn and grow as we help you create...

  • Big Data Engineer with Cloud Expertise

    4 weeks ago

    Phoenix, Arizona, United States Diverse Lynx Full time

    Job Title: Big Data Engineer with Cloud ExpertiseAbout the Role:We are seeking a highly skilled Big Data Engineer with expertise in cloud technologies to join our team at Diverse Lynx LLC. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using cloud-based technologies.Key...

  • Big Data Developer

    4 weeks ago

    Phoenix, Arizona, United States Group Nine LLC Full time

    Job Description:Group Nine LLC is seeking a highly skilled Big Data Developer to join our Loyalty Technology team. As a key member of our team, you will be responsible for designing and developing data engineering solutions on big data ecosystems, developing custom applications, and modifying existing applications to meet the changing business requirements...

  • Big Data Engineer

    5 days ago

    Phoenix, United States SysMind Tech Full time

    Skill: Big Data Engineer Job Description: Key Responsibilities: Responsible for designing system solutions, developing custom applications, and modifying existing applications to meet distinct and changing business requirements. Handle coding, debugging, and documentation, as well working closely with SRE team. Provide post implementation and ongoing...