Big Data Engineer

5 days ago

Phoenix, United States SysMind Tech Full time
Skill: Big Data Engineer

Job Description:

Key Responsibilities:
  • Responsible for designing system solutions, developing custom applications, and modifying existing applications to meet distinct and changing business requirements. Handle coding, debugging, and documentation, as well working closely with SRE team. Provide post implementation and ongoing production support
  • Develop and design software applications, translating user needs into system architecture. Assess and validate application performance and integration of component systems and provide process flow diagrams. Test the engineering resilience of software and automation tools.
  • You will be challenged with identifying innovative ideas and proof of concept to deliver against the existing and future needs of our customers. Software Engineers who join our Loyalty Technology team will be assigned to one of several exciting teams that are developing a new, nimble, and modern loyalty platform which will support the key element of connecting with our customers where they are and how they choose to interact with American Express.
  • Be part of an enthusiastic, high performing technology team developing solutions to drive engagement and loyalty within our existing cardmember base and attract new customers to the Amex brand.
  • The position will also play a critical role partnering with other development teams, testing and quality, and production support, to meet implementation dates and allow smooth transition throughout the development life-cycle.
  • The successful candidate will be focused on building and executing against a strategy and roadmap focused on moving from monolithic, tightly coupled, batch-based legacy platforms to a loosely coupled, event-driven, microservices-based architecture to meet our long-term business goals.
Must Have Qualifications:
  • Bachelor's degree in engineering or computer science or equivalent OR Master's in Computer Applications or equivalent.
  • 5+ years of software development experience and leading teams of engineers and scrum teams
  • 3+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL and PySpark)
  • Hands-on experience on writing and understanding complex SQL(Hive/PySpark-dataframes), optimizing joins while processing huge amount of data
  • Experience in UNIX shell scripting
Additional Good to have requirements:
  • Solid Datawarehousing concepts
  • Knowledge of Financial reporting ecosystem will be a plus
  • Experience with Data Visualization tools like Tableau, SiSense, Looker
  • Expert on Distributed ecosystem
  • Hands-on experience with programming using Python/Scala
  • Expert on Hadoop and Spark Architecture and its working principle
  • Ability to design and develop optimized Data pipelines for batch and real time data processing
  • Should have experience in analysis, design, development, testing, and implementation of system applications
  • Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows
  • Aptitude for learning and applying programming concepts.
  • Ability to effectively communicate with internal and external business partners. Preferred Qualifications:
  • Knowledge of cloud platforms like GCP/AWS, building Microservices and scalable solutions, will be preferred
  • 2+ years of experience in designing and building solutions using Kafka streams or queues
  • Experience with GitHub and leveraging CI/CD pipelines
  • Experience with NoSQL i.e., HBase, Couchbase, MongoDB

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States iSoftTek Solutions Inc Full time

    Job Title: Big Data Engineer Job Location: Phoenix, AZ Job Type: Contract Job Description: iSoftTek Solutions Inc is seeking a talented Big Data Engineer to join our team. In this role, you will be responsible for developing and maintaining big data solutions and frameworks. You will work closely with cross-functional teams to design, implement, and optimize...

  • Big Data Engineer

    1 month ago

    Phoenix, Arizona, United States Group Nine LLC Full time

    Job Description:As a Big Data Developer at Group Nine LLC, you will be responsible for designing and developing data engineering solutions on big data ecosystems. You will work closely with the SRE team to provide post-implementation and ongoing support. Your key responsibilities will include developing and designing software applications, assessing and...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Syntricate Technologies. The ideal candidate will have extensive experience working with large-scale application development using the Big Data ecosystem, including Hadoop, Hive, Kafka, Spark, Scala, and Snowflake utilities.Key...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Syntricate Technologies. The ideal candidate will have extensive experience working with large-scale application development using the Big Data ecosystem, including Hadoop, Hive, and Kafka.Key Responsibilities:Design and develop scalable data...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Syntricate Technologies. The ideal candidate will have expertise in MapReduce, Spark, Hive, and SQL, with a strong background in data warehousing and analytics.Key Responsibilities:Design and develop big data applications using Spark, Hive, and...

  • Big Data Engineer

    2 weeks ago

    phoenix, United States Zensar Technologies Full time

    Job Summary:We are seeking a skilled and motivated Software Developer with a focus on Big Data Engineering to join our dynamic team. In this role, you will work with cutting-edge technologies in the Hadoop Ecosystem to develop, maintain, and optimize large-scale data processing systems. As part of our engineering team, you will be responsible for building...

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States Zensar Technologies Full time

    Job Summary:We are seeking a skilled and motivated Software Developer with a focus on Big Data Engineering to join our dynamic team. In this role, you will work with cutting-edge technologies in the Hadoop Ecosystem to develop, maintain, and optimize large-scale data processing systems. As part of our engineering team, you will be responsible for building...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Diverse Lynx Full time

    Job Description:At Diverse Lynx LLC, we are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have expertise in Big Data and Hadoop ecosystems, with a strong focus on MapR. Additionally, they will have hands-on experience with programming languages such as Python and Java, as well as application design and software...

  • Big Data engineer

    2 weeks ago

    Phoenix, United States Diverse Lynx Full time

    Role: Big Data engineer Location: Phoenix,AZ Experience: 8+ Year Duration: 12+ Months Essential Skills: Desirable Skills: Must have good experience working in large scale application development using Big Data ecosystem - Hadoop (HDFS, MapReduce, Yarn), Hive, Kafka. -- Should have hands on experience using Spark, Sacal & Spark SQL.-- Hands-on experience...

  • Big data Engineer

    2 weeks ago

    Phoenix, United States Syntricate Technologies Full time

    Required Skills: Big Data Engineer with mapreduce , spark , hive , sql skillset Expert in SQL and Data warehousing concepts. Hands-on experience with public cloud data warehouse (GCP, Azure, AWS). GCP certification will be very good to have. MapR experience is must Strong Hands on experience with one or more programming languages ( Python or Java). ...

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States Omega Hires Full time

    Job Title: Big Data Engineer Location: Phoenix AZ/ New York City, NY Duration: Long Term Contract Job Description: - 8+ years of experience in Big Data, Java, Spark, Hive, and MapReduce. - Strong experience PL/SQL with Unix/Linux and shell scripting. - Familiarity with distributed systems and large-scale data processing. - Strong problem-solving and...

  • Big Data Engineer

    5 days ago

    Phoenix, United States Omega Hires Full time

    Job Title: Big Data Engineer Location: Phoenix AZ/ New York City, NY Duration: Long Term ContractJob Description: - 8+ years of experience in Big Data, Java, Spark, Hive, and MapReduce. - Strong experience PL/SQL with Unix/Linux and shell scripting. - Familiarity with distributed systems and large-scale data processing. - Strong problem-solving and debugging...

  • Big Data Engineer

    5 months ago

    Phoenix, United States Lorven Technologies Full time

    Hi, Our client is looking Big Data Engineer  for Long term  project in Multi Location (USA)  below is the detailed requirements. Job Title: Big Data Engineer Location: Multi Location (USA) Duration: Contract / Full-Time Primary Skill: AWS Cloud Python programming and AWS services (EC2, RDS, S3, Lambda, . Airflow DAG Python...

  • Big Data Engineer

    4 weeks ago

    Phoenix, Arizona, United States Syntricate Technologies Full time

    Job Description:We are seeking a highly skilled Big Data Developer to join our team at Syntricate Technologies. The ideal candidate will have a strong background in handling database operations in Cassandra and Postgres, as well as experience with Pyspark and Python. Additionally, they should have hands-on experience with scheduling tools like Event Engine...

  • Big Data Engineer

    2 weeks ago

    Phoenix, United States Diverse Lynx Full time

    Role name: Developer Role Description: Big Data Engineer with mapreduce , spark , hive , sql skillset"Expert in SQL and Data warehousing concepts.Hands-on experience with public cloud data warehouse (GCP, Azure, AWS). GCP certification will be very good to have. MapR experience is mustStrong Hands on experience with one or more programming languages (...

  • Junior Big Data Engineer

    2 weeks ago

    Phoenix, United States Saxon Global Full time

    We will not consider any relocation options! We are looking for a candidate that is already local to the Phoenix, AZ area. Data Engineer role Looking for candidates with Google Cloud Platform experience and Java experience. Spark and scala experience are preferred. SQL experience is preferred. Looking for a Junior Software Engineer with strong analytical...

  • Big Data Engineer with Cloud Experience

    4 weeks ago

    Phoenix, Arizona, United States BS Technologies Inc. Full time

    Job Requirements:We are seeking a skilled Big Data Engineer with experience in Google Cloud Platform (GCP) to join our team at BS Technologies Inc.Key Responsibilities:Design and develop scalable big data solutions using GCP services.Collaborate with cross-functional teams to integrate big data solutions with existing systems.Ensure data quality, security,...

  • Engineer - Big Data - LUMI

    4 months ago

    Phoenix, United States American Express Full time

    You Lead the Way. We’ve Got Your Back. With the right backing, people and businesses have the power to progress in incredible ways. When you join Team Amex, you become part of a global and diverse community of colleagues with an unwavering commitment to back our customers, communities and each other. Here, you’ll learn and grow as we help you create...

  • Big Data Engineer with Cloud Expertise

    4 weeks ago

    Phoenix, Arizona, United States Diverse Lynx Full time

    Job Title: Big Data Engineer with Cloud ExpertiseAbout the Role:We are seeking a highly skilled Big Data Engineer with expertise in cloud technologies to join our team at Diverse Lynx LLC. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using cloud-based technologies.Key...

  • Big Data Developer

    4 weeks ago

    Phoenix, Arizona, United States Group Nine LLC Full time

    Job Description:Group Nine LLC is seeking a highly skilled Big Data Developer to join our Loyalty Technology team. As a key member of our team, you will be responsible for designing and developing data engineering solutions on big data ecosystems, developing custom applications, and modifying existing applications to meet the changing business requirements...