Data Engineer

1 month ago


Phoenix, United States The IT Gigs Full time

Job title: Data Engineer

Location: Phoenix, AZ (Onsite)

Job type: W2 Only

VISA: USC and GC


JD:

Mandatory Skill Set: Apache Spark, Hive, Hadoop, BigQuery, BigTable, Cloud Composure, Dataflow, Google Cloud Storage, Python, SQL, Shell Scripting, Git.

Good to have Skill Set: CI/CD, Jenkins, Security and Networking, Scala, GCP Identity and Access Management (IAM).

Responsibilities:

Data Processing: Design, develop, and maintain scalable and efficient data processing pipelines using technologies such as Apache Spark, Hive, and Hadoop.

Programming Languages: Proficient in Python, Scala, SQL, and Shell Scripting for data processing, transformation, and automation.

Cloud Platform Expertise: Hands-on experience with Google Cloud Platform (GCP) services, including but not limited to BigQuery, BigTable, Cloud Composer, Dataflow, Google Cloud Storage, and Identity and Access Management (IAM).

Version Control and CI/CD: Implement and maintain version control using Git and establish continuous integration/continuous deployment (CI/CD) pipelines for data processing workflows.

Jenkins Integration: Experience with Jenkins for automating the building, testing, and deployment of data pipelines.

Data Modeling: Work on data modeling and database design to ensure optimal storage and retrieval of data.

Performance Optimization: Identify and implement performance optimization techniques for large-scale data processing.

Collaboration: Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to understand data requirements and deliver solutions.

Security and Networking: Possess basic knowledge of GCP Networking and GCP IAM to ensure secure and compliant data processing.

Documentation: Create and maintain comprehensive documentation for data engineering processes, workflows, and infrastructure.

Qualifications:

Proven experience with Apache Spark, Hive, and Hadoop.

Strong programming skills in Python, Scala, SQL, and Shell Scripting.

Hands-on experience with GCP services, including BigQuery, BigTable, Cloud Composer, Dataflow, Google Cloud Storage, and Identity and Access Management (IAM)

Familiarity with version control using Git and experience in implementing CI/CD pipelines.

Experience with Jenkins for automating data pipeline processes.

Basic understanding of GCP Networking.

Excellent problem-solving and analytical skills.

Strong communication and collaboration skills.


Please share your resume over shikha@theitgigs.com


  • Data Engineer

    2 days ago


    Phoenix, Arizona, United States Motion Recruitment Full time

    Job DescriptionWe are currently seeking a talented and motivated Data Engineer with expertise in Google Cloud Platform (GCP). This financial tech company offers the opportunity for growth and advancement in a dynamic and innovative work environment.The #1 feature of this opportunity is the chance to work with GCP, designing and implementing data solutions...

  • Data Engineer

    4 weeks ago


    Phoenix, United States eTek IT Services, Inc. Full time

    Job DescriptionJob DescriptionRequired SkillsBig Data Engineer with 3-4 years of hands-on Spark/Scala and Hive experience. The candidate must be able to work a hybrid schedule with the team in Phoenix, AZ (onsite every Tues/Wed/Thurs). If the candidate is not located in Phoenix, but wiling to relocate and work onsite, they can begin working remotely and then...


  • Phoenix, United States QTS Data Centers Full time

    Learn what makes QTS a unique place to grow your career! SUMMARY The Critical Operations Technician I is responsible for the installation, repair, and maintenance of IT, electrical, mechanical, plumbing/piping equipment, machinery and controls located in the interior and exterior of building. This role monitors operations and conducts a routine and ongoing...


  • Phoenix, United States Plex Consulting, LLC Full time

    Job DescriptionJob DescriptionJob Title: Data Engineering ConsultantCompany: Plex ConsultingLocation: Phoenix, AZAbout the company:Plex Consulting is a leading technology consulting firm offering data solutions to a diverse range of clients across industries. We combine our expertise in data engineering, analytics, and data science to provide innovative and...


  • Phoenix, United States Insight Global Full time

    Must be local to Arizona!DescriptionYou will be an independent contributor on a fast-growing Data Engineering team pursuing a vision of analytics-driven for our client. Your expertise in data engineering and software engineering will enable and empower our organization to build and deploy data driven solutions to production. At our client, they understand...


  • Phoenix, United States Insight Global Full time

    Must be local to Arizona!DescriptionYou will be an independent contributor on a fast-growing Data Engineering team pursuing a vision of analytics-driven for our client. Your expertise in data engineering and software engineering will enable and empower our organization to build and deploy data driven solutions to production. At our client, they understand...


  • Phoenix, United States NTT DATA Full time

    100% Remote NTT DATA's Client is seeking a Senior QA Automation Engineer with 6 or more years of experience of testing with RESTful services using Postman or Groovy scripts. 3 or more years of experience with Agile tools like Azure DevOps, JIRA, etc. 3 or more years of experience with CI/CD tools like ADO, Jenkins etc. 3 or more years of development...


  • Phoenix, United States Insight Global Full time

    Company: Amex Role: Data Software engineer Duration: 9 month contract to hire Location: Phoenix, FL OR Sunrise, Florida (3x a week) Must Have: -8+ years of Data Experience, working with NLP.-Proficient in Python coding and programming.-Exposure with speech to text functions and exposure into AI.Day-to-day: A large financial enterprise is looking for a Senior...


  • Phoenix, United States Insight Global Full time

    Company: Amex Role: Data Software engineer Duration: 9 month contract to hire Location: Phoenix, FL OR Sunrise, Florida (3x a week) Must Have: -8+ years of Data Experience, working with NLP.-Proficient in Python coding and programming.-Exposure with speech to text functions and exposure into AI.Day-to-day: A large financial enterprise is looking for a Senior...

  • Data Engineer

    4 weeks ago


    Phoenix, United States Forhyre Full time

    Job DescriptionJob DescriptionWe are looking for a passionate certified Data Engineer. The successful candidate will turn data into information, information into insight and insight into business decisions.Data analyst responsibilities include conducting full lifecycle analysis to include requirements, activities and design. Data analysts will develop...


  • Phoenix, United States Western Alliance Bank Full time

    Principal Data Engineer 1 with Western Alliance Bank (Phoenix, AZ) Job Description: Design, build and implement critical components of the enterprise data platform in accordance with the strategic data and analytics needs of the organization. Develop a cross-business unit data tier that is leveraged to improve products and deliver insights. Serve as a...

  • GCP Data Engineer

    4 weeks ago


    Phoenix, United States Triunity Software Full time

    Job DescriptionJob DescriptionNature: Day One On-siteDuration: 24 MonthsCandidates Required: 04Experience: 5 to 8 Years Following is the Job description for the role of a Data Engineer,Mandatory Skill Set: Apache Spark, Hive, Hadoop, BigQuery, BigTable, Cloud Composure, Dataflow, Google Cloud Storage, Python, SQL, Shell Scripting, Git.Good to have Skill Set:...

  • Data Engineer

    1 week ago


    Phoenix, United States The IT Gigs Full time

    Job title: Data Engineer/Data ArchitectLocation: Phoenix, AZ (Onsite)/ CaliforniaJob type: W2 OnlyVISA: USC, GC , OPT11+ years of experience OnlyJD:MUST HAVE AWS CERTIFICATIONSQualifications:Proven experience with Apache Spark, Hive, and Hadoop.Strong programming skills in Python, Scala, SQL, and Shell Scripting.Hands-on experience with GCP services,...

  • Data Engineer

    5 days ago


    Phoenix, United States The IT Gigs Full time

    Job title: Data Engineer/Data ArchitectLocation: Phoenix, AZ (Onsite)/ CaliforniaJob type: W2 OnlyVISA: USC, GC , OPT11+ years of experience OnlyJD:MUST HAVE AWS CERTIFICATIONSQualifications:Proven experience with Apache Spark, Hive, and Hadoop.Strong programming skills in Python, Scala, SQL, and Shell Scripting.Hands-on experience with GCP services,...


  • Phoenix, United States Recurring Decimal Full time

    Data Engineer Location: Phoenix, AZ or Los Angeles, CAHybridCompliance in Banking Domain knowledge is a mustKnowledge of cloud platforms (AWS, Azure, GCP) for data analysisand management is a mustProven experience (5+ years) as a Business Analyst, Data Engineer, or similar role within the financial industry, with a strong focus on compliance and regulatory...


  • Phoenix, United States Recurring Decimal Full time

    Data Engineer Location: Phoenix, AZ or Los Angeles, CAHybridCompliance in Banking Domain knowledge is a mustKnowledge of cloud platforms (AWS, Azure, GCP) for data analysisand management is a mustProven experience (5+ years) as a Business Analyst, Data Engineer, or similar role within the financial industry, with a strong focus on compliance and regulatory...


  • Phoenix, United States Recurring Decimal Full time

    Data Engineer Location: Phoenix, AZ or Los Angeles, CAHybridCompliance in Banking Domain knowledge is a mustKnowledge of cloud platforms (AWS, Azure, GCP) for data analysisand management is a mustProven experience (5+ years) as a Business Analyst, Data Engineer, or similar role within the financial industry, with a strong focus on compliance and regulatory...

  • Data Engineer

    4 weeks ago


    Phoenix, United States Tekdoors Full time

    Job DescriptionJob DescriptionJob Title: Data Engineer Location: Jacksonville, FL / Phoenix, AZ (Hybrid Work) Duration: 24+ Months Employment Type: Contract W2 Experience: 8+ YearsMust-Have: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Proven experience as a Data Engineer with a minimum of 8 years in...

  • AI/ML Data Engineer

    4 days ago


    Phoenix, United States Insight Global Full time

    Position: AI/ML Data EngineerLocation: Hybrid in Phoenix, AZ (3 days/week on site)Duration: 9 month contract to hireRequired Skills and Experience *-8+ years of hands on Data Engineering experience.-Very well versed with AI and Machine Learning.-Proficient with Application Lifecycle Management (ALM)-Understanding of public cloud environmentsJob Description...

  • AI/ML Data Engineer

    4 days ago


    Phoenix, United States Insight Global Full time

    Position: AI/ML Data EngineerLocation: Hybrid in Phoenix, AZ (3 days/week on site)Duration: 9 month contract to hireRequired Skills and Experience *-8+ years of hands on Data Engineering experience.-Very well versed with AI and Machine Learning.-Proficient with Application Lifecycle Management (ALM)-Understanding of public cloud environmentsJob Description...