See more Collapse

Lead Data Engineer

1 month ago

Yuma, United States Godspeed Workforce Full time
Job DescriptionJob Description

Job Title: Lead Data Engineer

Location: Scottsdale, AZ (100% onsite)

Hire Type: Contract/FTE/CTH

VISA: No OPT, CPT, H1B

Must-have skill set:

  • Spark, S3, Glue, AWS Redshift, Python, and StreamSets experience

Description:

We are seeking a highly skilled Lead Data Engineer to join our team in Scottsdale, AZ. The ideal candidate will have extensive experience in enterprise data architecture and management, with proficiency in Spark, S3, Glue, AWS Redshift, Python, and StreamSets.

Responsibilities:

  • Lead and architect enterprise-wide initiatives including system integration, data migration, transformation, data warehouse build, data mart build, and data lakes implementation/support.
  • Conceptualize, design, and implement streaming data pipelines using Spark, Structured Streaming, Delta Lake, and Databricks.
  • Develop and maintain data models including conceptual, logical, and physical data modeling, with expertise in relational and dimensional data modeling.
  • Troubleshoot and debug complex technical issues related to data pipelines and infrastructure.
  • Collaborate with cross-functional teams to ensure the successful deployment and optimization of big data applications in a high-performance enterprise environment.
  • Utilize advanced SQL skills for data manipulation, performance optimization, and RDBMS schema design.
  • Implement and maintain ETL/ELT processes using tools such as Data Migration Service.
  • Work with AWS environment, Lambda, S3, and other cloud services for data processing and storage.
  • Familiarity with data quality/validation frameworks such as Great Expectations is a plus.
  • Experience with Gitlabs, CloudWatch, CI/CD pipelines, and Lambda configuration and optimization.
  • Communicate effectively with technology partners, stakeholders, team members, and senior management to articulate technology solutions and constraints.

Qualifications:

  • Bachelor's degree in computer science, Information Technology, or related field.
  • 6-8 years of IT experience focusing on enterprise data architecture and management.
  • Proficiency in Spark, S3, Glue, AWS Redshift, Python, and StreamSets.
  • Experience with Databricks, Structured Streaming, Delta Lake, and Delta Live Tables.
  • Advanced understanding of streaming data pipelines and ETL/ELT processes.
  • Strong SQL skills with experience in RDBMS schema design and optimization.
  • Experience working in a team-based environment and ability to thrive in a fast-paced, dynamic organization.

If you possess the required skills and experience and are eager to join a dynamic team at Persistent Systems, we encourage you to apply for this exciting opportunity.

We have other current jobs related to this field that you can find below

  • Sr. Data Engineer

    4 weeks ago

    Yuma, United States Delta System and Software Inc. Full time

    Job DescriptionJob DescriptionJob Title Sr. Data EngineerLocation 5 Days Onsite at 18700 North Hayden Road, Scottsdale, AZDuration- 12 MonthsClient: Accenture- Jenius(SMBC) Bank Visa: USC/GC only Responsibilities:Job Summary & Principal Duties: A solid experience and understanding of considerations for large-scale solutioning and operationalization of data...

  • Big Data Engineer

    4 weeks ago

    Yuma, United States Tech3pillars Technologies Full time

    Job DescriptionJob DescriptionJob Title: Big Data EngineerLocation: Scottsdale, AZ (onsite)Must have Skills:In-depth Big Data, Hadoop, Hive Queries in AWS, Groovy Scripting, testing, test automation, debugging. REST API testing in AWS, Data and Data services testing, Spark Scala jobs testing and implementation in Hadoop,Java programmingGood to have...

  • Data Systems Manager

    1 week ago

    Yuma, United States Gowan Company Full time

    The Data Systems Manager will initially support the PLM architect with all aspects related to the implementation of Optiva PLM, the product life cycle application which will centralize and manage all data, processes, and collaboration related to a product throughout its lifecycle. They will be instrumental in identifying and working with key stakeholders to...

  • Data Systems Manager

    1 week ago

    Yuma, United States Gowan Company Full time

    The Data Systems Manager will initially support the PLM architect with all aspects related to the implementation of Optiva PLM, the product life cycle application which will centralize and manage all data, processes, and collaboration related to a product throughout its lifecycle. They will be instrumental in identifying and working with key stakeholders to...

  • Data Architect

    2 weeks ago

    Yuma, United States RBC Technology Solutions LLC Full time

    Job DescriptionJob DescriptionPosition: Data ArchitectLocation : Arizona (Day1 Onsite).Duration : 6+ monthsPrimary Skills: Azure Data Factory ,Microsoft Fabric,data modellingJob Description:We need an architect who is having minimum of 7 years of experienceData Architect who has extensive experience in bringing SAP data into Azure. The ideal candidate will...

  • Data Architect

    7 days ago

    Yuma, United States RBC Technology Solutions LLC Full time

    Job DescriptionJob DescriptionPosition: Data ArchitectLocation : Arizona (Day1 Onsite).Duration : 6+ monthsPrimary Skills: Azure Data Factory ,Microsoft Fabric,data modellingJob Description:We need an architect who is having minimum of 7 years of experienceData Architect who has extensive experience in bringing SAP data into Azure. The ideal candidate will...

  • Lead/Senior Application Engineer, Logistics

    2 weeks ago

    Yuma, United States Intone Networks Inc Full time

    Job DescriptionJob DescriptionHello!I have a new position I'm working! Please review the details below and let me know who you think would be a good fit.Please send resume, VISA, and confirm the best rate. Whoever does this will be considered first. Thank you!Please Note: As of July 22, 2021, our team will require that all candidate submissions include a...

  • Lead/Senior Application Engineer, Logistics

    12 hours ago

    Yuma, United States Intone Networks Inc Full time

    Job DescriptionJob DescriptionHello!I have a new position I'm working! Please review the details below and let me know who you think would be a good fit.Please send resume, VISA, and confirm the best rate. Whoever does this will be considered first. Thank you!Please Note: As of July 22, 2021, our team will require that all candidate submissions include a...

  • Lead/Senior Application Engineer, Logistics

    4 days ago

    Yuma, United States Intone Networks Inc Full time

    Job DescriptionJob DescriptionHello!I have a new position I'm working! Please review the details below and let me know who you think would be a good fit.Please send resume, VISA, and confirm the best rate. Whoever does this will be considered first. Thank you!Please Note: As of July 22, 2021, our team will require that all candidate submissions include a...

  • GCP Data Engineer1

    1 month ago

    Yuma, United States Centraprise LLC Full time

    Job DescriptionJob DescriptionPlease share your interest and resume at rahul.j@centraprise.comPosition : GCP Data Engineer Location : Phoenix, AZ (Day 1 Onsite) Duration : Long Term Contract Job Description: Skills required: GCP - Data flow, Data proc, Apache Beam skills 6-9 Years of hands on Exp with GCP - Data flow, Data proc, Apache Beam skills Writing...

  • GCP Data Engineer1

    4 weeks ago

    Yuma, United States Centraprise LLC Full time

    Job DescriptionJob DescriptionPlease share your interest and resume at rahul.j@centraprise.comPosition : GCP Data Engineer Location : Phoenix, AZ (Day 1 Onsite) Duration : Long Term Contract Job Description: Skills required: GCP - Data flow, Data proc, Apache Beam skills 6-9 Years of hands on Exp with GCP - Data flow, Data proc, Apache Beam skills Writing...

  • Sr AWS Data Developer

    4 weeks ago

    Yuma, United States Intone Networks Inc Full time

    Job DescriptionJob DescriptionPlease find the new open requirement for an AWS data architect.Role: Sr AWS Data Developer Client: CavaLocation: Phoenix, ArizonaDuration: 24 Months Experience: 12 YearsWork Authorisation: USC, Green Card, H1B (I94 travel approved)Open Requirements: 01Nature: Day one on-siteNote: The proposed candidate has to be on the...

  • Sr AWS Data Developer

    1 month ago

    Yuma, United States Intone Networks Inc Full time

    Job DescriptionJob DescriptionPlease find the new open requirement for an AWS data architect.Role: Sr AWS Data Developer Client: CavaLocation: Phoenix, ArizonaDuration: 24 Months Experience: 12 YearsWork Authorisation: USC, Green Card, H1B (I94 travel approved)Open Requirements: 01Nature: Day one on-siteNote: The proposed candidate has to be on the...

  • Sr AWS Data Developer

    2 weeks ago

    Yuma, Arizona, United States Intone Networks Inc Full time

    Job DescriptionJob DescriptionPlease find the new open requirement for an AWS data architect.Role: Sr AWS Data Developer Client: CavaLocation: Phoenix, ArizonaDuration: 24 Months Experience: 12 YearsWork Authorisation: USC, Green Card, H1B (I94 travel approved)Open Requirements: 01Nature: Day one on-siteNote: The proposed candidate has to be on the vendor's...

  • Data Security Architect

    8 hours ago

    Yuma, United States TagStaff Full time

    Job DescriptionJob DescriptionThe IS Data Security Architect plays a pivotal role in safeguarding the organization's data assets against unauthorized access, disclosure, alteration, and destruction. You will be responsible for the architectural design, development, and maintenance of comprehensive security frameworks that ensure data protection aligns...

  • Operational Test Engineer

    4 weeks ago

    Yuma, United States ORSA Technologies LLC Full time

    Job DescriptionJob DescriptionORSA Technologies is seeking experienced Operational Test Engineer (OTE)Responsibilities include:Plans and administers performance tests on aeronautical equipment, controls, systems, and other products and systems.Designs and directs engineering and technical personnel in testing and test-control apparatus and...

  • Operational Test Engineer

    4 weeks ago

    Yuma, United States ORSA Technologies LLC Full time

    Job DescriptionJob DescriptionORSA Technologies is seeking experienced Operational Test Engineer (OTE)Responsibilities include:Plans and administers performance tests on aeronautical equipment, controls, systems, and other products and systems.Designs and directs engineering and technical personnel in testing and test-control apparatus and...

  • Principal Electrical Engineer

    3 months ago

    Yuma, United States Pray & Company HR and Business Solutions, LLC Full time

    Job DescriptionJob DescriptionSummary/Description:Lead and support theElectricalEngineering team for projects in the US MMM (Mining, Minerals, and Metals) business. Manage small to large projects, ensuring quality, cost, and schedule adherence. Responsible for theElectricalEngineering team's output, technical integrity, and compliance with safety...

  • Senior Flight Test Engineer

    1 month ago

    Yuma, United States ORSA Technologies LLC Full time

    JOB DESCRIPTION: Senior Flight Test Engineer Orsa Technologies is seeking qualified candidates to provide Senior Engineer Operational Flight Test and Evaluation to Marine Operational Test and Evaluation Squadron One. The Operational Test and Evaluation Engineer will conduct operational tests and evaluations to ensure the effectiveness and suitability of...

  • Senior Flight Test Engineer

    4 weeks ago

    Yuma, United States ORSA Technologies LLC Full time

    Job DescriptionJob DescriptionJOB DESCRIPTION: Senior Flight Test EngineerOrsa Technologies is seeking qualified candidates to provide Senior Engineer Operational Flight Test and Evaluation to Marine Operational Test and Evaluation Squadron One. The Operational Test and Evaluation Engineer will conduct operational tests and evaluations to ensure the...