Lead Data Engineer

2 weeks ago

Phoenix, United States JobRialto Full time
Description

8-12 years of experience with Python, SQL, Databricks or Snowflake or Fabric. Spark tech stack is required. Azure preferred, but open to to GCP, AWS. Person is leading junior engineers who are all reporting to hiring manager.

Additional Skills

Certifications, 2+ years of data architecture, data engineering patterns, git repo, software engineering background, ML experience

Job Description

As the Technical Lead Data Engineer, your primary responsibility will be to spearhead the design, development, and implementation of data solutions aimed at empowering our organization to derive actionable insights from intricate datasets. You will take the lead in guiding a team of data engineers, fostering collaboration with cross-functional teams, and spearheading initiatives geared towards fortifying our data infrastructure, CI/CD pipelines, and analytics capabilities

Responsibilities:

Apply advanced knowledge of Data Engineering principles, methodologies and techniques to design and implement data loading and aggregation frameworks across broad areas of the organization.

Gather and process raw, structured, semi-structured and unstructured data using batch and real-time data processing frameworks.

Implement and optimize data solutions in enterprise data warehouses and big data repositories, focusing primarily on movement to the cloud.

Drive new and enhanced capabilities to Enterprise Data Platform partners to meet the needs of product / engineering / business.

Experience building enterprise systems especially using Databricks, Snowflake and platforms like Azure, AWS, GCP etc

Leverage strong Python, Spark, SQL programming skills to construct robust pipelines for efficient data processing and analysis.

Implement CI/CD pipelines for automating build, test, and deployment processes to accelerate the delivery of data solutions.

Implement data modeling techniques to design and optimize data schemas, ensuring data integrity and performance.

Drive continuous improvement initiatives to enhance performance, reliability, and scalability of our data infrastructure.

Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions.

Implement best practices for data governance, security, and compliance to ensure the integrity and confidentiality of our data assets.

Qualifications:

Bachelor's or master's degree in computer science, Engineering, or a related field.

Proven experience (8+) in a data engineering role, with expertise in designing and building data pipelines, ETL processes, and data warehouses.

Strong proficiency in SQL, Python and Spark programming languages.

Strong experience with cloud platforms such as AWS, Azure, or GCP is a must.

Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.

Knowledge of data lake and data warehouse solutions, including Databricks, Snowflake, Amazon Redshift, Google BigQuery, Azure Data Factory, Airflow etc.

Experience in implementing CI/CD pipelines for automating build, test, and deployment processes.

Solid understanding of data modeling concepts, data warehousing architectures, and data management best practices.

Education: Bachelors Degree

Additional client information:

  • Migration Data Engineer Lead

    3 weeks ago

    Phoenix, United States Compunnel Inc. Full time

    Job title: Migration Data Engineer LeadLocation: 3151 W Behrend Dr, Phoenix, AZ 85027Duration: 12 MonthsJob DescriptionData on Cloud project aimed at leveraging Google BigQuery for efficient data management. Responsibilities: As an Entry-Level to Mid-Level Application Lead, you'll use your expertise in Google BigQuery to design and implement effective data...

  • Migration Data Engineer Lead

    3 weeks ago

    Phoenix, United States Compunnel Inc. Full time

    Job title: Migration Data Engineer LeadLocation: 3151 W Behrend Dr, Phoenix, AZ 85027Duration: 12 MonthsJob DescriptionData on Cloud project aimed at leveraging Google BigQuery for efficient data management. Responsibilities: As an Entry-Level to Mid-Level Application Lead, you'll use your expertise in Google BigQuery to design and implement effective data...

  • Data Engineering

    2 weeks ago

    Phoenix, United States Diverse Lynx Full time

    Data Engineer- Oracle Fusion cloud: Learning & Talent AZ-Phoenix-85054 Relevant Experience(in yrs) = 5-8 years Technical/Functional Skills • Must have 5-8 years of Data Engineering experience in a Big Data environment. • Strong Data transformation/ETL skills using Spark, Python, PySpark, HiveQL • Must have hands on experience building data ingestion...

  • Critical Operations Tech Overnight

    1 month ago

    Phoenix, United States QTS Data Centers Full time

    Learn what makes QTS a unique place to grow your career! SUMMARY The Critical Operations Technician I is responsible for the installation, repair, and maintenance of IT, electrical, mechanical, plumbing/piping equipment, machinery and controls located in the interior and exterior of building. This role monitors operations and conducts a routine and ongoing...

  • Data Engineering Consultant

    1 month ago

    Phoenix, United States Plex Consulting, LLC Full time

    Job DescriptionJob DescriptionJob Title: Data Engineering ConsultantCompany: Plex ConsultingLocation: Phoenix, AZAbout the company:Plex Consulting is a leading technology consulting firm offering data solutions to a diverse range of clients across industries. We combine our expertise in data engineering, analytics, and data science to provide innovative and...

  • Senior Lead Mechanical Engineer

    1 month ago

    Phoenix, United States WSP Full time

    This Opportunity kW Mission Critical Engineering, a member of WSP USA, is currently initiating a search for a Senior Lead Mechanical Engineer for our kW Tempe, Arizona office: 40 E. Rio Salado Parkway, Suite 445 Tempe AZ 85281. As a Senior Lead Mechanical Engineer with us, you will design complex cooling and HVAC systems including air distribution...

  • Data Engineer

    2 weeks ago

    Phoenix, United States Cogliano IT Staffing Full time

    Data Engineer needed for a Direct Hire position in Phoenix, AZ! Position will be remote, but candidate needs to live within a reasonable distance of Phoenix. You will be project leader and independent contributor on a fast-growing Data Engineering team pursuing a vision of analytics-driven company. Your expertise in data engineering and software engineering...

  • Lead Engineering Technician

    2 weeks ago

    Phoenix, United States NESC Staffing Full time

    Lead Engineering Technician Provide laboratory support for the Flight Controls Compact Fly-by-Wire engineering team. Your work will include support for lab support, testing, data analysis, and the accurate reporting of data. You will support the team to conceptualize, design, plan and implement innovative technologies that enable new and competitive...

  • Data Engineer

    4 weeks ago

    Phoenix, United States eTek IT Services, Inc. Full time

    Job DescriptionJob DescriptionRequired SkillsBig Data Engineer with 3-4 years of hands-on Spark/Scala and Hive experience. The candidate must be able to work a hybrid schedule with the team in Phoenix, AZ (onsite every Tues/Wed/Thurs). If the candidate is not located in Phoenix, but wiling to relocate and work onsite, they can begin working remotely and then...

  • Data Engineer

    3 weeks ago

    Phoenix, United States Tekdoors Full time

    Job DescriptionJob DescriptionJob Title: Data Engineer Location: Jacksonville, FL / Phoenix, AZ (Hybrid Work) Duration: 24+ Months Employment Type: Contract W2 Experience: 8+ YearsMust-Have: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Proven experience as a Data Engineer with a minimum of 8 years in...

  • Data Engineer

    2 weeks ago

    Phoenix, United States Nucleusteq Full time

    What You'll Do  ● Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies  ● Partner with business teams and communicate with product managers to automate business processes, build visualizations on key performance indicators, and provide real-time...

  • Lead Engineering Technician

    1 week ago

    Phoenix, United States NESC Staffing Full time

    Job Description:Pay rate: $40-45Day ShiftNo people management duties.Lead Engineering TechnicianProvide laboratory support for the Flight Controls Compact Fly-by-Wire engineering team. Your work will include support for lab support, testing, data analysis, and the accurate reporting of data. You will support the team to conceptualize, design, plan, and...

  • Lead Engineering Technician

    2 weeks ago

    Phoenix, United States NESC Staffing Full time

    Job Description:Pay rate: $40-45Day ShiftNo people management duties.Lead Engineering TechnicianProvide laboratory support for the Flight Controls Compact Fly-by-Wire engineering team. Your work will include support for lab support, testing, data analysis, and the accurate reporting of data. You will support the team to conceptualize, design, plan, and...

  • Lead Engineering Technician

    2 weeks ago

    Phoenix, United States NESC Staffing Full time

    Job Description:Pay rate: $40-45Day ShiftNo people management duties.Lead Engineering TechnicianProvide laboratory support for the Flight Controls Compact Fly-by-Wire engineering team. Your work will include support for lab support, testing, data analysis, and the accurate reporting of data. You will support the team to conceptualize, design, plan, and...

  • Principal Data Engineer 1

    3 days ago

    Phoenix, AZ, United States Western Alliance Bank Full time

    Principal Data Engineer 1 with Western Alliance Bank (Phoenix, AZ) Job Description: Design, build and implement critical components of the enterprise data platform in accordance with the strategic data and analytics needs of the organization. Develop a cross-business unit data tier that is leveraged to improve products and deliver insights. Serve as a...

  • Data Engineering

    4 weeks ago

    Phoenix, United States Tata Consultancy Services Full time

    Job Title Data Engineering - Oracle Fusion cloud: Learning & Talent  Work Location (State, City and Zip) AZ-Phoenix-85054 Relevant Experience (in yrs) 5-8 years Technical/Functional Skills ·Must have 5-8 years of Data Engineering experience in a Big Data environment. ·Strong Data transformation/ETL skills using Spark, Python,...

  • Senior QA Automation Engineer

    3 weeks ago

    Phoenix, United States NTT DATA Full time

    100% Remote NTT DATA's Client is seeking a Senior QA Automation Engineer with 6 or more years of experience of testing with RESTful services using Postman or Groovy scripts. 3 or more years of experience with Agile tools like Azure DevOps, JIRA, etc. 3 or more years of experience with CI/CD tools like ADO, Jenkins etc. 3 or more years of development...

  • Lead Engineering Technician

    3 weeks ago

    Phoenix, United States BEPC Inc. - Business Excellence Professional Consulting Full time

    BEPC is actively looking for a Lead Engineering Technician in Phoenix, AZ area!6 - month contract with possible extensions! Benefits include medical, dental, vision, and life insurance Pay Rate: $40.00-$45.00/hour Shift Time: Monday to Friday 8:00am-5:00pm Provide laboratory support for the Flight Controls Compact Fly-by-Wire engineering team. Your work will...

  • Lead Engineering Technician

    3 weeks ago

    Phoenix, United States BEPC Inc. - Business Excellence Professional Consulting Full time

    BEPC is actively looking for a Lead Engineering Technician in Phoenix, AZ area!6 - month contract with possible extensions! Benefits include medical, dental, vision, and life insurance Pay Rate: $40.00-$45.00/hour Shift Time: Monday to Friday 8:00am-5:00pm Provide laboratory support for the Flight Controls Compact Fly-by-Wire engineering team. Your work will...

  • Lead Engineering Technician

    3 weeks ago

    Phoenix, United States BEPC Inc. - Business Excellence Professional Consulting Full time

    BEPC has an open position for a Lead Engineering TechnicianLocation: Phoenix, AZBenefits: Medical, Dental, Vision, and Life Insurance Pay Rate: $40.00 - $45.00 Hour DOEShift: Mon – Fri 8:00 AM to 5:00 PMTerm: 6-month contract with possible extensions or permanency (Based on Performance)Requirements: High School Diploma or GED / EET Degree or equivalent. /...