Lead Data Processing Engineer

6 days ago


Dallas, Texas, United States Tek Ninjas Full time
Job Description

Estimated Salary: $120,000 - $180,000 per year in Dallas, TX

We are seeking a highly skilled Lead Data Processing Engineer to join our team at Tek Ninjas.

About the Role:

This is a challenging and rewarding role that involves designing and implementing data processing workflows using PySpark and Spark Pyspark.

The ideal candidate will have extensive experience with Unix and UnixShellScripting for automating data workflows.

Responsibilities:

  • Analyze business requirements to develop data solutions that align with organizational goals and objectives.
  • Collaborate with cross-functional teams to gather and document detailed business and technical requirements.
  • Design and implement data processing workflows using PySpark and Spark Pyspark to ensure efficient data handling.
  • Develop and maintain Unix and UnixShellScripting scripts to automate data processing tasks and improve operational efficiency.
  • Oversee the integration of data from various sources to create comprehensive datasets for analysis and reporting.
  • Provide technical support and troubleshooting for data processing issues to ensure smooth operations.
  • Conduct data quality assessments and implement measures to ensure data accuracy and consistency.
  • Create and maintain detailed documentation of data processes, workflows, and technical specifications.
  • Collaborate with data scientists and analysts to develop and optimize data models and algorithms.
  • Monitor and optimize the performance of data processing workflows to ensure timely and accurate data delivery.
  • Lead efforts to identify and implement process improvements to enhance data processing efficiency and effectiveness.
  • Provide training and support to team members on data processing tools and techniques.
  • Ensure compliance with data governance and security policies to protect sensitive information.
  • Possess a Bachelor's degree in Computer Science, Information Technology, or a related field.
  • Demonstrate strong proficiency in PySpark and SparkPyspark for data processing and analysis.
  • Have extensive experience with Unix and UnixShellScripting for automating data workflows.
  • Exhibit excellent analytical and problem-solving skills to address complex data challenges.
  • Show strong communication skills to effectively collaborate with cross-functional teams.
  • Display a keen attention to detail to ensure data accuracy and consistency.


  • Dallas, Texas, United States AArete Full time

    Unlock your full potential in a dynamic and innovative environment at AArete, where we empower our people to drive purposeful change and deliver actionable outcomes.About the RoleWe are seeking an experienced Data Engineering Lead to join our team of experts in strategic profitability improvement, digital transformation, and advisory services. As a key...


  • Dallas, Texas, United States Resource Informatics Group Inc Full time

    Job Title: Senior Data EngineerJob Description:At Resource Informatics Group Inc, we are seeking a highly skilled Senior Data Engineer to lead our data engineering projects.The ideal candidate will have a strong background in data engineering and experience in mentoring team members.Key Responsibilities:• Manage and deliver ETL solutions by working closely...


  • Dallas, Texas, United States Tekstrom Inc Full time

    At Tekstrom Inc, we are seeking a highly skilled Data Science and Engineering Lead to join our team. This role will lead the development of big data analytics systems using Hadoop technologies.Job DescriptionThe ideal candidate for this position will have at least 3+ years of hands-on experience with big data or analytics systems. Strong SQL/analytical SQL...


  • Dallas, Texas, United States INSPYR Solutions Full time

    We are seeking a highly skilled and experienced Data Engineering Team Lead to join our dynamic team at INSPYR Solutions. This role involves leading the design and development of robust data pipelines, streaming ingestion frameworks, and scalable data architectures to support our data-driven initiatives.


  • Dallas, Texas, United States MDS (Micro-Data Systems) Full time

    Data Center Networking ExpertiseMDS (Micro-Data Systems) requires a skilled Data Center Infrastructure Engineer to join our team. As a key contributor, you will be responsible for designing, implementing, and optimizing our data center network infrastructure. Your expertise in Cisco networking data center equipment, network monitoring, and troubleshooting...


  • Dallas, Texas, United States Anblicks Full time

    Anblicks is looking for a Cloud Data Engineering Lead to manage offshore development teams and be the onsite SPOC for client stakeholders.The estimated salary for this position is $160,000 - $200,000 per year.This role involves collaborating with onsite business stakeholders to understand, analyze and translate the requirements to the development team.You...


  • Dallas, Texas, United States Insight Global Full time

    **Job Overview**We are seeking a skilled Data Engineer to join our team in Mexico. As a data expert, you will be responsible for designing and implementing data solutions in an Azure cloud environment.Key Responsibilities:Design and develop data models and storage solutions in AzureProgram with Python (primary) and other languages such as Java and ScalaBuild...


  • Dallas, Texas, United States Capgemini Full time

    Job DescriptionAbout the RoleWe are seeking a skilled Data Engineer to join our team at Sogeti, a leading provider of professional technology services.Key ResponsibilitiesDesign, develop, and maintain data pipelines to collect, store, process, and analyze large volumes of dataPartner with stakeholders to understand business needs and translate them into...


  • Dallas, Texas, United States Resource Informatics Group Inc Full time

    About the RoleResource Informatics Group Inc is seeking a highly skilled Data Engineering Lead to join our team in Dallas, TX – Remote. This 12-month opportunity will involve leading data engineering and ETL projects, mentoring junior staff members, and architecting and developing ETL solutions.The ideal candidate will have experience managing and...


  • Dallas, Texas, United States Tiger Analytics Full time

    About UsTiger Analytics is a pioneering AI and analytics consulting firm with a global presence. Our mission is to harness the power of data and technology to drive transformative solutions that impact millions worldwide.Career OpportunitiesThis role presents an exceptional opportunity for growth in a dynamic, entrepreneurial environment with high individual...


  • Dallas, Texas, United States Onapsis Full time

    About the JobThe world's most critical business applications have been neglected for far too long. Onapsis provides cybersecurity solutions dedicated to business-critical applications. We help companies understand threats and risks across their SAP and Oracle landscapes.We are seeking a Senior Data Engineer to join our mission-driven team. This role is ideal...


  • Dallas, Texas, United States evolv consulting Full time

    Job Title: Data Engineering ManagerJob Summary:We're seeking an experienced Data Engineering Manager to lead our team in designing and implementing scalable big data platforms. As a key member of our team, you'll be responsible for building and maintaining large-scale data infrastructure, ensuring high-quality data delivery, and driving innovation in data...

  • Data Management Lead

    2 weeks ago


    Dallas, Texas, United States Infogain Full time

    About the Role: We are seeking a seasoned Data Management Lead to spearhead our Master Data Management (MDM) initiative. As a key member of our team, you will be responsible for designing, implementing, and maintaining our MDM solution using Profisee. Responsibilities: Lead the end-to-end implementation of the Profisee MDM solution, including...


  • Dallas, Texas, United States Advanced Automation Consulting Full time

    Job Title: Senior Data EngineerWe are seeking an experienced Senior Data Engineer to join our team at Advanced Automation Consulting, Inc.About the Role:The ideal candidate will have a strong background in software development, automation technologies, and data engineering. You will be responsible for designing, developing, and maintaining large-scale data...

  • Data Architect Lead

    6 days ago


    Dallas, Texas, United States Anblicks Full time

    About AnblicksAnblicks is a leading technology consulting firm that helps businesses drive innovation and growth through data-driven insights.Salary Range: $120,000 - $180,000 per yearData Solutions Architect Job DescriptionWe are seeking a highly skilled Data Solutions Architect to join our team in Dallas, Texas. As a key member of our data architecture...


  • Dallas, Texas, United States eTek IT Services, Inc. Full time

    Job Title: Big Data EngineerAbout eTek IT Services, Inc.eTek IT Services, Inc. is a leading provider of innovative technology solutions.Salary:$120,000 - $150,000 per yearJob Description:Data Engineer PositionThe ideal candidate will have extensive experience in big data engineering, with a strong background in Google Cloud Platform (GCP) technologies such...


  • Dallas, Texas, United States Harnham Full time

    Job Title: Leading Data Science Expert - Machine Learning EngineerAbout the Role:Harnham is seeking a talented Machine Learning Engineer to join their Data Science team. The successful candidate will play a key role in developing and deploying scalable, innovative models that drive business growth.About You:You have a strong background in machine learning...


  • Dallas, Texas, United States AT&T Full time

    Job Title: Lead Data AnalysisJob SummaryAt AT&T, we are seeking a talented Lead Data Analysis professional to join our Data, Technology and Automation team in AT&T Finance.Key ResponsibilitiesBuild and maintain technical solutions in Azure using SQL Server.Utilize Data Architecture knowledge and skillsets to ensure efficient data curation.Provide data...


  • Dallas, Texas, United States Accellor Full time

    Job DescriptionAccellor is a trusted consulting partner that leverages best-of-breed Cloud technology to deliver exceptional customer engagement and business outcomes for clients across Financial, Retail, High Tech, Healthcare, and Retail verticals.We foster an atmosphere of curiosity, continuous learning, and perseverance. Our employees are encouraged to...

  • Lead Data Architect

    7 days ago


    Dallas, Texas, United States Inficare Full time

    Job SummaryInficare is seeking a highly skilled Data Warehouse Specialist to lead our data engineering efforts. This role is responsible for designing and implementing data warehouses, ensuring data quality and integrity, and developing complex SQL queries. Key skills include proficiency in Snowflake, dimensional data models, data warehouse concepts, and...