Pyspark Technical Lead

2 weeks ago

Chicago IL, United States Cognizant Full time

About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the future—a future in which trial-and-error business decisions have been replaced by informed choices and data-supported strategies.

By applying AI and data science, we help leading companies to prototype, refine, validate, and scale their AI and analytics products and delivery models. Cognizant’s AIA practice takes insights that are buried in data, and provides businesses a clear way to transform how they source, interpret and consume their information. Our clients need flexible data structures and a streamlined data architecture that quickly turns data resources into informative, meaningful intelligence.

Job Summary :

We are seeking a Pyspark Technical Lead with 7 to 11 years of experience to join our team. The ideal candidate will have expertise in Pyspark , Cloudera HUE, Cloudera Data Platform, Apache Iceberg, Spark. This role involves working in a hybrid model with day shifts. The candidate will play a crucial role in driving our research and development initiatives, leveraging their technical skills to deliver impactful solutions.

Experience :

7to11yrs

Required Skills :

Cloudera HUE,PySpark,Spark Pyspark,Cloudera Data Platform,Apache Iceberg

Responsibilities :

Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform

Supervise the integration of Apache Iceberg and Spark into existing data workflows

Provide technical guidance and mentorship to junior team members on PySpark best practices

Collaborate with cross-functional teams to understand data requirements and deliver scalable solutions

Ensure data quality and integrity through meticulous testing and validation processes

Develop and maintain documentation for data architectures and workflows

Optimize data processing pipelines for performance and efficiency

Conduct code reviews to ensure alignment to coding standards and best practices

Solve and resolve complex technical issues related to data processing

Stay updated with the latest advancements in data technologies and incorporate them into projects

Drive innovation by exploring new tools and techniques in the research and development domain

Communicate effectively with stakeholders to gather requirements and provide project updates

Contribute to the overall success of the company by delivering high-quality data solutions that drive business value

Qualifications
  • Possess strong expertise in Cloudera HUE and Cloudera Data Platform
  • Demonstrate proficiency in Apache Iceberg and Spark
  • Have extensive experience with PySpark for data processing
  • Show a solid understanding of data architecture and data engineering principles
  • Exhibit excellent problem-solving skills and attention to detail
  • Have a background in research and development is a plus
  • Display strong communication and collaboration skills
  • Be able to work effectively in a hybrid work model
  • Have a proactive approach to learning and staying current with industry trends
  • Be capable of mentoring and guiding junior team members
  • Demonstrate the ability to deliver high-quality solutions within deadlines
  • Show a commitment to continuous improvement and innovation
  • Possess a strong sense of ownership and accountability for project outcomes.

Benefits : Cognizant offers the following benefits for this position, subject to applicable eligibility requirements:

  • Medical/Dental/Vision/Life Insurance
  • Paid holidays plus Paid Time Off
  • 401(k) plan and contributions
  • Long-term/Short-term Disability
  • Paid Parental Leave
  • Employee Stock Purchase Plan

Disclaimer: The salary, other compensation, and benefits information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.

  • Pyspark Technical Lead

    2 weeks ago

    Chicago, IL, United States Cognizant Full time

    About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the...

  • Pyspark Technical Lead

    2 months ago

    Chicago, United States Cognizant Full time

    About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the...

  • Pyspark Technical Lead

    2 months ago

    Chicago, United States Cognizant Full time

    About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the...

  • Pyspark Technical Lead

    2 weeks ago

    Chicago, United States Cognizant Full time

    About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the...

  • Pyspark Technical Lead

    3 weeks ago

    Chicago, United States Cognizant Full time

    About AI & Analytics: Artificial intelligence (AI) and the data it collects and analyzes will soon sit at the core of all committed, human-centric businesses. By decoding customer needs, preferences, and behaviors, our clients can understand exactly what services, products, and experiences their consumers need. Within AI & Analytics, we work to craft the...

  • Pyspark Tech Lead

    2 days ago

    Chicago, United States Themesoft Inc. Full time

    Role: Pyspark Tech Lead Location: Chicago, Illinois(Onsite)Responsibilities:Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform- Oversee the integration of Apache Iceberg and Spark into existing data workflowsProvide technical guidance and mentorship to junior team members on PySpark best practices- Collaborate...

  • Pyspark Tech Lead

    2 days ago

    Chicago, United States Themesoft Inc. Full time

    Role: Pyspark Tech Lead Location: Chicago, Illinois(Onsite)Responsibilities:Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform- Oversee the integration of Apache Iceberg and Spark into existing data workflowsProvide technical guidance and mentorship to junior team members on PySpark best practices- Collaborate...

  • Python Pyspark Consultant

    5 months ago

    Chicago, United States CapB InfoteK Full time

    CapB is looking for Python PySpark developers Full time or Contract. Able to communicate technical concepts including presenting and whiteboarding  2  Ability to design and implement architectures within AWS utilizing Databricks  3  Hands on Experience writing Pyspark pipelines in Databricks  4  Minimum 5 7 years of experience with Python Spark and the...

  • Technical Lead

    1 week ago

    Chicago, Illinois, United States RIT Solutions, Inc. Full time

    Join Our Team: RIT Solutions, Inc. is a dynamic and innovative company that offers a competitive salary range of $150,000 - $180,000 per year, depending on experience. Our comprehensive benefits package includes medical, dental, and vision insurance, 401(k) matching, and generous paid time off. We are committed to fostering a diverse and inclusive work...

  • Technical Lead

    2 weeks ago

    Chicago, Illinois, United States Technical Paradigm LLC Full time

    Job Title: Technical LeadDescription: Are you a seasoned software engineer looking to take on new challenges? We are seeking a highly skilled and experienced technical lead to join our team at Technical Paradigm LLC. This is an exciting opportunity for a motivated individual with extensive experience in fixed income securities trading, clearing and...

  • Big Data Architect Lead

    4 weeks ago

    Chicago, Illinois, United States NielsenIQ Full time

    Job OverviewNielsenIQ is seeking a Senior Data Engineer with hands-on experience in Python and Pyspark to support software development activities in an Agile environment. The ideal candidate will possess strong technical, analytical, and interpersonal skills.About NielsenIQNielsenIQ is the world's leading consumer intelligence company, delivering...

  • Cloud AI Engineering Lead

    1 week ago

    Chicago, Illinois, United States PeopleCaddie Full time

    Job DescriptionAs a Senior Machine Learning Engineer, you will play a critical role in designing and implementing machine learning workflows and architectures that maximize the impact of our algorithms. You will partner with data scientists to develop prototype solutions of algorithmic products leveraging appropriate AWS services. The position requires...

  • Technical lead

    4 weeks ago

    Chicago, United States Omni Inclusive Full time

    Mandatory: AWS MLOps (Primary) Data Science Machine Learning, Python Mandatory Skills: DataIKU MLOps (Primary) Data Science Machine Learning Python DatatIKU MLOps certification. Detailed JD: Bachelor's degree in engineering, computer science, mathematics, or related field. At least 5 years of extensive experience in Software Engineering (Python) / DevOps/...

  • Technical Lead

    3 weeks ago

    Chicago, IL, United States Cognizant Full time

    Job Summary We are seeking an experienced Technical Lead (OSI Pi) with 10 to 14 years of experience in the Medical Devices domain. The ideal candidate will have expertise in POSIX based OS OSIsoft PI Historian OSIsoft PI/ASPEN IP21 and Osi ADMS. This hybrid role requires a deep understanding of technical and domain-specific skills to drive innovation and...

  • Technical Lead

    5 days ago

    Chicago, IL, United States Cognizant Full time

    Job Summary We are seeking an experienced Technical Lead (OSI Pi) with 10 to 14 years of experience in the Medical Devices domain. The ideal candidate will have expertise in POSIX based OS OSIsoft PI Historian OSIsoft PI/ASPEN IP21 and Osi ADMS. This hybrid role requires a deep understanding of technical and domain-specific skills to drive innovation and...

  • Technical Lead

    2 days ago

    Chicago, United States Maven Companies Inc. Full time

    ServiceNow Technical Lead - W2 role onlyQualifications:Deep technical expertise in ServiceNow development and configurations.Proven experience leading development teams, including code reviews and Agile/Scrum methodologies.Strong problem-solving skills and ability to navigate complex team dynamics.Experience in ITSM, ITOM, and Knowledge Management...

  • Technical Lead

    2 days ago

    Chicago, United States Maven Companies Inc. Full time

    ServiceNow Technical Lead - W2 role onlyQualifications:Deep technical expertise in ServiceNow development and configurations.Proven experience leading development teams, including code reviews and Agile/Scrum methodologies.Strong problem-solving skills and ability to navigate complex team dynamics.Experience in ITSM, ITOM, and Knowledge Management...

  • Technical Lead

    3 weeks ago

    Chicago, United States Inficare Full time

    Job Title: Technical Lead Job Location: Chicago, IL (Onsite) Mandatory Skills: 12+ years of experience in Tech Lead role. Microsoft Azure -10+ Years C#- 10+ Years .NET -12+ Years Rest API - 8+ Years Team Handling- 5+ Years Location - Chicago, IL A senior technologist with having at least 12-14 years and in following areas 1. Very crucial and key to success...

  • Technical Lead

    2 weeks ago

    Chicago, Illinois, United States Eateam Full time

    ScheduleWe are seeking a full-time Technical Lead to join our team. The position requires a minimum of 40 hours per week and may involve occasional overtime.LocationThe position is based in our headquarters in [City]. We offer a flexible work-from-home policy, but regular office attendance is expected.About the JobThis is an exciting opportunity to lead a...

  • Technical Lead

    3 weeks ago

    Chicago, United States Cognizant Technology Solutions Full time

    Job Summary We are seeking an experienced Technical Lead (OSI Pi) with 10 to 14 years of experience in the Medical Devices domain. The ideal candidate will have expertise in POSIX based OS OSIsoft PI Historian OSIsoft PI/ASPEN IP21 and Osi ADMS. This hybrid role requires a deep understanding of technical and domain-specific skills to drive innovation and...