Lead Big Data Engineer

3 weeks ago


Philadelphia, United States Clarivate Analytics US LLC Full time

We are looking for a Lead Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development, are metrics-focused and obsessed with CI/CD and DevOps we would love to speak with you

About You - experience, education, skills, and accomplishments

  • Bachelor's degree in computer science or equivalent experience in software development.
  • Minimum 5 years of relevant experience in developing Big Data systems.
  • Minimum 5 years' experience in working with all aspects of distributed processing, including Hadoop, Hive, Apache Spark and Cassandra.
  • Minimum 5 years' experience coding in key technologies like Java, Apache Spark, Hive, PostgreSQL, Scala, Git, Linux, AWS, and Greenplum.
  • Minimum 5 years' experience in AWS Ecosystem with hands-on knowledge of ec2, S3 and services
  • Minimum 5 years' experience in working with build tools like Maven, Gradle and CI/CD tools like Jenkins.
  • Minimum 5 years' experience in optimizing Big Data system performance.

It would be great if you also had . . .

  • Experience in Big Data design and development
  • Experience on Spark streaming
  • Ability to guide developers-Front end, back end, and full-stack developers across different tech functions
  • An understanding of existing complex processes and workflows
  • Hands on experience with the full SDLC
  • Experience working in an Agile Scrum methodology

What will you be doing in this role?

  • Work closely with users, technical leads, project managers, and other cross-functional development teams to design and deliver software solutions.
  • Design software solutions, making modifications to existing software and integrating applications within the core pathway.
  • Must develop software using disciplined software development processes, adhering to team/company standards and software best practice guidelines, and corporate policies.
  • Responsible for applying working knowledge of OO design patterns to develop enterprise level and cloud native solutions.
  • Demonstrate ability of translating plain requirements into effective technical design and solutions
  • Deliver quality product to our QA and be responsible for creating and executing unit tests on the code that they write for their applications.
  • Able to write high-performance, reliable, and maintainable code.
  • Demonstrate excellent communication and collaboration skills and has experience in working with cross functional and cross geography teams.

Hours of Work

  • Full time, permanent
  • Hybrid working schedule 2-3 days/week on-site
  • Must live within a commutable distance to our Philadelphia, PA office

Clarivate is an Equal Opportunity Employer Vets/Minorities/Women/Disabled



  • Philadelphia, United States Clarivate, plc Full time

    We are looking for a Lead Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development Data Engineer, Big Data, Engineer, Lead, Software, Technology, Data


  • Philadelphia, United States Clarivate Full time

    We are looking for a Lead Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development, are metrics-focused and obsessed with CI/CD and DevOps we would love to speak with you! About You -...

  • Azure Big Data Lead

    1 week ago


    Philadelphia, United States Quadratic Systems, Inc. Full time

    Role: Azure Big Data LeadLocation: Philadelphia, PA (Hybrid work)Type: Full Time and 12+ ContractJob DescriptionPrimary Skill :1 Azure databricks- Build Data Pipelines using ADB-Leveraging Python and SparkBuild Data Transformation in ADBConvert Scala Pgms to Py Spark PgmsUnderstand the SCALA program in DataBricks2 ADFSecondary Skills1 Azure data lake gen 22....

  • Data Engineer

    2 days ago


    Philadelphia, United States Marvica Technologies Full time

    Job DescriptionJob DescriptionData Engineer Location: Philadelphia (Hybrid), Contract to hireExp: 8+ YearsResponsibilities: Develop data-driven solutions with Cloud based technologies to meet evolving business needs. Deliver impactful, scalable, and highly performant Data solution to our customers based in US, UK, Canada and India based customers Provide...


  • Philadelphia, United States Perpay Inc. Full time

    About the Role As a Data Engineering Manager at Perpay, you will lead data engineers and work closely with data scientists and analysts to help drive our vision of building data products and insights for our stakeholders and customers, accelerating Perpay's mission to create inclusive financial products that improve the lives of our members. We are actively...

  • Sr Data Engineer

    3 weeks ago


    Philadelphia, United States codesbright Full time

    Responsibilities Develop and maintain database architectures data schemas and data integration processes. Conduct performance tuning troubleshooting and data quality checks to guarantee the accuracy reliability and availability of data. Collaborate with software engineers to integrate data-driven capabilities into applications. Stay up-to-date with...

  • Sr Data Engineer

    3 weeks ago


    Philadelphia, Pennsylvania, United States codesbright Full time

    Responsibilities Develop and maintain database architectures data schemas and data integration processes. Conduct performance tuning troubleshooting and data quality checks to guarantee the accuracy reliability and availability of data. Collaborate with software engineers to integrate data-driven capabilities into applications. Stay up-to-date with emerging...


  • Philadelphia, United States augmentjobs Full time

    Job DescriptionJob DescriptionJob Description:Responsibilities:Data Architecture and Strategy: Define and implement the overarching data architecture and strategy for the organization. Design scalable, reliable, and efficient data pipelines, data warehouses, and data lakes to support business needs and objectives.Data Engineering Leadership: Provide...


  • Philadelphia, United States Proscia Full time

    **About Proscia** Pathology is at the center of medicine, and is undergoing a profound transformation as the final frontier of digitization in healthcare. We started Proscia to accelerate pathology’s transition from microscope to images, and to use that data to change the way we think about cancer. We can’t change medicine alone. That’s why we’re...


  • Philadelphia, United States HealthVerity Full time

    **VP of Engineering, Data Warehouse** Philadelphia, Pennsylvania / Engineering Leadership / Full-time **How you will help** The VP of Engineering, Data Warehouse will own the development of HealthVeritys current and future mastered data solutions. HealthVerity is looking for a candidate that sits at the intersection of healthcare data expertise and...


  • Philadelphia, United States Proscia Full time

    Job DescriptionJob DescriptionSalary: About ProsciaPathology is at the center of medicine, and is undergoing a profound transformation as the final frontier of digitization in healthcare. We started Proscia to accelerate pathology’s transition from microscope to images, and to use that data to change the way we think about cancer. We can’t change...


  • Philadelphia, United States augmentjobs Full time

    Job DescriptionJob DescriptionJob Description:Responsibilities:Data Architecture and Design: Design and implement scalable and efficient data architectures, including data warehouses, data lakes, and data marts. Define data models, schemas, and standards to ensure consistency and reliability across the organization.Data Pipeline Development: Develop and...


  • Philadelphia, United States Comcast Full time

    Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and...


  • Philadelphia, United States Perpay - Career's Page Full time

    Job DescriptionJob DescriptionAbout Us:We believe everyone deserves a chance to improve their financial future. We're dedicated to building simple and inclusive financial products that help our members create healthy habits and achieve economic stability.Some things we're excited about:$500 million in spending power used by our membersIncreasing...


  • Philadelphia, United States augmentjobs Full time

    Job DescriptionJob DescriptionJob Description:Responsibilities:Technical Leadership: Lead machine learning projects from conception to completion, providing technical guidance, direction, and mentorship to a team of machine learning engineers. Set technical direction, define project objectives, and ensure alignment with organizational goals and...

  • Data Scientist

    22 hours ago


    Philadelphia, Pennsylvania, United States Barbaricum Full time

    Barbaricum is a rapidly growing government contractor providing leading-edge support to federal customers, with a particular focus on Defense and National Security mission sets. We leverage more than 15 years of support to stakeholders across the federal government, with established and growing capabilities across Intelligence, Analytics, Engineering,...


  • Philadelphia, United States Big Brother & Big Sister of Independence Region Full time

    Big Brothers Big Sisters IndependencePosition: Lead School-Based MentorLocation: Philadelphia, PA (on-site Schools 4-5 days/week)Employment Status: Non-exempt, FulltimeReporting to: Manager, In-School ProgramEffective: May 17, 2024Salary: $45,000 to $48,000AGENCY OVERVIEW JOIN A FORWARD-THINKING AGENCY THAT SEEKS TO ENSURE THAT ALL YOUTH ACHIEVE THEIR FULL...


  • Philadelphia, United States Data-Core Systems, Inc. Full time

    As a Machine Learning Engineer you will take a lead role in executing deep learning (CNN LSTM) clustering and classification algorithm fractal and spectral data analysis feature engineering and synthetic data generation.Duties and Responsibilities Execute deep learning (CNN LSTM) Clustering and classification algorithm Fractal and spectral data...


  • Philadelphia, Pennsylvania, United States Data-Core Systems, Inc. Full time

    As a Machine Learning Engineer you will take a lead role in executing deep learning (CNN LSTM) clustering and classification algorithm fractal and spectral data analysis feature engineering and synthetic data generation.Duties and Responsibilities Execute deep learning (CNN LSTM) Clustering and classification algorithm Fractal and spectral data analysis...


  • Philadelphia, United States Comcast Corporation Full time

    Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and...