Lead Big Data Engineer

3 weeks ago


Philadelphia, United States Clarivate Full time

We are looking for a Lead Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development, are metrics-focused and obsessed with CI/CD and DevOps we would love to speak with you

About You - experience, education, skills, and accomplishments

  • Bachelor's degree in computer science or equivalent experience in software development.
  • Minimum 5 years of relevant experience in developing Big Data systems.
  • Minimum 5 years' experience in working with all aspects of distributed processing, including Hadoop, Hive, Apache Spark and Cassandra.
  • Minimum 5 years' experience coding in key technologies like Java, Apache Spark, Hive, PostgreSQL, Scala, Git, Linux, AWS, and Greenplum.
  • Minimum 5 years' experience in AWS Ecosystem with hands-on knowledge of ec2, S3 and services
  • Minimum 5 years' experience in working with build tools like Maven, Gradle and CI/CD tools like Jenkins.
  • Minimum 5 years' experience in optimizing Big Data system performance.
It would be great if you also had . . .
  • Experience in Big Data design and development
  • Experience on Spark streaming
  • Ability to guide developers-Front end, back end, and full-stack developers across different tech functions
  • An understanding of existing complex processes and workflows
  • Hands on experience with the full SDLC
  • Experience working in an Agile Scrum methodology
What will you be doing in this role?
  • Work closely with users, technical leads, project managers, and other cross-functional development teams to design and deliver software solutions.
  • Design software solutions, making modifications to existing software and integrating applications within the core pathway.
  • Must develop software using disciplined software development processes, adhering to team/company standards and software best practice guidelines, and corporate policies.
  • Responsible for applying working knowledge of OO design patterns to develop enterprise level and cloud native solutions.
  • Demonstrate ability of translating plain requirements into effective technical design and solutions
  • Deliver quality product to our QA and be responsible for creating and executing unit tests on the code that they write for their applications.
  • Able to write high-performance, reliable, and maintainable code.
  • Demonstrate excellent communication and collaboration skills and has experience in working with cross functional and cross geography teams.
Hours of Work
  • Full time, permanent
  • Hybrid working schedule 2-3 days/week on-site
  • Must live within a commutable distance to our Philadelphia, PA office


Clarivate is an Equal Opportunity Employer Vets/Minorities/Women/Disabled

  • Philadelphia, United States Clarivate, plc Full time

    We are looking for a Lead Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development Data Engineer, Big Data, Engineer, Lead, Software, Technology, Data


  • Philadelphia, United States Clarivate Full time

    We are looking for a Senior Big Data Engineer to join our IMS team in Philadelphia, PA. This is an exciting new opportunity to support data excellence and research intelligence projects. If you have a solid background in Big Data design and development, are metrics-focused and obsessed with CI/CD and DevOps we would love to speak with you! About You -...

  • Azure Big Data Lead

    2 weeks ago


    Philadelphia, United States Quadratic Systems, Inc. Full time

    Role: Azure Big Data LeadLocation: Philadelphia, PA (Hybrid work)Type: Full Time and 12+ ContractJob DescriptionPrimary Skill :1 Azure databricks- Build Data Pipelines using ADB-Leveraging Python and SparkBuild Data Transformation in ADBConvert Scala Pgms to Py Spark PgmsUnderstand the SCALA program in DataBricks2 ADFSecondary Skills1 Azure data lake gen 22....


  • Philadelphia, United States JPMorgan Chase Full time

    Job Description Embrace this pivotal role as an essential member of a high performing team dedicated to reaching new heights in data engineering. Your contributions will be instrumental in shaping the future of one of the world's largest and most influential companies.As a Senior Lead Data Engineer - Data Management at JPMorgan Chase within the Consumer...

  • big data developer

    6 days ago


    Philadelphia, United States Saxon Global Full time

    Client: comcast Location: remote Duration: 1 year+ USC/ GC / H4EAD Big Data Developer Requires a bachelor's degree in area of specialty and 3-5 years of experience in the field or in a related area. Good knowledge of standard concepts, practices, and procedures within a particular field. 2-3 years of Python experience. Strong communication...

  • Lead Data Engineer

    4 days ago


    Philadelphia, United States JPMorgan Chase Full time

    Job Description Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.As a Lead Data Engineer - AWS at JPMorgan Chase within the Consumer and Community Banking, Data Technology team, you are an integral...

  • Lead Data Engineer

    17 hours ago


    Philadelphia, United States JPMorgan Chase Full time

    Job Description Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.As a Lead Data Engineer - AWS at JPMorgan Chase within the Consumer and Community Banking, Data Technology team, you are an integral...

  • Lead Data Engineer

    16 hours ago


    Philadelphia, United States Cadent Full time

    Cadent powers the evolution of TV brand advertising. We provide marketers, agencies, operators, and media owners with data-driven solutions for buying and selling TV advertising. By connecting brands with opportunities across national inventory sourcescable, broadcast, and digital mediaour technology improves efficiencies and boosts the results of linear,...

  • Data Engineer

    1 week ago


    Philadelphia, United States Marvica Technologies Full time

    Job DescriptionJob DescriptionData Engineer Location: Philadelphia (Hybrid), Contract to hireExp: 8+ YearsResponsibilities: Develop data-driven solutions with Cloud based technologies to meet evolving business needs. Deliver impactful, scalable, and highly performant Data solution to our customers based in US, UK, Canada and India based customers Provide...


  • Philadelphia, United States Perpay Inc. Full time

    About the Role As a Data Engineering Manager at Perpay, you will lead data engineers and work closely with data scientists and analysts to help drive our vision of building data products and insights for our stakeholders and customers, accelerating Perpay's mission to create inclusive financial products that improve the lives of our members. We are actively...

  • Sr Data Engineer

    4 weeks ago


    Philadelphia, United States codesbright Full time

    Responsibilities Develop and maintain database architectures data schemas and data integration processes. Conduct performance tuning troubleshooting and data quality checks to guarantee the accuracy reliability and availability of data. Collaborate with software engineers to integrate data-driven capabilities into applications. Stay up-to-date with...

  • Sr Data Engineer

    4 weeks ago


    Philadelphia, Pennsylvania, United States codesbright Full time

    Responsibilities Develop and maintain database architectures data schemas and data integration processes. Conduct performance tuning troubleshooting and data quality checks to guarantee the accuracy reliability and availability of data. Collaborate with software engineers to integrate data-driven capabilities into applications. Stay up-to-date with emerging...


  • Philadelphia, United States augmentjobs Full time

    Job DescriptionJob DescriptionJob Description:Responsibilities:Data Architecture and Strategy: Define and implement the overarching data architecture and strategy for the organization. Design scalable, reliable, and efficient data pipelines, data warehouses, and data lakes to support business needs and objectives.Data Engineering Leadership: Provide...


  • Philadelphia, United States Proscia Full time

    **About Proscia** Pathology is at the center of medicine, and is undergoing a profound transformation as the final frontier of digitization in healthcare. We started Proscia to accelerate pathology’s transition from microscope to images, and to use that data to change the way we think about cancer. We can’t change medicine alone. That’s why we’re...


  • Philadelphia, United States Comcast Corporation Full time

    FreeWheel, a Comcast company, provides comprehensive ad platforms for publishers, advertisers, and media buyers. Powered by premium video content, robust data, and advanced technology, we’re making it easier for buyers and sellers to transact across all screens, data types, and sales channels. As a global company, we have offices in nine countries and can...


  • Philadelphia, United States HealthVerity Full time

    **VP of Engineering, Data Warehouse** Philadelphia, Pennsylvania / Engineering Leadership / Full-time **How you will help** The VP of Engineering, Data Warehouse will own the development of HealthVeritys current and future mastered data solutions. HealthVerity is looking for a candidate that sits at the intersection of healthcare data expertise and...


  • Philadelphia, United States Comcast Full time

    Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and...


  • Philadelphia, United States augmentjobs Full time

    Job DescriptionJob DescriptionJob Description:Responsibilities:Data Architecture and Design: Design and implement scalable and efficient data architectures, including data warehouses, data lakes, and data marts. Define data models, schemas, and standards to ensure consistency and reliability across the organization.Data Pipeline Development: Develop and...


  • Philadelphia, United States Perpay - Career's Page Full time

    Job DescriptionJob DescriptionAbout Us:We believe everyone deserves a chance to improve their financial future. We're dedicated to building simple and inclusive financial products that help our members create healthy habits and achieve economic stability.Some things we're excited about:$500 million in spending power used by our membersIncreasing...


  • Philadelphia, United States Perpay Inc. Full time

    About Us: We believe everyone deserves a chance to improve their financial future. We’re dedicated to building simple and inclusive financial products that help our members create healthy habits and achieve economic stability. Some things we’re excited about: $500 million in spending power used by our members Increasing members’ credit by 36 points...