Senior Lead Data Engineer

10 hours ago

New York, United States Capital One Full time
11 West 19th Street (22008), United States of America, New York, New York

Senior Lead Data Engineer (Python, Spark, AWS)

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Senior Lead Data Engineer, you'll have the opportunity to be on the forefront of driving a major transformation within Capital One.

The Senior Lead Data Engineer sits within the Card Data Organization & the Data Foundations group, our team works on the foundational capabilities of how data should be produced and consumed. These foundational capabilities include a variety of initiatives as well as AI Assist Tools to enable us to speed the production & consumption of data. The teams are involved in migrating our Batch workloads to the new publishing platform for improved governance and observability. Our environment involves Batch Pipelines involving core technologies such as Scala, Python, Spark, EMR, Glue, Airflow, Step Functions, Delta, Uniform Catalog, Iceberg, DynamoDB amongst many others. On a day to day basis this team tackles problems of massive scale, critical data footprint that has bottom line business impact, resiliency & availability requiring the best techniques within Big Data Analytics and Machine Learning all while providing a superior analyst experience on our tool.

What You'll Do:
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
  • Lead a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
  • Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance


Basic Qualifications:
  • Bachelor's Degree
  • At least 6 years of experience in application development (Internship experience does not apply)
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)


Preferred Qualifications:
  • Master's Degree
  • 9+ years of experience in application development including Python, SQL, Scala, or Java
  • 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
  • 5+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
  • 4+ year experience working on real-time data and streaming applications
  • 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
  • 4+ years of data warehousing experience (Redshift or Snowflake)
  • 4+ years of experience with UNIX/Linux including basic commands and shell scripting
  • 2+ years of experience with Agile engineering practices


Capital One will consider sponsoring a new qualified applicant for employment authorization for this position.

The minimum and maximum full-time annual salaries for this role are listed below, by location. Please note that this salary information is solely for candidates hired to perform work within one of these locations, and refers to the amount Capital One is willing to pay at the time of this posting. Salaries for part-time roles will be prorated based upon the agreed upon number of hours to be regularly worked.

New York City (Hybrid On-Site): $234,700 - $267,900 for Sr. Lead Data Engineer

Illinois (Hybrid On-Site): $218,900 - $249,800 for Sr. Lead Data Engineer

Candidates hired to work in other locations will be subject to the pay range associated with that location, and the actual annualized salary amount offered to any candidate at the time of hire will be reflected solely in the candidate's offer letter.

This role is also eligible to earn performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI). Incentives could be discretionary or non discretionary depending on the plan.

Capital One offers a comprehensive, competitive, and inclusive set of health, financial and other benefits that support your total well-being. Learn more at the . Eligibility varies based on full or part-time status, exempt or non-exempt status, and management level.

This role is expected to accept applications for a minimum of 5 business days.

No agencies please. Capital One is an equal opportunity employer committed to diversity and inclusion in the workplace. All qualified applicants will receive consideration for employment without regard to sex (including pregnancy, childbirth or related medical conditions), race, color, age, national origin, religion, disability, genetic information, marital status, sexual orientation, gender identity, gender reassignment, citizenship, immigration status, protected veteran status, or any other basis prohibited under applicable federal, state or local law. Capital One promotes a drug-free workplace. Capital One will consider for employment qualified applicants with a criminal history in a manner consistent with the requirements of applicable laws regarding criminal background inquiries, including, to the extent applicable, Article 23-A of the New York Correction Law; San Francisco, California Police Code Article 49, Sections 4901-4920; New York City's Fair Chance Act; Philadelphia's Fair Criminal Records Screening Act; and other applicable federal, state, and local laws and regulations regarding criminal background inquiries.

If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at 1- or via email at . All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

For technical support or questions about Capital One's recruiting process, please send an email to

Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.

Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).

  • Data Engineering Lead

    5 days ago

    New York, New York, United States The Custom Group of Companies Full time

    Job Overview:The Custom Group of Companies seeks a highly skilled Senior Data Engineer to lead the design, development, and maintenance of data pipelines in Databricks on an AWS 6 ecosystem. As a key member of our team, you will collaborate with cross-functional teams to understand data needs and translate them into effective data pipeline solutions.About...

  • Senior Data Engineer

    2 hours ago

    New York, United States ingenium.agency Full time

    Job DescriptionJob DescriptionSenior Data Engineer $150k - $170kCompany DescriptionWe are a leading cloud-based mobile patient intake and registration system. Our platform allows patients to complete their paperwork from the comfort of their own homes using their smartphones or computers, minimizing in-person contact and streamlining the check-in process....

  • Data Engineering Lead

    3 days ago

    New York, New York, United States Grata Full time

    Job DescriptionWe are seeking a highly skilled and experienced Data Engineering Lead to join our team at Grata. Our company is the leading private market dealmaking platform, making it easy for investors, investment bankers, management consultants, and corporate development teams to discover and win more deals.The ideal candidate will have a strong...

  • Senior Data Engineer Lead

    3 days ago

    New York, New York, United States Disability Solutions Full time

    Job SummaryWe are seeking a highly skilled Principal Data Engineer to lead the development of consumer-centric applications on RDBMS platforms (Oracle, MYSQL) and Big Data platforms (Hadoop). This role requires a strong understanding of data engineering principles, experience with large-scale deployments, and excellent leadership skills.About UsAt Disability...

  • Data Engineering Lead

    4 days ago

    New York, New York, United States Grubhub Full time

    About the JobWe are seeking a highly skilled Data Engineering Lead to join our team at Grubhub. Our company is dedicated to connecting hungry diners with over 300,000 restaurants nationwide through user-friendly platforms and streamlined delivery capabilities.At Grubhub, we treat data as a valuable asset that drives decision making. The mission of the...

  • Senior Data Engineering Manager

    5 days ago

    New York, New York, United States Consortia Full time

    About the OpportunityAre you a motivated and experienced Senior Data Engineering Manager looking for a new challenge? Do you have a passion for data engineering and a strong background in financial services or consulting?We are seeking a seasoned professional to join our team as a Senior Data Engineering Manager, where you will be responsible for leading the...

  • Senior Data Engineer Role

    1 week ago

    New York, New York, United States TheGuarantors Full time

    About The PositionWe are seeking a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines while ensuring data reliability, accuracy, and integrity. As a key member of our data engineering team, you will lead data ingestion, transformation, and warehousing initiatives to support business objectives. Your responsibilities...

  • Senior Data Engineer

    1 week ago

    New York, New York, United States Collabera Full time

    Collabera is seeking a highly skilled Senior Data Engineer to join our team in New York. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems that support business growth and revenue. You will work closely with stakeholders to understand their needs and develop data-driven solutions that inform...

  • Data Engineering Lead

    5 days ago

    New York, New York, United States Trepp, Inc. Full time

    About the Role">We are seeking an experienced Senior Data Engineer to join our team at Trepp, Inc. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying large-scale data systems and pipelines. Your expertise in data engineering will help us to improve the accuracy and speed of our data processing...

  • Senior Data Engineer

    10 hours ago

    New York, United States TechnoGen Full time

    Please Note: As of July 22, 2021, our team will require that all candidate submissions include a LinkedIn profile. Please do not submit any candidates that do not have a LinkedIn. Kforce has a client in Brooklyn, NY that is seeking a Senior Data Engineer. Our team is headquartered in Brooklyn but has a remote-first culture and we encourage remote applicants...

  • Senior Director of Data Engineering

    7 days ago

    New York, New York, United States Hearst Full time

    **Job Summary**Hearst is seeking a highly skilled Senior Director of Data Engineering to lead our data engineering team and drive the development of our corporate data engineering roadmap.**Key Responsibilities**Lead the design and implementation of scalable and robust data architectures aligned to deliver business outcomes and objectives.Develop and execute...

  • Data Engineer Lead

    7 days ago

    New York, New York, United States Marsh & McLennan Full time

    Company OverviewGuy Carpenter, a business of Marsh McLennan, is a leading global risk advisory and reinsurance specialist and broker. We provide innovative solutions to help our clients navigate complex risks and opportunities.Salary and BenefitsThe base salary range for this role is $94,700 to $170,300. In addition to the base salary, this position may be...

  • Senior Mechanical Engineer

    3 days ago

    New York, New York, United States CoreWeave Full time

    We are seeking a highly skilled and experienced Senior Mechanical Engineer - Data Centers to join our team at CoreWeave. As a senior engineer, you will be responsible for leading and managing mechanical engineering initiatives, designing mechanical systems for our data centers, and collaborating with cross-functional teams to ensure complete and cohesive...

  • Senior Lead Data Engineer

    7 days ago

    New York, United States Capital One Full time

    Senior Lead Data Engineer (Python, Spark, AWS)Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real...

  • Senior Data Engineer

    6 days ago

    New York, United States Capital One Full time

    Senior Data Engineer at Capital One summary: The Senior Data Engineer at Capital One focuses on developing and supporting technical solutions within a collaborative environment. With expertise in programming languages and big data technologies, they play a key role in delivering cloud-based systems that enhance financial experiences for users. This...

  • Senior Data Engineer

    2 hours ago

    New York, United States SumerSports Full time

    Job DescriptionJob DescriptionSenior Data Engineer About SumerSports:SumerSports is a leading football intelligence technology company that specializes in providing an innovative suite of products for football fans and NFL clubs. We are a collection of executives, engineers, data scientists, and visionaries from NFL clubs, technology startups, finance, and...

  • Lead Market Data Systems Engineer

    6 days ago

    New York, New York, United States Luxoft Full time

    Job DescriptionLuxoft is seeking a highly skilled Senior Backend Developer to join our team as a Lead Market Data Systems Engineer. In this role, you will be responsible for designing and implementing market data management systems that support both real-time and end-of-day processes critical to client's Macro trading business.This is a highly visible role...

  • Data Engineer Lead

    5 days ago

    New York, New York, United States Capital One Full time

    About Capital OneAt Capital One, we're on a mission to change the way people think about banking. We believe that innovative technologies and forward-thinking approaches can help us better serve our customers and make a meaningful impact in the world.We're seeking a highly skilled Senior Data Engineer to join our team. As a key member of our engineering...

  • Senior Data Engineer

    3 days ago

    New York, New York, United States Early Warning Services Full time

    Early Warning Services, a trusted name in payments, partners with thousands of institutions to increase access to financial services and protect transactions for hundreds of millions of consumers and small businesses.About the RoleWe are seeking an experienced Senior Data Engineer to develop, document, test, debug, and maintain data and analytics engineering...

  • Senior Data Engineer

    3 days ago

    New York, United States Capital One Full time

    Senior Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are...