Senior Data Engineering Manager
4 weeks ago
We are seeking a highly skilled Lead Data Engineer to join our dynamic team at Insight Global. As a Lead Data Engineer, you will be responsible for designing and developing data pipelines across large-scale, distributed systems using PySpark. You will also work closely with architects to understand high-level designs and ensure seamless integration with the data platform.
The ideal candidate will have 10+ years of experience developing data solutions with experience as a Lead Data Engineer and an understanding of software engineering principles/best practices. You will also have experience working on table formats using HUDI and Iceberg, and storing data in S3.
As a Lead Data Engineer, you will be responsible for guiding and mentoring teams of high-level engineers. You will also be expected to work closely with the architect team to understand high-level designs and ensure seamless integration with the data platform.
We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances.
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/
Key Responsibilities:
- Design and develop data pipelines across large-scale, distributed systems using PySpark
- Work closely with architects to understand high-level designs and ensure seamless integration with the data platform
- Guide and mentor teams of high-level engineers
- Work on table formats using HUDI and Iceberg, and storing data in S3
Requirements:
- 10+ years of experience developing data solutions with experience as a Lead Data Engineer and an understanding of software engineering principles/best practices
- Experience designing and developing data pipelines across large-scale, distributed systems using PySpark
- AWS subject matter expert with experience implementing data pipelines with AWS and running them in EMR
- Experience working on table formats using HUDI and Iceberg, and storing data in S3
- Leadership experience guiding/mentoring teams of high-level engineers
What We Offer:
- A dynamic and inclusive work environment
- Opportunities for professional growth and development
- A competitive salary and benefits package
We are committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances.
If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.
-
Senior Data Engineer
1 month ago
San Diego, California, United States BRAIN CORP Full timeJob Title: Senior Data EngineerBRAIN CORP is seeking a highly skilled Senior Data Engineer to lead the development of data-centric products that enhance our innovative BrainOS platform.This individual will be responsible for designing, constructing, and maintaining the systems and infrastructure to facilitate efficient collection, storage, and analysis of...
-
Senior Data Engineer
4 weeks ago
San Diego, California, United States Vets Hired Full timeAbout the JobAs a Senior Data Engineer at Vets Hired, you will play a critical role in building quality data pipelines that drive analytic solutions. Your expertise in data architecture, data engineering, and data analysis will enable the organization to advance its data-driven decision-making capabilities.Key ResponsibilitiesDesign, develop, and maintain...
-
Senior Data Engineer
4 weeks ago
San Diego, California, United States Lockheed Martin Full timeJob OverviewWe are seeking a highly skilled Senior Data Engineer to join our team at Lockheed Martin's Enterprise Business and Digital Transformation organization. As a key member of our Data Engineering development efforts, you will be responsible for delivering full-stack data solutions across the entire data processing pipeline.Key ResponsibilitiesDesign...
-
Senior Data Engineer
1 month ago
San Diego, California, United States BioPhase Full timePosition OverviewThe Senior Data Engineer will play a pivotal role in designing, implementing, and maintaining data pipelines that process and analyze large volumes of instrument data, clinical biomarker inputs, and patient information. This role requires a strong technical foundation, experience in startup environments, and the ability to navigate the...
-
Senior Opto-Mechanical Engineer
3 weeks ago
San Diego, California, United States Quartus Engineering Full timeWe are seeking a highly skilled Senior Opto-Mechanical Engineer to join our team at Quartus Engineering, a leading provider of engineering services and integrated hardware solutions. As an Opto-Mechanical Senior Engineer, you will design and develop robust mechanics for high-performance optical systems in challenging environments.Key Responsibilities:Design...
-
Senior Data Science Engineer
1 month ago
San Diego, California, United States Intuit Full timeJob SummaryWe are seeking a highly skilled Senior Machine Learning Engineer to join our team at Intuit. As a key member of our data science team, you will be responsible for designing, developing, and deploying machine learning models at scale using the latest industry tools and technologies.Key Responsibilities:Design and develop machine learning models...
-
Senior Data Engineer
1 month ago
San Francisco, California, United States Syntricate Technologies Full timeJob Title: Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team at Syntricate Technologies. As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining large-scale data pipelines using SQL and Snowflake. You will work closely with our data team to ensure the smooth...
-
Senior Data Engineer
4 weeks ago
San Diego, California, United States AppFolio, Inc Full timeAbout the RoleWe are seeking a highly skilled Sr. Data Engineer to join our growing Data Engineering and Operations team at AppFolio, Inc.The successful candidate will design, build, and operate next-generation data pipeline infrastructure based on Apache Kafka and its ecosystem.Key Responsibilities:Design and implement scalable and fault-tolerant data...
-
Senior Data Solutions Engineer
4 weeks ago
San Diego, California, United States Neurocrine Biosciences Full timeAbout the Role:We are seeking a highly skilled Senior Data Solutions Engineer to join our team at Neurocrine Biosciences. As a key member of our data operations team, you will be responsible for developing and maintaining platforms, patterns, pipelines, assets, and business intelligence solutions that support cross-functional data operations and...
-
Senior Data Engineer
4 weeks ago
San Francisco, California, United States SoFi Full timeShape a Brighter Financial Future with SoFiWe're a next-generation financial services company and national bank using innovative, mobile-first technology to help our millions of members reach their goals.The industry is going through an unprecedented transformation, and we're at the forefront. We're seeking a highly motivated Senior Software Engineer to join...
-
Senior Mechanical Engineer
4 weeks ago
San Diego, California, United States ATA Engineering Full timeJob DescriptionWe are seeking a highly skilled Senior Mechanical Engineer to join our team at ATA Engineering. The ideal candidate will have a strong background in mechanical engineering and experience with CAD and CAE software tools such as NX, Femap, and Simcenter Nastran.The successful candidate will be responsible for developing and executing an annual...
-
Senior Data Engineer
4 weeks ago
San Diego, California, United States CoStar Realty Information, Inc. Full timeJob DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at CoStar Realty Information, Inc. This is a unique opportunity to work on cloud-based data pipelines for machine learning, data processing with Apache Spark, and database development.ResponsibilitiesDesigning, building, testing, and deploying scalable, reusable, and...
-
Senior Data Architect
1 month ago
San Diego, California, United States NavAide Full timeJob Summary:NavAide is seeking a highly skilled Senior Data Engineer to lead and manage data engineering initiatives. The ideal candidate will have extensive experience in data architecture, data processing, and analytics solutions. This role is critical for supporting the Navy's mission by ensuring the integrity, availability, and security of data assets....
-
Senior Data Engineer
4 weeks ago
San Diego, California, United States BRAIN CORP Full timeJob Summary:Brain Corp is a leading AI company that creates transformative core technology for the robotics industry. We are seeking a highly skilled Senior Data Engineer to lead the development of data-centric products that enhance our innovative BrainOS platform.This individual will be responsible for designing, constructing, and maintaining the systems...
-
Senior Data Engineer
1 month ago
San Diego, California, United States Apple Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at Apple. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data solutions that support our software engineering group.Key Responsibilities:Develop and maintain Tableau dashboards and other analytical tools to provide visibility into...
-
Senior Data Engineer
3 weeks ago
San Francisco, California, United States Social Finance Ltd Full timeData Engineering RoleShape a brighter financial future with Social Finance Ltd. We're a next-generation financial services company and national bank using innovative, mobile-first technology to help our millions of members reach their goals.The industry is going through an unprecedented transformation, and we're at the forefront. We're proud to come to work...
-
Senior Data Engineer II
4 weeks ago
San Francisco, California, United States Cruise Full timeAbout the RoleWe're seeking a highly skilled Senior Data Engineer to join our team at Cruise, a self-driving service designed for the cities we love. As a Senior Data Engineer, you will be responsible for architecting and automating end-to-end data processing and data management lifecycle on Google Cloud Platform (GCP).You will collaborate with...
-
Senior Data Engineer
1 month ago
San Francisco, California, United States Archipelago Analytics Full timeArchipelago Analytics: Transforming Commercial Property InsuranceWelcome to Archipelago Analytics, where we're revolutionizing the commercial property insurance industry with our cutting-edge AI-powered data network.We're seeking a highly skilled Senior Data Engineer to join our team and help us take our document processing pipeline to the next level.As a...
-
Senior Data Scientist, Minerva
4 weeks ago
San Diego, California, United States Amazon Full timeAbout the RoleAs a Senior Data Scientist at Amazon, you will be part of the Minerva team, a cross-functional science, analytics, and ML org of highly experienced scientists, BI/data engineers, and software engineers. We build science-based systems for marketing and content-discovery for indie authors, fraud/abuse, and content risk for our publishing...
-
Senior Software Engineer
4 weeks ago
San Diego, California, United States ServiceNow Full timeJob OverviewServiceNow is seeking a highly skilled Senior Software Engineer - Data Platform to join our Cloud Platform Persistence team. As a key member of this group, you will be responsible for designing and developing scalable and efficient data storage solutions for our customers.Key Responsibilities:Design and implement data storage solutions using...