Sr. Data Pipeline Engineer
1 week ago
BS or MS in Computer Science, a related field, or equivalent industry experience
. 3 years of professional experience engineering complex, high-volume data pipelines using SQL, Python, and Airflow
. 3 years of experience building cloud scalable and high-performance data lake / data warehouse solutions using AWS products - S3, Athena, Glue, and EMR
. Experience with binary data serialization formats such as Parquet
. Deep understanding of data structures and algorithms
. Understanding of code versioning tools such as GIT
. Have a passion for data solutions
Preferred Qualifications Nice to Have:
. Exposure to AWS cloud data pipeline tools such as Managed Airflow and Glue
. Experience integrating with Ad Tech platforms such as Operative and STAQ
. Exposure and opinions regarding alternate orchestration tooling beyond Airflow
. Understanding of SOX compliance needs and how they affect system design.
. Have worked with a variety of Airflow Operator types, including REST, Lambda, ECS
. Can flex between Python and Javascript/Typescript.
Technical Environment
. Aurora/Hive (databases)
. Spark (large-scale data processing)
. Airflow (workflow management)
. Docker (software packaging and delivery)
. AWS (development and hosting) Required Education BS/MS in Computer Science or sim. field Preferred Education Additional Information - This team is considered Hybrid so looking for someone that's close to a Hulu Office. So.Cal/Seattle/NYC - Can be remote to start but expectation is in office part time if needed.
-
Sr. Data Engineer
2 weeks ago
New York, United States Planet Technology USA Full timeSr. Data Engineer- SQL Server Expert with Investment experience Fulltime Job in NY - Hybrid-Please do not apply if not local to NY No C2C - Must be able to work without sponsorship Position: Sr Data Engineer - Strong SQL Application Development You will be working with a skilled team and help in defining the future of their data platform environment. You...
-
Sr. Data Engineer
3 weeks ago
New York, United States Planet Technology Full timeSr. Data Engineer- SQL Server Expert with Investment experienceFulltime Job in NY - Hybrid-Please do not apply if not local to NYNo C2C - Must be able to work without sponsorshipPosition: Sr Data Engineer - Strong SQL Application DevelopmentYou will be working with a skilled team and help in defining the future of their data platform environment. You will be...
-
Sr. Data Engineer
3 days ago
New York, United States Planet Technology Full timeSr. Data Engineer- SQL Server Expert with Investment experienceFulltime Job in NY - Hybrid-Please do not apply if not local to NYNo C2C - Must be able to work without sponsorshipPosition: Sr Data Engineer - Strong SQL Application DevelopmentYou will be working with a skilled team and help in defining the future of their data platform environment. You will be...
-
Sr. Data Engineer
2 weeks ago
New York, United States Planet Technology Full timeSr. Data Engineer- SQL Server Expert with Investment experienceFulltime Job in NY - Hybrid-Please do not apply if not local to NYNo C2C - Must be able to work without sponsorshipPosition: Sr Data Engineer - Strong SQL Application DevelopmentYou will be working with a skilled team and help in defining the future of their data platform environment. You will be...
-
Sr. Data Engineer
3 weeks ago
New York, United States Dexian Full timeJob Summary:Dexian is seeking a Sr. Data Engineer for an opportunity with a client located in New York, NY. Responsibilities:Implement new mechanisms for modeling and predicting reader behaviorDesign and implement experiments and models to answer targeted questionsLead technical efforts to modernize and replace legacy reporting solutionsLead daily...
-
Sr. Data Platform Engineer
6 days ago
New York, United States Ampcus Full timeRole: Sr. Data Platform Engineer Location: New York, NY JOB DESCRIPTION: "Responsibilities: Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm Integrating technical functionality, ensuring data accessibility, accuracy, and security. Architect the Unity Catalog to provide centralized access control,...
-
Senior Data Engineer
4 days ago
New York, United States Optomi Full timeSr. Data Engineer - (Scala / Spark)100% Remote, must sit in hub area - New York, Orlando, San Jose, Seattle, Santa MonicaW2 Only - USC or GC Holder - Due to the nature of this opportunity, please do not apply for this position if you need to do so on a C2C basis or through a 3rd party.4 month contract, will likely extend past the fiscal yearOptomi, in...
-
Sr. Data Engineer
3 weeks ago
New York, New York, United States HealthFirst Full timeHello Thank you for considering Healthfirst for your next career opportunity. Who we are: Healthfirst was founded by a collection of major healthcare systems in the New York area to emphasize high quality of care, and better patient outcomes. We have grown into New York's largest not-for-profit health insurer, and cover over 1.7 million members in New York...
-
Sr. Data Engineer
3 weeks ago
New York, New York, United States HealthFirst Full timeHello Thank you for considering Healthfirst for your next career opportunity. Who we are: Healthfirst was founded by a collection of major healthcare systems in the New York area to emphasize high quality of care, and better patient outcomes. We have grown into New York's largest not-for-profit health insurer, and cover over 1.7 million members in New York...
-
Sr Data Engineer
4 weeks ago
New York, United States enexusglobal Full timeJob DescriptionJob DescriptionWe are looking to hire a Sr. Data Engineer in New York City, NY Summary:We are looking to hire Sr. Data Engineer with a financial background. Who will play a crucial role in a cloud-based data warehousing platform, to manage and analyze financial data.Job ResponsibilitiesDesign and develop robust, scalable software solutions for...
-
Senior Data Engineer
1 week ago
New York, United States Sun Pharma Full timeAs a member of our Software Engineering Group, we look first and foremost for people passionate about solving business problems through innovation and engineering practices. You'll be required to apply your depth of knowledge and expertise to all aspects of the software development lifecycle and partner continuously with your many stakeholders daily to stay...
-
Sr. Data Architect
2 days ago
New York, New York, United States TWO95 International Full timeHi,Title – Sr.Data ArchitectPosition – Contract to Hire (3+ Months)Location – RemoteRate - $Open (Best Possible)ENVIRONMENTNeo4j, Gremlin , Java, Node.JS, AWS Lambda, AWS Kinesis, AWS lifecycle management with GitLab, Kubernetes, Other AWS servicesREQUIREMENTS7+ years of experience with Data Engineering/Architecture5+ years of experience engineering...
-
Data Engineer, Sr. Consultant
2 weeks ago
New York, United States MigrationIT Full timeRequirements Cloud computing platforms: AWS/Azure/GCP Programming languages: Python/Java/Scala & Unix Shell scripting Data warehouse technologies: Redshift/Snowflake/Big Query Data lake technologies: Hadoop, Apache Beam, Spark, and Hive Orchestration: Airflow/Control-M/(Any Orchestration tool) Machine learning frameworks: TensorFlow, PyTorch, and...
-
Sr. Data Engineer
3 weeks ago
New Bern, North Carolina, United States Core Scientific Full timeWho We Are Bold. Unapologetic. Hardworking. We are building something special. Put simply: We power and secure the next global financial system. At Core Scientific, we host best-in-class enterprise level blockchain hardware in expansive, state-of-the-art data center facilities. Our proprietary software stack automates blockchain mining optimization,...
-
Sr. Data Engineer
1 month ago
New Bern, North Carolina, United States Core Scientific Full timeWho We Are Bold. Unapologetic. Hardworking. We are building something special. Put simply: We power and secure the next global financial system. At Core Scientific, we host best-in-class enterprise level blockchain hardware in expansive, state-of-the-art data center facilities. Our proprietary software stack automates blockchain mining optimization,...
-
Sr. Azure Engineer
1 month ago
New York, United States Creative Data Resources Full timeRequired 8+ years overall IT experience. 4+ years in a cloud engineering role using service and hosting solutions such as private/public cloud IaaS, PaaS and SaaS platforms. SME Level knowledge of: ASP.NET ,C#,XML,YAML,SQL,CSS, JavaScript Angular UI development experience Azure SQL Server Stored Procedure Development (TSQL) Azure Services Knowledge Azure...
-
Sr. Azure Engineer
3 weeks ago
New York, United States Creative Data Resources Full timeRequired 8+ years overall IT experience. 4+ years in a cloud engineering role using service and hosting solutions such as private/public cloud IaaS, PaaS and SaaS platforms. SME Level knowledge of: ASP.NET ,C#,XML,YAML,SQL,CSS, JavaScript Angular UI development experience Azure SQL Server Stored Procedure Development (TSQL) Azure Services Knowledge Azure...
-
Sr. Azure Engineer
2 days ago
New York, New York, United States Creative Data Resources Full timeRequired 8+ years overall IT experience. 4+ years in a cloud engineering role using service and hosting solutions such as private/public cloud IaaS, PaaS and SaaS platforms. SME Level knowledge of: ASP.NET ,C#,XML,YAML,SQL,CSS, JavaScript Angular UI development experience Azure SQL Server Stored Procedure Development (TSQL) Azure Services Knowledge Azure...
-
Sr. DevOps Engineer
5 days ago
New York, United States Diverse Lynx Full timeTitle: Sr. Devops Engineer (Cloud Implementation with GCP) (2 Roles) Location: Berkley Heights, NJ / Alpharetta, GA (5 Days Onsite) Job Description: 6-10 years of overall experience 3+ years of Devops experience on GCP Strong cloud engineering skills, experienced and certified on GCP Strong understanding of cloud security principles, architecture, and best...
-
Data Engineer-Testing
2 days ago
New York, New York, United States Federal Reserve System Full timeCome Work with Us at the Federal Reserve Bank of New YorkWorking at the Federal Reserve Bank of New York puts you right in the middle of all things finance, giving you a special view of both national and international markets and economies. You'll be part of a diverse team of seasoned professionals dedicated to the strength and prosperity of our economic and...