Senior Data Engineer

17 hours ago


New York, United States The Custom Group of Companies Full time
Senior Data Engineer
New York, NY - Hybrid position Onsite 2 days a week (local to area)
Rate $100-$110 per hour
1+ year contract

Must be a USC to obtain L2 security clearance
No Corp to Corp or 3rd party agencies

We are looking for a Senior Data Engineer to join our team of professionals. The selected individual should be well versed in Databricks and AWS.

In this position you will:
Work on a team of 15 and have expert level skills in coding and testing pipelines in Databricks on AWS 6
Other solid skills Python, SQL, Spark, AWS, Trino

Position Description
  • Design, develop, monitor, test and maintain data pipelines in Databricks on an AWS 6 ecosystem with Delta Lake, Python, SQL and Starburst as the technology stack. Collaborate with cross-functional teams to understand data needs and translate them into effective data pipeline solutions.
  • Establish data quality checks and ensure data integrity and accuracy throughout the data lifecycle.
  • Automate testing of the data pipelines and configure as part of CICD
  • Optimize data processing and query performance for large-scale datasets within AWS and Databricks environments.
  • Document data engineering processes, architecture, and configurations.
  • Troubleshooting and debugging data-related issues on the AWS Databricks platform.
  • Integrating Databricks with other AWS products such as SNS, SQS, and MSK.
Qualifications
  • Minimum of 5 years of experience in data engineering roles, with a focus on AWS and Expert level skills coding and testing Databricks in an AWS 6 Environment.
  • Experience with Python, PySpark, SQL, Spark
  • Gitlab with CI/CD.
  • AWS Services like S3, RDS, Lambda, SQS, SNS, MSK is required
  • Databricks Certified is highly desired
  • Strong SQL skills to perform data analysis and understanding of source data.
  • Experience with data pipeline orchestration tools
  • Communication, Collaboration, experimentation, inquisitiveness, detailed oriented
Must be Databricks Certified

  • New York, United States VAST Data Full time

    VAST Data is looking for a Senior Systems Engineer to join our growing team! This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence. "VAST's data management vision is the future of the...

  • Senior Data Engineer

    17 hours ago


    New York, United States TechnoGen Full time

    Please Note: As of July 22, 2021, our team will require that all candidate submissions include a LinkedIn profile. Please do not submit any candidates that do not have a LinkedIn. Kforce has a client in Brooklyn, NY that is seeking a Senior Data Engineer. Our team is headquartered in Brooklyn but has a remote-first culture and we encourage remote applicants...

  • Senior Data Engineer

    10 hours ago


    New York, United States Match Group Full time

    As a Senior Data Engineer on the Product Platform team, you will be instrumental in designing, building, and optimizing data pipelines and products that support foundational product services that are crucial to the operation of Hinge. Your work will have a direct impact on the reliability and scalability of these services, which are essential for delivering...


  • New York, United States Hudson Data Full time

    Job Title: Sr. Azure Data Engineer Location: NYC, NY Duration: Long Term W2 Role Job Description: The candidate should have hands-on experience with various Azure services including but not limited to Azure Data Factory, Databricks, Data Lake, and Power BI. They should be able to design, build, and maintain ETL pipelines, manage data lakes, and...


  • New York, New York, United States Acquired Data Solutions Full time

    Job Title: Senior Security Systems EngineerWe are seeking a highly skilled Senior Security Systems Engineer to join our team at Acquired Data Solutions, Inc. in Egg Harbor, NJ.Job Summary:The successful candidate will be responsible for performing in-depth engineering analysis of security-related systems to determine their potential for further development,...

  • Senior Data Engineer

    2 months ago


    New York, United States Coda Search│Staffing Full time

    A tech startup company in the healthcare space is looking to hire a Senior Data Engineer.Duties & Essential Job FunctionsDevelop, release, and maintain high quality data pipelines using Python, FiveTran, DBT, and SnowflakeOwn and guide the development of our data infrastructureDevelop custom integrations using DagsterConfigure reverse ETL integrations using...

  • Senior Data Engineer

    10 hours ago


    New York, United States CV Library Full time

    SENIOR DATA ENGINEER AT BOUTIQUE ANALYTICS FOCUSED HEDGE FUNDBase Salary is $175K to $200K (based upon candidate and experience); Plus Bonus; and Excellent BenefitsIncredible Organization with Equally Incredible People | New York (SoHo), NY | Onsite Four (4) Days per WeekMulti-office, solidly established, international, boutique analytics-focused hedge fund...

  • Senior Data Engineer

    13 hours ago


    New York, United States MDCalc Full time

    Overview MDCalc is the most trusted and broadly used medical reference for clinical decision making tools, used by millions of clinical users worldwide. We're the #1 medical reference for clinical decision tools and content and we're used by more than 65% of US doctors quarterly. To continue to further accelerate and steward this growth, we are seeking a...

  • Senior Data Engineer

    13 hours ago


    New York, United States MDCalc Full time

    Overview MDCalc is the most trusted and broadly used medical reference for clinical decision making tools, used by millions of clinical users worldwide. We're the #1 medical reference for clinical decision tools and content and we're used by more than 65% of US doctors quarterly. To continue to further accelerate and steward this growth, we are seeking a...

  • Senior Data Engineer

    2 months ago


    New York, United States Capital One Full time

    Center 1 (19052), United States of America, McLean, VirginiaSenior Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who...

  • Senior Data Engineer

    3 months ago


    New York, United States Roots Automation Full time

    Senior Data Engineer Company Overview: At Roots, our mission is to make work more human. We are developing fully autonomous, AI-powered Digital Coworkers that streamline tedious and repetitive tasks for the enterprise. By tackling core challenges in natural language understanding and computer vision, we are building an automation product that embodies the...

  • Senior Data Engineer

    1 month ago


    New York, New York, United States VALiNTRY Full time

    Job SummaryVALiNTRY is seeking a highly skilled Senior Analytics Engineer to support our Business Intelligence lead in managing the enterprise's data pipeline and ecosystem from end to end. As a key member of our team, you will collaborate with cross-functional business partners to understand their needs and transform raw data into actionable insights.Key...


  • New York, New York, United States Vimeo Full time

    Job Title: Senior Data Infrastructure EngineerJob Summary:Vimeo is seeking a highly skilled Senior Data Infrastructure Engineer to manage and optimize our vector databases and data stores, including Elastic Search and AWS OpenSearch. The ideal candidate will have a strong background in cloud platforms and SRE principles, as well as experience leading data...

  • Senior Data Engineer

    12 hours ago


    New York, United States Delphi-US Full time

    Job Title: Senior Data Engineer - Martech (Contract) - Job#5214 Location: Remote We are looking for a Senior Data Engineer to join our growing team and help us transition our legacy Martech pipelines to our modern ELT architecture. You will work with BASH, Perl, dbt, Fivetran, Snowflake, GitHub, and modern CI/CD & IAC tools to design, develop, test, deploy,...


  • New York, United States Delphi-US Full time

    Job Title: Senior ELT Data Engineer (Contract) - Job#5213 Location: Remote Additional Information: 12 month+ Contract, EST hours We are looking for a Senior Data Engineer to join our growing team and help us build our new ELT architecture from the ground up. You will work with dbt, Fivetran, Snowflake, GitHub, and modern CI/CD & IAC tools to design, develop,...

  • Senior Data Engineer

    2 months ago


    New York, United States Capital One Full time

    114 5th Ave (22114), United States of America, New York, New YorkSenior Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors,...

  • Senior Data Engineer

    4 weeks ago


    new york city, United States Coda Search│Staffing Full time

    A tech startup company in the healthcare space is looking to hire a Senior Data Engineer.Duties & Essential Job FunctionsDevelop, release, and maintain high quality data pipelines using Python, FiveTran, DBT, and SnowflakeOwn and guide the development of our data infrastructureDevelop custom integrations using DagsterConfigure reverse ETL integrations using...

  • Senior Data Engineer

    1 month ago


    new york city, United States Coda Search│Staffing Full time

    A tech startup company in the healthcare space is looking to hire a Senior Data Engineer.Duties & Essential Job FunctionsDevelop, release, and maintain high quality data pipelines using Python, FiveTran, DBT, and SnowflakeOwn and guide the development of our data infrastructureDevelop custom integrations using DagsterConfigure reverse ETL integrations using...


  • New York, United States Publicis Sapient Full time

    Job Description Your Impact: The purpose of the Senior Associate, Data Science is to work with clients undergoing a data-driven transformation (DDT) across the globe. The Senior Associate will accelerate and drive the DDT strategy for clients, working in partnership with digital directors, client teams, and practice capability. Furthermore, the...


  • New York, United States Harnham Full time

    Senior Data Platform EngineerFully Remote9-Month Contract$75-$100/hr It's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their software to join...