Databricks Data Engineer

7 days ago


New York, United States Fides Info Sys, Inc. Full time

Only U.S. Citizens with Top Secret Clearance should apply.

FIDESis looking for a Data Engineer with an active Top Secret Clearance andprior Databricks experience.

TheDatabricks Data Engineer is responsible fordesigning, building, and maintainingscalabledata pipelines and infrastructure.The successful candidate will have a solid foundation in data engineering principles, substantial experience with Databricks, and a proven track record of implementing robust data solutions.Responsibilities for this role include:

Collaborate with cross-functional teams to understand data requirements and design appropriate solutions using Databricks.

Develop and maintain efficient ETL pipelines to ingest, process, and transform large volumes of data.

Implement data quality checks and monitoring processes to ensure data integrity and reliability.

Optimize and tune Databricks clusters for performance and scalability.

Work closely with data scientists and analysts to support their data needs and enable advanced analytics.

Requirements

Bachelor's degree in Computer Science, Engineering, or related field.

3-5 years of experience in data engineering and big data technologies.

Active Top Secret (TS) clearance with SCI eligibility.

Experience supporting the Department of Defense (DoD) or Intelligence Community (IC).

Proficiency with Databricks, Python, and SQL.

Strong knowledge of data warehousing concepts and best practices.

Hands-on experience deploying and maintaining data pipelines using Databricks or similar tools.

Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.

Preferred Qualifications

Certification in Databricks and other relevant certifications.

Experience with other big data technologies such as Apache Spark, Hadoop, or Kafka.

Knowledge of containerization technologies such as Docker and Kubernetes.

Experience with version control systems such as Git.

#J-18808-Ljbffr


  • Engineering Manager

    4 weeks ago


    New York, United States Databricks Full time

    P-1313 Founded in late 2020 by a small group of machine learning engineers and researchers, MosaicML enables companies to securely fine-tune, train and deploy custom AI models on their own data, for maximum security and control. Compatible with all major cloud providers, the MosaicML platform provides maximum flexibility for AI development. Introduced in...

  • Databricks Engineer

    1 month ago


    New York, United States AI Talent Flow Full time

    Job DescriptionJob Description AI TalentFlow: Databricks EngineerFuel Innovation in Financial Services with Data Engineering ExpertiseAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with...

  • Databricks Engineer

    2 weeks ago


    New York, United States AI Talent Flow Full time

    Job DescriptionJob DescriptionAI TalentFlow: Databricks EngineerAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed to fueling the innovative spirit of our...


  • New York, United States Harnham Full time

    Senior Data ArchitectFully Remote6-Month Contract, possible extensions$95-$115/hrIt's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their software...


  • New York, United States Harnham Full time

    Senior Data ArchitectFully Remote6-Month Contract, possible extensions$95-$115/hrIt's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their software...

  • Data Engineer

    1 week ago


    New York, United States ICTerGezocht Full time

    Tip: Gebruik boolean search om woordentoe te voegen, te groeperen of uit te sluiten. Voorbeeld: "PHP Developer" +Java -Wordpress bekijk uitgebreideuitleg Soortgelijke vacatures omgeving Amsterdam . Stuur mij vacatures uit deze zoekopdracht Ontvang de nieuwste vacatures direct na plaatsing opICTerGezocht.nl per e-mail . Stuur mij vacatures uit deze...


  • New York, United States ASB Resources Full time

    • Build and operationalize complex data solutions, correct problems, apply transformations, and recommend data cleansing/quality solutions on the Databricks platform within Azure Cloud• Configuring the data lake, create and optimize data pipelines, integrating data from different sources• Implementing data transformations, ensuring data cleanliness•...


  • New York, United States Databricks Full time

    Company Description At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development. We do this by building and running the world’s best data and AI platform, so our customers can focus on the high value challenges that are central to their own missions. The...


  • New York, United States Databricks Full time

    Company Description At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development. We do this by building and running the world’s best data and AI platform, so our customers can focus on the high value challenges that are central to their own missions. The...


  • New York, United States Harnham Full time

    Senior Data Architect Fully Remote6-Month Contract, possible extensions$95-$115/hr It's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their...


  • New York, United States Harnham Full time

    Senior Data Architect Fully Remote6-Month Contract, possible extensions$95-$115/hr It's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their...


  • New York, United States Diverse Lynx Full time

    Title Sr. Data Architect Azure & Databricks Duration 12 Months Location New York / Weehawken, NJ (Day 1 onsite ) Mandatory Skills Databricks, Azure Datalake, PySpark, Java. No of Open Positions 1 Job Description : Minimum 12+ years of IT experience and at least 5+ years in cloud native data engineering solution design/architecture and...


  • New York, United States Diverse Lynx Full time

    Title Sr. Data Architect Azure & Databricks Duration 12 Months Location New York / Weehawken, NJ (Day 1 onsite ) Mandatory Skills Databricks, Azure Datalake, PySpark, Java. No of Open Positions 1 Job Description : Minimum 12+ years of IT experience and at least 5+ years in cloud native data engineering solution design/architecture and...


  • New York, United States Slalom Full time

    Who You'll Work With Slalom is looking for an experienced Data Architect to join our New York Market and work closely with the Data Platform, BI and Enterprise architecture teams to influence the technical direction of Slalom's data and analytics initiatives. You will build, own, and oversee the overall data architecture for the data platform - from data...

  • Data Engineer

    2 weeks ago


    New York, United States RiVi Consulting Group L.L.C Full time

    This role sits within the Data and Analytics Office which directs all aspects of data management within the Bank, and coordinates policies that will get promoted across the Federal Reserve System. Serves as a coordinating and centralizing body for data and analytics, organizing data access, establishing data foundations, while also driving innovation through...


  • New York, United States Ampcus Full time

    Role: Sr. Data Platform Engineer Location: New York, NY JOB DESCRIPTION: "Responsibilities: Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm Integrating technical functionality, ensuring data accessibility, accuracy, and security. Architect the Unity Catalog to provide centralized access control,...

  • Data Engineering Lead

    4 weeks ago


    New York, United States Munich Re America Services, Inc. Full time

    Munich Re America Services, Inc. seeks a Data Engineering Lead for its New York, NY location.Duties: Lead the design and development of comprehensive data architecture solutions using Azure and Databricks, ensuring they align with business goals and industry best practices. Design and implement robust data ingestion pipelines from a variety of sources into...

  • Data Engineering Lead

    4 weeks ago


    New York, United States Munich Re Full time

    THIS NOTICE IS BEING POSTED IN CONNECTION WITH THE FILING OF AN APPLICATION FOR PERMANENT ALIEN LABOR CERTIFICATION. ANY PERSON MAY PROVIDE DOCUMENTARY EVIDENCE BEARING ON THE APPLICATION TO THE CERTIFYING OFFICER OF THE U.S. DEPARTMENT OF LABOR AT THE FOLLOWING ADDRESS: U.S. Department of Labor Employment and Training Administration Office of Foreign Labor...


  • New York, New York, United States DataBricks Full time

    SLSQ424R186 While candidates in the listed location(s) are encouraged for this role, candidates in other locations will be considered. Databricks is seeking a Regional Director, Enterprise Sales for Financial Services to join our world-class sales organization. Reporting to our Regional VP of Enterprise Sales for Financial Services you will be responsible...

  • Databricks Tech Lead

    2 months ago


    New York, United States Diverse Lynx Full time

    Skill: Databricks Tech Lead Python, Databricks, perl, spark , Kubernetes, dockers and other cloud native tools and technologies. Data warehousing, AWS Cloud Business, Intelligence, Informatica and Databricks(data flow design, development, enhancement and maintenance). Real time data ingestion using Streamsets. Experience UDM Model. Experience in Dimensional...