Databricks Engineer

4 weeks ago

New York, United States AI TalentFlow Full time

AI TalentFlow: Databricks Engineer

Fuel Innovation in Financial Services with Data Engineering Expertise

About AI TalentFlow:

AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed to fueling the innovative spirit of our clients with top-tier talent in AI, Data, Analytics, Product, and Agile.

About the Role:

Are you a passionate data engineer with deep experience in the financial services industry? Do you thrive in a fast-paced environment and possess a strong understanding of data pipelines, machine learning, and cloud platforms? If so, this role might be the perfect fit for you

In this exciting opportunity, you will play a critical role in leveraging data to drive innovation within the financial services sector. You will collaborate with business stakeholders, data scientists, and software engineers to design, develop, and implement efficient and scalable data pipelines on Databricks. You will also be responsible for identifying and building valuable use cases for data within the financial services domain (e.g., trading, broker-dealer, asset management, investment management). This role offers the chance to make a real impact by contributing to the development of robust and repeatable data-driven solutions that deliver tangible business value.

Responsibilities:

  • Partner with business stakeholders to understand their data needs and identify high-impact use cases for data within the financial services industry.
  • Design, develop, and implement efficient and scalable data pipelines using Apache Spark on Databricks.
  • Develop and maintain data pipelines for various financial services domains, such as trading, risk management, portfolio analysis, and regulatory compliance.
  • Perform data cleaning and transformation tasks to ensure data quality and consistency.
  • Build robust and reusable data pipelines that adhere to best practices and coding standards.
  • Collaborate with data scientists to prepare data for machine learning models.
  • Develop and integrate Python scripts and SQL queries for data manipulation and analysis.
  • Leverage cloud platforms like AWS and Snowflake for data storage and integration.
  • Monitor and maintain data pipelines to ensure smooth operation and identify potential issues.
  • Stay up-to-date on the latest advancements in data engineering technologies and financial services trends.
  • Document data pipelines and processes for clear communication and knowledge sharing.

Qualifications:

  • 3+ years of experience as a Data Engineer or related role.
  • Strong understanding of data warehousing concepts and data modeling principles.
  • Extensive experience with Apache Spark and Databricks (or similar data processing frameworks).
  • Proficiency in Python and SQL, with experience in data manipulation and analysis libraries.
  • Experience building data pipelines for the financial services industry (e.g., trading, broker-dealer, asset management, investment management) is a strong plus.
  • Familiarity with cloud platforms like AWS and Snowflake for data storage and integration.
  • Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.
  • A passion for learning and staying up-to-date on the latest advancements in data engineering technologies.

Ready to join a dynamic team and leverage your expertise to shape the future of financial services? We encourage you to apply

  • Engineering Manager

    3 weeks ago

    New York, United States Databricks Full time

    P-1313 Founded in late 2020 by a small group of machine learning engineers and researchers, MosaicML enables companies to securely fine-tune, train and deploy custom AI models on their own data, for maximum security and control. Compatible with all major cloud providers, the MosaicML platform provides maximum flexibility for AI development. Introduced in...

  • Databricks Engineer

    4 weeks ago

    New York, United States AI TalentFlow Full time

    AI TalentFlow: Databricks EngineerFuel Innovation in Financial Services with Data Engineering ExpertiseAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed...

  • Databricks Engineer

    3 months ago

    New York, United States AI TalentFlow Full time

    AI TalentFlow: Databricks EngineerFuel Innovation in Financial Services with Data Engineering ExpertiseAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed...

  • Databricks Engineer

    2 months ago

    New York, United States AI TalentFlow Full time

    AI TalentFlow: Databricks EngineerFuel Innovation in Financial Services with Data Engineering ExpertiseAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed...

  • Databricks Engineer

    4 weeks ago

    New York, United States AI Talent Flow Full time

    Job DescriptionJob Description AI TalentFlow: Databricks EngineerFuel Innovation in Financial Services with Data Engineering ExpertiseAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with...

  • Databricks Engineer

    7 days ago

    New York, United States AI Talent Flow Full time

    Job DescriptionJob DescriptionAI TalentFlow: Databricks EngineerAbout AI TalentFlow:AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed to fueling the innovative spirit of our...

  • Genai Research Scientist

    2 weeks ago

    New York, United States Databricks Full time

    Company Description At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development. We do this by building and running the world’s best data and AI platform, so our customers can focus on the high value challenges that are central to their own missions. The...

  • Staff Genai Research Scientist

    7 days ago

    New York, United States Databricks Full time

    Company Description At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development. We do this by building and running the world’s best data and AI platform, so our customers can focus on the high value challenges that are central to their own missions. The...

  • Data Engineer

    20 hours ago

    New York, United States ICTerGezocht Full time

    Tip: Gebruik boolean search om woordentoe te voegen, te groeperen of uit te sluiten. Voorbeeld: "PHP Developer" +Java -Wordpress bekijk uitgebreideuitleg Soortgelijke vacatures omgeving Amsterdam . Stuur mij vacatures uit deze zoekopdracht Ontvang de nieuwste vacatures direct na plaatsing opICTerGezocht.nl per e-mail . Stuur mij vacatures uit deze...

  • Databricks - Data Engineer

    4 days ago

    New York, United States ASB Resources Full time

    • Build and operationalize complex data solutions, correct problems, apply transformations, and recommend data cleansing/quality solutions on the Databricks platform within Azure Cloud• Configuring the data lake, create and optimize data pipelines, integrating data from different sources• Implementing data transformations, ensuring data cleanliness•...

  • Databricks Technical Lead

    4 weeks ago

    New Brunswick, United States Diverse Lynx Full time

    Certification: Databricks certified Data Engineer Professional, SQL, Azure data Engineer, Python Primary Skillset: Apache Spark: Strong understanding and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming. Databricks: Proficiency in using Azure Databricks, including setting up and managing clusters, notebooks, and jobs. Python and...

  • Senior Data Architect, Databricks Expert

    3 weeks ago

    New York, United States Harnham Full time

    Senior Data Architect Fully Remote6-Month Contract, possible extensions$95-$115/hr It's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their...

  • Senior Data Architect, Databricks Expert

    2 weeks ago

    New York, United States Harnham Full time

    Senior Data Architect Fully Remote6-Month Contract, possible extensions$95-$115/hr It's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company is seeking an individual who is ready to combine their data ops experience with their...

  • Sr. Data Architect Azure

    2 weeks ago

    New York, United States Diverse Lynx Full time

    Title Sr. Data Architect Azure & Databricks Duration 12 Months Location New York / Weehawken, NJ (Day 1 onsite ) Mandatory Skills Databricks, Azure Datalake, PySpark, Java. No of Open Positions 1 Job Description : Minimum 12+ years of IT experience and at least 5+ years in cloud native data engineering solution design/architecture and...

  • Sr. Data Architect Azure

    3 weeks ago

    New York, United States Diverse Lynx Full time

    Title Sr. Data Architect Azure & Databricks Duration 12 Months Location New York / Weehawken, NJ (Day 1 onsite ) Mandatory Skills Databricks, Azure Datalake, PySpark, Java. No of Open Positions 1 Job Description : Minimum 12+ years of IT experience and at least 5+ years in cloud native data engineering solution design/architecture and...

  • Databricks Data Architect

    3 weeks ago

    New York, United States Slalom Full time

    Who You'll Work With Slalom is looking for an experienced Data Architect to join our New York Market and work closely with the Data Platform, BI and Enterprise architecture teams to influence the technical direction of Slalom's data and analytics initiatives. You will build, own, and oversee the overall data architecture for the data platform - from data...

  • Sr. Data Platform Engineer

    2 weeks ago

    New York, United States Ampcus Full time

    Role: Sr. Data Platform Engineer Location: New York, NY JOB DESCRIPTION: "Responsibilities: Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm Integrating technical functionality, ensuring data accessibility, accuracy, and security. Architect the Unity Catalog to provide centralized access control,...

  • Data Engineering Lead

    3 weeks ago

    New York, United States Munich Re America Services, Inc. Full time

    Munich Re America Services, Inc. seeks a Data Engineering Lead for its New York, NY location.Duties: Lead the design and development of comprehensive data architecture solutions using Azure and Databricks, ensuring they align with business goals and industry best practices. Design and implement robust data ingestion pipelines from a variety of sources into...

  • Data Engineer

    3 days ago

    New York, United States RiVi Consulting Group L.L.C Full time

    This role sits within the Data and Analytics Office which directs all aspects of data management within the Bank, and coordinates policies that will get promoted across the Federal Reserve System. Serves as a coordinating and centralizing body for data and analytics, organizing data access, establishing data foundations, while also driving innovation through...

  • Cloud Data Engineer

    22 hours ago

    New York, United States Fides Info Sys, Inc. Full time

    Only U.S. Citizens with Top Secret Clearance should apply. FIDESis looking for a Data Engineer with an active Top Secret Clearance andprior cloud experience. TheCloud Data Engineer is responsible fordesigning, building, and maintainingscalabledata pipelines and infrastructure.The successful candidate will have a solid foundation in data engineering...