Sr. Data Platform Engineer

2 weeks ago

New York, United States Ampcus Full time

Role: Sr. Data Platform Engineer
Location: New York, NY

JOB DESCRIPTION:
"Responsibilities:
Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm
Integrating technical functionality, ensuring data accessibility, accuracy, and security.
Architect the Unity Catalog to provide centralized access control, auditing, lineage, and data discovery capabilities across Databricks workspaces.
Define and organize data assets (structured and unstructured) within the Unity Catalog.
Enable data analysts and etl engineers to Client and classify data, notebooks, dashboards, and files across clouds and platforms.
Implement a single permission model for data and AI assets.
Define access policies at a granular level (rows, columns, features) to ensure secure and consistent access management across workspaces and platforms.
Leverage Delta Sharing to enable easy data sharing across regions, and platforms.
Ensure that data and AI assets can be securely shared with minimal replication, maintaining a unified experience for users.
Monitoring and Observability: utilize AI to automate monitoring, diagnose errors, and maintain data and quality.
Set up alerts for personally identifiable information (PII) detection, and operational intelligence.
Work closely with data scientists, analysts, and engineers to promote adoption of the Unity Catalog.
Provide training and documentation to ensure effective usage and compliance with governance policies.
Skills:
Designed data warehouse and data lake solutions along with data processing Pipeline using PySpark using Databricks
Performed Data Modelling on Databricks [Delta Table] for transactional and analytical need.
Designed and developed pipelines to load data to Data Lake
Databricks Platform Proficiency, including its components like Databricks SQL, Delta Live Tables, Databricks Repos, and Task Orchestration.
Deep understanding of data governance principles, especially related to data cataloging, access control, lineage, and metadata management.
Strong SQL skills for querying and managing data
Ability to design and optimize data models for structured and unstructured data.
Understand how to manage compute resources, including clusters and workspaces.
Ability to adapt to changes and emerging trends in data engineering and governance.
Involved in hands on development and configuration of Unity Catalog"

Thanks

  • Sr. Data Engineer

    2 weeks ago

    New York, United States Planet Technology USA Full time

    Sr. Data Engineer- SQL Server Expert with Investment experience Fulltime Job in NY - Hybrid-Please do not apply if not local to NY No C2C - Must be able to work without sponsorship Position: Sr Data Engineer - Strong SQL Application Development You will be working with a skilled team and help in defining the future of their data platform environment. You...

  • Sr. Data Engineer

    4 weeks ago

    New York, United States Planet Technology Full time

    Sr. Data Engineer- SQL Server Expert with Investment experienceFulltime Job in NY - Hybrid-Please do not apply if not local to NYNo C2C - Must be able to work without sponsorshipPosition: Sr Data Engineer - Strong SQL Application DevelopmentYou will be working with a skilled team and help in defining the future of their data platform environment. You will be...

  • Sr. Data Engineer

    3 weeks ago

    New York, United States Planet Technology Full time

    Sr. Data Engineer- SQL Server Expert with Investment experienceFulltime Job in NY - Hybrid-Please do not apply if not local to NYNo C2C - Must be able to work without sponsorshipPosition: Sr Data Engineer - Strong SQL Application DevelopmentYou will be working with a skilled team and help in defining the future of their data platform environment. You will be...

  • Sr. Data Engineer

    7 days ago

    New York, United States Planet Technology Full time

    Sr. Data Engineer- SQL Server Expert with Investment experienceFulltime Job in NY - Hybrid-Please do not apply if not local to NYNo C2C - Must be able to work without sponsorshipPosition: Sr Data Engineer - Strong SQL Application DevelopmentYou will be working with a skilled team and help in defining the future of their data platform environment. You will be...

  • Sr Data Engineer

    1 month ago

    New York, United States enexusglobal Full time

    Job DescriptionJob DescriptionWe are looking to hire a Sr. Data Engineer in New York City, NY Summary:We are looking to hire Sr. Data Engineer with a financial background. Who will play a crucial role in a cloud-based data warehousing platform, to manage and analyze financial data.Job ResponsibilitiesDesign and develop robust, scalable software solutions for...

  • Cloud Data Platform Engineer

    1 week ago

    New York, United States Global Channel Management Full time

    Cloud Data Platform Engineer needs 5+ years' implementing data applications or data platforms with BigData/Hadoop, Python/Java/Spark full stack, etc. 5+ year support experience of Big Cloud Data Platform Engineer requires: 3 days a week, hybrid Locations: Iselin, NJ, Charlotte, NC, NY, NY Data Engineering 60-70 %. Infrastructure 30-40%. (Moving Data in & Out...

  • Cloud Data Platform Engineer

    7 days ago

    New York, New York, United States Global Channel Management Full time

    We are looking for a skilled Cloud Data Platform Engineer with extensive experience in data applications and platforms.5+ years of hands-on experience implementing data applications or platforms involving BigData/Hadoop, Python/Java, and Spark among others.Experience supporting Big Cloud Data Platforms for 5+ years.This position requires:Part-time commitment...

  • Sr. Data Engineer

    4 weeks ago

    New York, United States Dexian Full time

    Job Summary:Dexian is seeking a Sr. Data Engineer for an opportunity with a client located in New York, NY. Responsibilities:Implement new mechanisms for modeling and predicting reader behaviorDesign and implement experiments and models to answer targeted questionsLead technical efforts to modernize and replace legacy reporting solutionsLead daily...

  • Sr. Data Engineer

    4 weeks ago

    New York, New York, United States HealthFirst Full time

    Hello Thank you for considering Healthfirst for your next career opportunity. Who we are: Healthfirst was founded by a collection of major healthcare systems in the New York area to emphasize high quality of care, and better patient outcomes. We have grown into New York's largest not-for-profit health insurer, and cover over 1.7 million members in New York...

  • Sr. Data Engineer

    3 weeks ago

    New York, New York, United States HealthFirst Full time

    Hello Thank you for considering Healthfirst for your next career opportunity. Who we are: Healthfirst was founded by a collection of major healthcare systems in the New York area to emphasize high quality of care, and better patient outcomes. We have grown into New York's largest not-for-profit health insurer, and cover over 1.7 million members in New York...

  • Cloud Data Platform Engineer Lead

    3 days ago

    New York, United States Bank of America Full time

    Description : At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. Responsible Growth is how we run our company and how we deliver for our clients, teammates, communities and shareholders every day. One of the keys to driving Responsible Growth is being a great place to work...

  • Lead Software Engineer, Data Platform

    3 weeks ago

    New York, New York, United States Personio Full time

    The Role: How you'll make an impact at Personio This position is based in New York City, and in some cases a remote role can be considered for the U.S. East Coast (CT, FL, MA, NJ, SC and VT). At Personio, your work transforms the way millions of people experience work every day. Join our Product & Technology team, where we drive our customers' outcomes by...

  • Lead Software Engineer, Data Platform

    4 weeks ago

    New York, New York, United States Personio Full time

    The Role: How you'll make an impact at Personio This position is based in New York City, and in some cases a remote role can be considered for the U.S. East Coast (CT, FL, MA, NJ, SC and VT). At Personio, your work transforms the way millions of people experience work every day. Join our Product & Technology team, where we drive our customers' outcomes by...

  • Data Security Platform Engineer/Architect

    6 days ago

    New York, New York, United States Experis Full time

    We have an excellent opportunity with our direct client.Please see below Job description for your referenceData Security Platform Engineer/ArchitectRemoteContract on W2Job descriptionHave the need for a resource to help a client with to support HSM build-out and management.Need someone with experience with EKMS, HSMs, but particularly around the Thales...

  • Data Security Platform Engineer/Architect

    3 weeks ago

    New York, United States Experis Full time

    We have an excellent opportunity with our direct client. Please see below Job description for your reference   Data Security Platform Engineer/Architect Remote Contract on W2   Job description Have the need for a resource to help a client with to support HSM build-out and management. Need someone with experience with EKMS, HSMs, but particularly around the...

  • Data Security Platform Engineer/Architect

    2 weeks ago

    New York, New York, 10017, Manhattan, NY, United States Experis Full time

    We have an excellent opportunity with our direct client. Please see below Job description for your reference Data Security Platform Engineer/Architect Remote Contract on W2 Job description Have the need for a resource to help a client with to support HSM build-out and management. Need someone with experience with EKMS, HSMs, but particularly around the...

  • Senior Data Engineer, Platform Engineering

    4 weeks ago

    New York, United States Stack Overflow Full time

    Job DescriptionJob DescriptionEvery developer has a tab open on Stack Overflow. We are one of the most popular websites in the world - a community-based space focused on increasing productivity, decreasing cycle times, accelerating time to market, and protecting institutional knowledge. Innovation is at the heart of everything we do. We embrace...

  • Senior Software Engineer

    2 weeks ago

    New York, United States Informa PLC Full time

    Job Description Curinos is currently seeking a  Senior Software Engineer  to join the Data Platforms group. This team owns the shared data ecosystems upon which our B2B SaaS applications are built. As a company that specializes in data-driven insights, the reliability, scalability, and effectiveness of our data platform is integral to our product...

  • Data Engineer, Sr. Consultant

    2 weeks ago

    New York, United States MigrationIT Full time

    Requirements Cloud computing platforms: AWS/Azure/GCP Programming languages: Python/Java/Scala & Unix Shell scripting Data warehouse technologies: Redshift/Snowflake/Big Query Data lake technologies: Hadoop, Apache Beam, Spark, and Hive Orchestration: Airflow/Control-M/(Any Orchestration tool) Machine learning frameworks: TensorFlow, PyTorch, and...

  • Data Security Platform Engineer/Architect

    2 weeks ago

    New York, United States Manpower Group Inc. Full time

    We have an excellent opportunity with our direct client. Please see below Job description for your reference Data Security Platform Engineer/Architect Remote Contract on W2 Job description Have the need for a resource to help a client with to support HSM build-out and management. Need someone with experience with EKMS, HSMs, but particularly around the...