Senior Data Engineer-Databricks, Python
2 weeks ago
Role: Senior Data Engineer-Databricks, Python
Location: New Jersey(Hybrid)
Duration: 6+ Months
Job Description:
Must have: Databricks, Spark, Python, ADF, SQL
Responsibilities
Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
Optimize data pipelines for performance, scalability, and reliability
Ensure data quality and integrity throughout the data lifecycle
Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
Requirements
Qualifications
Bachelor's degree in Computer Science, Engineering, or related field, or equivalent work experience
5+ years of experience in data engineering with Databricks and Spark
Proficient in SQL and Python andPyspark
Experience with Azure Databricks Medallion Architecture with DLT, Iceberg
Financial/Corporate Banking context would be a plus
Experience with data integration and ETL tools, such as Azure Data Factory
Experience with Azure cloud platform and services
Experience with data warehouse and data lake concepts and architectures
Good to have experience with big data technologies, such as Kafka, Hadoop, Hive, etc
Strong analytical and problem-solving skills
Excellent communication and teamwork skills-
Databricks Engineer
3 weeks ago
Jersey City, United States CHUBB Full timeJob Description: We are searching for a seasoned Databricks Engineer to pioneer advanced Databricks solutions. The Databricks Engineer will serve as a central figure in our data engineering team, delivering highly sophisticated technological solutions using Databricks, Delta Live Tables, Snowflake, and Medallion Architecture. Key Responsibilities: ...
-
AWS Databricks Engineer
2 weeks ago
Jersey City, United States KTek Resourcing Full timeResponsibilities:Evaluate and understand the latest features and capabilities of the Databricks platform and assess their potential impact on our organization's data ecosystem.Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions using Databricks.Help to design, develop, and implement scalable...
-
AWS Databricks Engineer
2 weeks ago
Jersey City, United States KTek Resourcing Full timeResponsibilities:Evaluate and understand the latest features and capabilities of the Databricks platform and assess their potential impact on our organization's data ecosystem.Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions using Databricks.Help to design, develop, and implement scalable...
-
Senior Data Engineer with SAS, Databricks
2 weeks ago
Jersey City, United States Diverse Lynx Full timeSenior Data Engineer with SAS/Databricks/PySpark Locations Mt Laurel, NJ & Charlotte, NC (Hybrid Work: 3 Days Onsite Per Week) Required Experience: 10-12+ Years Primary Tech skills SAS, Databricks, Pyspark Secondary Tech skills Azure, Synapse Experience in building end to end architecture for Data Lakes, Data Warehouses and Data Marts Experience...
-
Data Engineer
2 weeks ago
Jersey City, United States ClifyX Full timeJob title : Data Engineer - Databricks Location : New Jersey (Remote) Duration : Contract / Fulltime Health Care Company $75/Hr (Contract) salary 120-135K/Annum (Fulltime) Job Description: • Experience with PySpark or Python • DataBricks experience is mandatory • Experience with big data tools Spark, Hadoop Kafka would be good to have • Hands on...
-
Databricks Architect
3 weeks ago
Jersey City, United States Inizio Partners Full timeRole & Responsibilities Overview:Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoringContinuously improve systems through...
-
Databricks Architect
5 days ago
Jersey City, United States Inizio Partners Corp Full timeJob DescriptionJob DescriptionRole & Responsibilities Overview:Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for...
-
Databricks Architect
4 weeks ago
Jersey City, United States Inizio Partners Corp Full timeJob DescriptionJob DescriptionRole & Responsibilities Overview:Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for...
-
Cloud Data Engineer
2 days ago
Jersey City, United States ShiftCode Analytics Full timeTitle: Cloud Data Engineer Location: Charlotte, NC This role is 100% remote But Local Candidate is highly Preferred Sector: Banking Duration: 8 month (C2H) (USC or GC candidates Only***) Job Summary: Project Details: Job Description: Be the subject matter expert for data, pipeline design, architecture, and other data engineering technologies for the...
-
Cloud Data Engineer
2 weeks ago
Jersey City, United States ShiftCode Analytics Full timeTitle: Cloud Data Engineer Location: Charlotte, NC This role is 100% remote But Local Candidate is highly Preferred Sector: Banking Duration: 8 month (C2H) (USC or GC candidates Only***) Job Summary: Project Details: Job Description: Be the subject matter expert for data, pipeline design, architecture, and other data engineering technologies for the...
-
Senior Data Engineer
2 weeks ago
Arizona City, United States Saxon Global Full timeUnitedHealth Group, Inc. (UHG) is looking for a Senior Data Engineer to work 100% remote (will work 9am-5pm either AZ time zone or EST time zone) to design, develop, test, and maintain pipelines ingesting critical real-time claims data sets into OnePaaS (Cloud). Project Initiative: There is need to ingest critical claims datasets (real time) into the cloud...
-
Principal Data Engineer
4 weeks ago
Jersey City, New Jersey, United States Tiger Analytics, LLC Full timeAs a Principal Data Engineer (Azure), you would have hands on experience working on Azure as cloud, Databricks and some exposure/experience on Data Modelling.using different Open Source, Big Data, and Cloud technologies on Microsoft Azure.Experience in implementing Data Lake with technologies like Azure Data Factory (ADF), PySpark, Databricks,...
-
Data Engineer
2 weeks ago
Jersey City, United States A.M. Best Full timeOverview The Data Engineer will build data pipelines / data orchestrator and data structures for existing operational data stores, Azure data lake & data warehouse. They will develop proof-of-concepts to evaluate new technologies and suggest improved and better ways to re-architect existing data loading processes and data structures. They will work...
-
Azure Databricks Architect
2 weeks ago
Kansas City, United States ClifyX Full timeRR-344784.1 zure Databricks Architect Kansas City,Missouri Day 1 onsite Burns & McDonnell Pay Rate $95/hr or $150 K customer was explicitly looking for a profile with design experience in Databricks Medallion architecture, and hands-on development as well. • Must have minimum of 3 Years of Data & Analytics Solution Experience as an Architect • Overall...
-
Data Engineer
2 weeks ago
Jersey City, United States Diverse Lynx Full timeJob Title: Data Engineer ( Python + AWS) Location : Jersey City , NJ (Onsite, Local s only) Job responsibilities : Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems Develops secure high-quality...
-
Director Of Data Engineering
1 week ago
Jersey City, United States Kavyos Consulting Full timeAs a leader of our data engineering team, you'll help define the foundation that is critical for analyzing everything from heavy, cross-channel analysis of media interaction logs to optimizing Paid Media to distilling purchase data down to understanding our customers better to evaluating our warehouse and sourcing data to improve our operational efficiency....
-
Director Of Data Engineering
7 days ago
Jersey City, United States Kavyos Consulting Full timeAs a leader of our data engineering team, you'll help define the foundation that is critical for analyzing everything from heavy, cross-channel analysis of media interaction logs to optimizing Paid Media to distilling purchase data down to understanding our customers better to evaluating our warehouse and sourcing data to improve our operational efficiency....
-
Databricks Administrator
4 weeks ago
Foster City, United States Royalty Staffing Full timeRoyalty Staffing is helping an on-demand, autonomous ride-hailing company hire an experienced Business Intelligence Administrator to manage and maintain company data stack tools.In this role, you will be responsible for various duties that will include user management, data modeling/ETL, building pipelines, system configuration, security and providing...
-
Databricks Administrator
4 weeks ago
Foster City, United States Royalty Staffing Full timeRoyalty Staffing is helping an on-demand, autonomous ride-hailing company hire an experienced Business Intelligence Administrator to manage and maintain company data stack tools.In this role, you will be responsible for various duties that will include user management, data modeling/ETL, building pipelines, system configuration, security and providing...
-
Databricks Administrator
20 hours ago
Foster City, United States Royalty Staffing Full timeRoyalty Staffing is helping an on-demand, autonomous ride-hailing company hire an experienced Business Intelligence Administrator to manage and maintain company data stack tools.In this role, you will be responsible for various duties that will include user management, data modeling/ETL, building pipelines, system configuration, security and providing...