Databricks Architect

2 months ago


Trenton, United States Zortech Solutions Full time

Role: Decision Analytics/Databricks Architect Location: Dublin, Ohio, New Jersey-Travel once in a month Duration: Fulltime Job Description: Experience Level: 7 to 12 Must Have: Strong in Data engineering Strong developer profile Strong Coding skills Hand-on experience on:- Azure / AWS / Databricks (Good to have) / Analytics skills. Experience- 7-12 years. Role & Responsibilities Overview: · Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.) · Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoring · Continuously improve systems through performance enhancements and cost reductions in compute and storage · Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs · Lead Data Engineering Projects to manage and implement data-driven communication systems · Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations · Integrate data across different systems and platforms · Strong verbal and written communication skills to manage client discussions Candidate Profile: · 8+ years experience in developing and implementing ETL pipelines from various data sources using Databricks on cloud · Based out of US · Some experience in insurance domain/ data is must · Programming Languages – SQL, Python · Technologies - IaaS (AWS or Azure or GCP), Databricks platform, Delta Lake storage, Spark (PySpark, Spark SQL). o Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure Devops · Project Management using Agile, Scrum · B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering or other science field), Information Systems, Information Processing or engineering.

#J-18808-Ljbffr