Lead Databricks Engineer with Pyspark

5 days ago

Iselin, NJ, United States Zortech Solutions Full time
Role: Lead Databricks Engineer with Pyspark & Python Exp

Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office)

Duration: 6+ Months
  • uery/Redshift experience is good to have
  • 3+ years Python(numpy, pandas) coding experience
  • Experience with Snowflake utilities such as SnowSQL and SnowPipe - good to have
  • Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
  • Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills
  • Capable of discussing enterprise level services independent of technology stack
Job Description:

Inperson interview is must, Expenses will be paid for in-person if not local

(Financial background is must)

Major Responsibilities:
  • Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
  • Design, develop, and deploy Databricks jobs to process and analyze large volumes of data.
  • Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines.
  • Optimize Databricks jobs for performance and scalability to handle big data workloads.
  • Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
  • Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
  • Demonstrated proficiency with Data Analytics, Data Insights
  • Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process
  • Azure Synapse/Bigquery/Redshift is good to have.
  • Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.
Skills:
  • 5+ years' - Strong experience in Finance / Banking industry - Capital markets, investment banking - Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
  • 10+ years - Enterprise Data Management
  • 10+ years - SQL Server based development of large datasets
  • 5+ years with Data Warehouse Architecture, hands-on experience with Databricks platform. Extensive experience in PySpark coding.
  • Azure Synapse/BigqExperience with Cloud based data architectures, messaging, and analytics
  • Superior communication skills
  • Cloud certification(s)
  • Any experience with Regulatory Reporting is a Plus
Education:
  • Minimally a BA degree within an engineering and/or computer science discipline
  • Master's degree strongly preferred

  • Lead Databricks Engineer with Pyspark

    2 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Hello Professionals, We have urgent requirement for Lead Databricks Engineer with Pyspark & Python Exp Please have a look at the below job description, if interested please share your updated resume, email me, pavan@zortechsolutions.ca or share any References Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3...

  • Lead Databricks Engineer with Pyspark

    1 day ago

    Iselin, NJ, United States Zortech Solutions Full time

    Hello Professionals, We have urgent requirement for Lead Databricks Engineer with Pyspark & Python Exp Please have a look at the below job description, if interested please share your updated resume, email me, pavan@zortechsolutions.ca or share any References Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3...

  • Lead Databricks Engineer with Pyspark

    6 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Hello Professionals, We have urgent requirement for Lead Databricks Engineer with Pyspark & Python Exp Please have a look at the below job description, if interested please share your updated resume, email me, pavan@zortechsolutions.ca or share any References Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3...

  • Lead Databricks Engineer with Pyspark

    3 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Hello Professionals, We have urgent requirement for Lead Databricks Engineer with Pyspark & Python Exp Please have a look at the below job description, if interested please share your updated resume, email me, pavan@zortechsolutions.ca or share any References Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3...

  • Lead Databricks Engineer with Pyspark

    4 hours ago

    Iselin, NJ, United States Zortech Solutions Full time

    Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Job Description: Inperson interview is must, Expenses will be paid for in-person if not local (Financial background is must) Major Responsibilities: Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash...

  • Databricks Engineer with Pyspark

    2 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Role: Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Duration: 6+ Months Job Description: Job Description: This position is for a Cloud Data / Reporting engineer with a background in SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable...

  • Databricks Engineer with Pyspark

    1 day ago

    Iselin, NJ, United States Zortech Solutions Full time

    Role: Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Duration: 6+ Months Job Description: Job Description: This position is for a Cloud Data / Reporting engineer with a background in SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable...

  • Databricks Engineer with Pyspark

    6 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Role: Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Duration: 6+ Months Job Description: Job Description: This position is for a Cloud Data / Reporting engineer with a background in SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable...

  • Databricks Engineer with Pyspark

    3 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Role: Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Duration: 6+ Months Job Description: Job Description: This position is for a Cloud Data / Reporting engineer with a background in SQL and data warehousing for enterprise level systems. The position calls for someone that is comfortable...

  • Databricks Engineer with Pyspark

    2 days ago

    Iselin, NJ, United States Zortech Solutions Full time

    Hello Professionals, We have urgent requirement for Databricks Engineer with Pyspark & Python Exp Please have a look at the below job description, if interested please share your updated resume, email me, pavan@zortechsolutions.ca or share any References. Role: Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days'...