Senior Data Engineer in Azure using Databricks for Scala/Python

2 weeks ago

Raleigh, United States UBS Full time
Your role

Do you like building complex, secure platforms with a touch of a button? Are passionate about developing automated infrastructure as code that is successfully rolled out across a global implementation? Do you have what it takes to build robust solution that aide data engineers in delivering their data pipelines?
UBS , the Group Compliance & Regulatory Governance (GCRG) Technology team is looking for a hands-on Data Engineer on Azure leveraging Databricks for Scala/Python to:
engineer reliable data pipelines for sourcing, processing, distributing, and storing data in different ways, using Databricks and Airflow
craft complex transformation pipelines on multiple datasets producing valuable insights that inform business decisions, making use of our internal data platforms and educate others about Best Practices for analytics big data
develop, train, and apply data engineering techniques to automate manual processes, and solve challenging business problems and ensure the quality, security, reliability, and compliance of our solutions by applying our digital principles and implementing both functional and non-functional requirements
build observability into our solutions, monitor production health, help to resolve incidents, and remediate the root cause of risks and issues
leverage Airflow to build complex branching Data Driven pipelines and leverage Databricks to build the spark layer of data pipelines
leverage Python and Scala for low level complex data operations, codify best practices, methodology and share knowledge with other engineers in UBS

Your team

You will be working as part of the Group Compliance Regulatory Governance Technology stream that focuses on Data Analytics and Reporting. Our crew is using the latest data platforms to further the group's data strategy to realize the true potential of data as an asset through utilizing data lakes, data virtualization, for use with advanced analytics, AI/ML. The crew also ensure the data is managed with strategic sourcing and data quality tooling. Your team will be responsible for building the central GCRG data lake, developing data pipelines to strategically source data from master and authoritative source, creating data virtualization layer, building connectivity for advanced analytics and elastic search capabilities with the aid of cloud computing.

Diversity helps us grow, together. That's why we are committed to fostering and advancing diversity, equity, and inclusion. It strengthens our business and brings value to our clients.

Your expertise
Bachelor's or master's degree in computer science or any similar engineering is highly desired
ideally 5+ years of total IT experience in SWD or engineering and ideally 3+ years of hand-on experience designing and building scalable data pipelines for large datasets on cloud data platforms
ideally 3+ years of hand-on experience in distributed processing using Databricks, Apache Python/Spark, Kafka & leveraging Airflow scheduler/executor framework
ideally 2+ years of hand-on experience programming experience in Scala (must have), Python & Java (preferred)
experience with monitoring solutions such as Spark Cluster Logs, Azure Logs, AppInsights, Graphana to optimize pipelines and knowledge in Azure capable languages, Python, Scala or Java
proficiency at working with large and complex code base management systems like: Github/Gitlab, Gitflow as a project commiter at both command-line and IDEs levels using: tools like: IntelliJ/AzureStudio
experience working with Agile development methodologies and delivering within Azure DevOps, automated testing on tools used to support CI and release management
expertise in optimized dataset structures in Parquet and Delta Lake formats, with ability to design and implement complex transformations between datasets
expertise in optimized Airflow DAGS and branching logic for Tasks to implement complex pipelines and outcomes and expertise in both traditional SQL and NO-SQL authorship

About us

UBS is the world's largest and the only truly global wealth manager. We operate through four business divisions: Global Wealth Management, Personal & Corporate Banking, Asset Management and the Investment Bank. Our global reach and the breadth of our expertise set us apart from our competitors..

We have a presence in all major financial centers in more than 50 countries.

Join us

At UBS, we embrace flexible ways of working when the role permits. We offer different working arrangements like part-time, job-sharing and hybrid (office and home) working. Our purpose-led culture and global infrastructure help us connect, collaborate, and work together in agile ways to meet all our business needs.

From gaining new experiences in different roles to acquiring fresh knowledge and skills, we know that great work is never done alone. We know that it's our people, with their unique backgrounds, skills, experience levels and interests, who drive our ongoing success. Together we're more than ourselves. Ready to be part of #teamUBS and make an impact?

Disclaimer / Policy Statements

UBS is an Equal Opportunity Employer. We respect and seek to empower each individual and support the diverse cultures, perspectives, skills and experiences within our workforce.

  • Azure and Databricks Cloud Data Engineer

    2 months ago

    Raleigh, North Carolina, United States Infosys Limited Digital Full time

    Job detailsCountryUSAState / Region / ProvinceNorth Carolina, TexasWork LocationRaleigh, NC, Richardson, TXDomainDeliveryInterest GroupInfosys LimitedSkillsTechnology|Analytics - Packages|Python - Big Data, Technology|Cloud Integration|Azure Data Factory (ADF)CompanyITL USARequisition ID120750BRJob description Infosys is seeking an Azure and Databricks Cloud...

  • Senior Data Migration Engineer

    2 days ago

    Raleigh, North Carolina, United States Golden Technology Full time

    About the RoleWe are seeking a highly skilled Senior Data Migration Engineer to join our team at Golden Technology. As a key member of our data engineering team, you will be responsible for designing, implementing, and migrating data systems from on-premises to the cloud, specifically using Snowflake and Azure.Key ResponsibilitiesPlan and execute a smooth...

  • Azure Data Engineer with Medallion Exp

    3 days ago

    Raleigh, United States Magicforce Full time

    Job Title: Azure Data Engineer with Medallion Architecture Location: Remote Duration: Long Term Must have 13 years of experience Job Description: Sr. Azure Data Engineer Data Bricks ADF Medallion Architecture 15 years of experience in implementing large data and analytics platforms. Deep experience architecting and building data platforms on...

  • Azure Data Engineer

    3 months ago

    Raleigh, United States Focused HR Solutions Full time

    This position is 100% on-site in Raleigh, NC.  In-Person Interview REQUIRED on 1st round. Our direct client has an opening for a Azure Data Engineer position # 730992. This position is for 4+ months, with option of extension, and will be worked 100% on-site in Raleigh, NC. If you are interested, please submit the following:YOUR CURRENT RESUMEYOUR HOURLY...

  • Senior Data Engineer

    2 weeks ago

    Raleigh, United States Golden Technology Full time

    In this role, you will help the client move their data systems from on-premises to the cloud, specifically using Snowflake and Azure. You will work closely with the client to plan and carry out a smooth transition, making sure their data is securely and efficiently moved to the cloud. Your skills in data modeling, advanced SQL, and cloud tools like...

  • Senior Data Engineer

    3 weeks ago

    Raleigh, United States Golden Technology Full time

    In this role, you will help the client move their data systems from on-premises to the cloud, specifically using Snowflake and Azure. You will work closely with the client to plan and carry out a smooth transition, making sure their data is securely and efficiently moved to the cloud. Your skills in data modeling, advanced SQL, and cloud tools like...

  • Senior Data Engineer

    3 weeks ago

    Raleigh, United States Golden Technology Full time

    In this role, you will help the client move their data systems from on-premises to the cloud, specifically using Snowflake and Azure. You will work closely with the client to plan and carry out a smooth transition, making sure their data is securely and efficiently moved to the cloud. Your skills in data modeling, advanced SQL, and cloud tools like...

  • Senior Data Engineer

    3 weeks ago

    Raleigh, United States Golden Technology Full time

    Our client is looking for a Data Engineer to join their team! In this role, you will help the client move their data systems from on-premises to the cloud, specifically using Snowflake and Azure. You will work closely with the client to plan and carry out a smooth transition, making sure their data is securely and efficiently moved to the cloud. Your skills...

  • Senior Data Engineer

    3 weeks ago

    Raleigh, United States Golden Technology Full time

    Our client is looking for a Data Engineer to join their team! In this role, you will help the client move their data systems from on-premises to the cloud, specifically using Snowflake and Azure. You will work closely with the client to plan and carry out a smooth transition, making sure their data is securely and efficiently moved to the cloud. Your skills...

  • Azure Data Architect with Medallion Expertise

    2 days ago

    Raleigh, North Carolina, United States Magicforce Full time

    Job Title: Azure Data Engineer with Medallion ArchitectureLocation: RemoteDuration: Long TermRequirements:We are seeking a highly experienced Azure Data Engineer with expertise in Medallion Architecture to join our team at Magicforce. The ideal candidate will have a strong background in designing and implementing large-scale data platforms on Azure.Key...

  • Python Data Engineer

    3 months ago

    Raleigh, United States Saxon Global Full time

    Client: Optum Job Title: Python Data Engineer Duration: 6+ month contract Location: Raleigh, NC is the preferred location, open to Remote PROJECT: This team supports data analytics for HEDIS. They have a Python engine which reports HEDIS standards for each year. Code runs on big data and cloud DAY TO DAY: Supporting data analytics for HEDIS Python...

  • Lead Java Microservices and Python Engineer

    2 weeks ago

    Raleigh, North Carolina, United States UBS Full time

    Position OverviewWe are seeking a highly skilled professional to join our team as a Senior Developer specializing in Java Microservices and Python. In this role, you will have the opportunity to shape and enhance our digital offerings, playing a pivotal role in our transformation into a more agile organization.Your ResponsibilitiesAs a key contributor, you...

  • Azure Data Engineer

    2 weeks ago

    Raleigh, North Carolina, United States Focused HR Solutions Full time

    About Focused HR Solutions Focused HR Solutions is a leading provider of talent acquisition and workforce solutions, connecting top-tier professionals with exciting opportunities across diverse industries. Azure Data Engineer - Cloud Migration Specialist We are currently seeking a highly skilled Azure Data Engineer to join our client's team. This role...

  • Lead Java Microservices and Python Engineer

    2 weeks ago

    Raleigh, North Carolina, United States UBS Full time

    Position OverviewAs a pivotal member of our team, you will be responsible for architecting and developing innovative digital solutions that enhance our operational efficiency and client engagement.Your ResponsibilitiesIn this role, you will:Design and implement robust software components for our Credit Risk monitoring platform.Utilize your expertise to guide...

  • Lead Java Microservices and Python Engineer

    2 weeks ago

    Raleigh, North Carolina, United States UBS Full time

    Job Reference: 296714BRLocation: New York, RaleighJob Type: Full TimeYour Role: Are you passionate about crafting innovative digital solutions? At UBS, we are redefining our operational approach to enhance agility and responsiveness. We aim to transform our interactions with colleagues, clients, and partners, ultimately delivering exceptional value.Being...

  • Lead Java Microservices and Python Engineer

    1 week ago

    Raleigh, North Carolina, United States UBS Full time

    Job Reference: 296714BRLocation: New York, RaleighJob Type: Full TimeYour Role: Are you passionate about crafting engaging digital solutions? Do you aspire to be a pivotal contributor in our journey towards an agile organization? At UBS, we are redefining our operational methodologies, enhancing connections among colleagues, clients, and partners, and...

  • Strategic Data Architect for Databricks Solutions

    1 week ago

    Raleigh, North Carolina, United States Lovelytics, LLC Full time

    Job Title: Databricks Certified Solutions ArchitectAbout the Role:Lovelytics, LLC is seeking a highly skilled and experienced Databricks certified Solutions Architect to play a critical role in delivering and selling strategic Databricks and data engineering client engagements.Key Responsibilities:Own the end-to-end presales solutioning activities and...

  • Senior Java Microservices and Python Developer

    2 weeks ago

    Raleigh, United States UBS Full time

    Your role Do you want to design and build attractive digital products and services? Do you want to play a key role in transforming our firm into an agile organization? At UBS, we re-imagine the way we work, the way we connect with each other - our colleagues, clients and partners - and the way we deliver value. Being agile will make us more responsive, more...

  • Lead Java Microservices and Python Engineer

    1 week ago

    Raleigh, North Carolina, United States UBS Full time

    Job Reference: 296714BRLocation: New York, RaleighJob Type: Full TimeYour Role: Are you passionate about designing and developing innovative digital solutions? At UBS, we are committed to transforming our organization into a more agile entity. We are rethinking how we collaborate with our colleagues, clients, and partners to deliver exceptional...

  • Senior Java Microservices and Python Developer

    4 days ago

    Raleigh, North Carolina, United States UBS Full time

    About the RoleWe are seeking a highly skilled Java Microservices and Python Developer to join our team at UBS. As a key member of our Credit Review Monitoring and Contingent Market Risk team, you will play a critical role in designing and building attractive digital products and services that meet the needs of our clients and employees.Key...