Regulatory Reporting Data Engineer

Regulatory Reporting Data Engineer IT (Information Technology) Permanent contract New York, New York, United States Hybrid Salary from $100,000 to $125,000 Reference 25000LBH Start date 2025/12/01 Publication date 2025/10/10 Responsibilities We are seeking a New York based Data Engineer to join our Regulatory Technology and Control group.This is a key role for Bernstein that involves working with high-volume trading data to improve the accuracy, timeliness, and completeness of the firm’s regulatory reporting and trade recordkeeping. We are looking for a data expert who is passionate about technology and eager to learn more about financial services and our Equity broker dealer business.The Data Engineer will be responsible for receiving and transforming high-volume Equity trading data for onward reporting to the firm’s regulators and business users. Trading data is ingested to Microsoft Azure Synapse, transformed using Apache Spark, and written to Delta Lake tables on Azure. The successful candidate will help to migrate legacy SQL Server tables and T-SQL stored procedures to our future-state Microsoft Azure data platform.The Data Engineer will propose and implement improvements to the firm’s regulatory reporting implementation and will work with regulatory control and compliance to understand reporting requirements and deliver enhancements. The Data Engineer’s goal is to ensure that accurate, timely and complete data is supplied to regulators and business users in a format that they can use and understand.The Data Engineer will work closely with Equity trading and trade support to understand trading scenarios and validate that Consolidated Audit Trail (CAT) Reporting accurately reflects the firm’s Equity trading activity. The successful candidate will promote innovative data analysis techniques and will gain knowledge in cash & electronic trading scenarios, trading technology, data analysis, design, and regulatory compliance.The key job responsibilities include, but are not limited to:Enhance and maintain the Microsoft Azure Synapse and Spark data pipelines that generate Bernstein’s CAT reporting and populate Bernstein’s trading records.Migrate legacy SQL Server tables and T-SQL stored procedures to Microsoft Azure Synapse, PySpark, and Delta LakeUnderstand Equity trading scenarios and how these are represented in the data models of Bernstein trading systems.Understand how CAT Reporting applies to Bernstein’s Equity trading businessIdentify and troubleshoot data issues by analyzing high-volume datasets in innovative ways, by using data analytics tools such as Python, Azure Synapse, and PowerBIDesign and implement transactional and reference data reconciliations to maintain data quality.Support critical regulatory reporting processes in productionWork closely with business users to create dashboards and reports that improve risk management and enable better business decisionsHybrid role with a minimum of 3 days in the officeThe role provides in-depth learning on how to work with high-volume financial services data and would offer a motivated technologist the opportunity to champion the use of new technology within a large financial services organization. The Data Engineer will be trained in Equity trading scenarios and CAT Reporting, areas that are in high demand within the financial services industry.The role provides the following opportunities for learning and professional growth:Gain in-depth financial services knowledge, including Equity trading scenariosLearn the data model and business processes of a major brokerage firmGain experience in PySpark data analysis and Azure cloud-based technologies such as Azure Synapse, Spark, Delta Lake, and PowerBIBecome an expert in CAT Reporting Profile required We are seeking a motivated data engineer who is passionate about improving business outcomes through use of the latest data technologies. The candidate should have experience working with high-volume data using Microsoft Azure, Spark, Delta Lake, or equivalent cloud-based technologies.Code quality is important at Bernstein, and the candidate should demonstrate the ability to write efficient, readable, and maintainable code in a language such as Python, C++, Java, C, or C#. Candidates should have good knowledge of common development practices such as version control, automated testing, continuous integration, and automated deployment.Experience working with Microsoft SQL Server and T-SQL is preferred, and existing knowledge of Equity Cash trading scenarios and CAT Reporting would be beneficial.The successful candidate will collaborate closely with other developers and data analysts and should be comfortable communicating with colleagues. We are looking for a candidate who is excited by technology and can share their enthusiasm with other team members to improve the adoption of new tools.Basic Qualifications:2+ years of experience transforming and analyzing high-volume datasets using Azure Synapse, Spark, or equivalent cloud-based technologies.2+ years of development experience with one or more modern programming languages such as Python, C++, Java, C, or C#.Experience using GitHub or an equivalent developer platform for version control, continuous integration, and automated deployment.Strong inter-personal communication skills and the ability to collaborate effectively with colleagues.Preferred Qualifications:Bachelors or Masters degree in a STEM (Science, Technology, Engineering or Math) field.2+ years of experience working with Microsoft SQL Server, T-SQL, SSIS, or equivalent technologies.Experience building applications using Azure Cloud and / or Kubernetes.Knowledge of Equity Cash trading scenarios and / or experience generating CAT Reports.