Lead Data Engineer

Overview:

At New York Blood Center Enterprises (NYBCe), one of the most comprehensive blood centers in the world, our focus is on cultivating excellence by merging cutting-edge innovation with diligent customer service, groundbreaking research, and comprehensive program and service development. Join us as we work towards meeting and exceeding the growing needs of our diverse communities, further our lifesaving strategic goals in a rapidly changing environment, and expand our impact on the local, national, and global communities we serve.

As a Lead Data Engineer, you will serve as a key architect in evolving our data infrastructure, under the technical direction of the Director of Data Engineering. Your primary responsibilities will include leading the design, implementation, and continuous enhancement of our data architectures, data pipelines, and interfaces to maximize scalability and efficiency. With advanced proficiency in SQL, Python, and cloud technologies (Azure, AWS, or Google Cloud), you will spearhead the development of data solutions that are synchronized with our strategic objectives. Your deep understanding of Big Data concepts, including Spark and Cloud ETL tools like Databricks, will be crucial in addressing complex data challenges. Leveraging Agile/SCRUM methodologies, you will guide innovative and prompt project completions. This hands-on leadership role involves mentoring senior data engineers and fostering a culture of excellence and proactive growth. As a leading figure in our team, you will collaborate extensively with data scientists, BI teams, software engineers, and business stakeholders to devise and execute effective data engineering strategies that drive significant business impact.

Candidates must be able to report into one of the following NYBCe locations:New York City, NY; Kansas City, Missouri; St. Paul, Minnesota; Providence , RI and Newark, DE.

Primary Responsibilities:

• Strategic Data Pipeline Design & Optimization: Lead the architectural design and optimization of robust, scalable data pipelines using SQL, Python, and cloud-based ETL tools such as Databricks. Oversee data flow and processing to support large-scale operations and ensure system-wide efficiency.
• Advanced Data Modeling: Direct the development and refinement of complex data models to accurately mirror critical business processes, ensuring seamless integration with our comprehensive data architecture, including Big Data frameworks like Spark.
• Data Architecture Leadership: Shape and enhance our data architecture strategy, making high-level decisions on data storage, consumption, integration, and management across cloud platforms (Azure, AWS, or Google Cloud).
• Agile/SCRUM Leadership: Orchestrate Agile/SCRUM frameworks to drive efficient project delivery. Lead sprints and stand-ups, implementing these methodologies to optimize development processes across teams.
• Senior-Level Collaboration: Lead collaboration efforts with data scientists, BI teams, and engineering groups to translate intricate data requirements into executable engineering strategies. Act as a primary technical liaison among stakeholders.
• Mentorship of Senior Engineers: Mentor senior data engineers and technical team leads, instilling best practices in SQL, Python, and cloud technologies, while nurturing a culture of excellence and continuous advancement.
• Quality & Governance Oversight: Establish and enforce the highest data quality standards and governance policies, ensuring reliability and regulatory compliance across all data operations.
• Performance Tuning Expertise: Continuously monitor and refine the performance of data infrastructure, identifying and resolving high-level bottlenecks or inefficiencies in cloud and Big Data environments.
• Innovation and Strategic Implementation: Stay at the forefront of emerging data engineering technologies and methodologies, leading the evaluation and strategic implementation of innovative tools and practices that enhance our capabilities.

Education:

• Bachelors Degree in Computer Science, Data Science, Information Technology or other quantitative disciplines such as Science, Statistics, Economics, or Mathematics.

Essential Experience:

• 10+ years of progressive experience in data engineering, with proven expertise in designing, implementing, and leading initiatives for optimizing databases and data pipelines.
• Extensive hands-on experience with SQL Server, Oracle, or other relational database management systems (RDBMS).
• Advanced proficiency in SQL and Python for sophisticated data manipulation and analytics.
• Demonstrated leadership in data modeling and architecture for both analytics and transactional systems within large-scale environments.
• At least 2 years of experience leading teams of data engineers: Proven ability to mentor and guide teams in the development and optimization of data systems, fostering a collaborative and innovative working environment.

Cloud and Big Data Experience:

• Expertise with at least one major cloud data platform (Azure, AWS, Google Cloud) with extensive application in data engineering projects.
• Deep knowledge of Big Data technologies such as Spark and Cloud ETL tools like Databricks, with a focus on scalability and real-time processing capabilities.

Methodology and Tools:

• Senior-level experience with Agile and SCRUM methodologies, leading successful project delivery in a dynamic development environment.
• Skilled in developing data models for integration and analysis that support complex business intelligence and data analytics initiatives.

Any combination of education, training, and experience equivalent to the requirements above that has supplied the necessary knowledge, skills, and experience to perform the essential functions of the job.

Required Knowledge, Skills & Abilities:

Knowledge

• Fluent Communication: Ability to articulate complex data concepts and project updates clearly to both technical and non-technical stakeholders.
• Strong Data Analysis Ability: Expertise in analyzing large datasets to derive insights and inform business decisions.
• Proficiency in SQL and Python: High level of skill in SQL and Python for data analysis, data manipulation and scripting.
• ETL/ELT Architecture: In-depth knowledge of developing and managing ETL and ELT architectures using various tools and frameworks.
• Cloud Experience: Experience with cloud platforms such as Azure, AWS, or Google Cloud, and their respective data services and tools.
• Big Data Concepts: Understanding of Big Data technologies and frameworks, including Spark and Cloud ETL tools such as Databricks.
• Agile and SCRUM Knowledge: Familiarity with Agile methodologies and SCRUM practices, capable of integrating these into project management and daily workflows.
• Quality Assurance and Data Governance: Knowledge of data quality standards and governance, ensuring data integrity and compliance across all processes.

Skills

• Collaboration: Ability to work effectively with cross-functional teams, including data scientists, BI analysts, and software engineers, to implement data solutions.
• Mentorship and Leadership: Skills in mentoring junior engineers and leading project teams to promote knowledge sharing and professional growth within the team.
• Innovation and Continuous Learning: Commitment to staying updated on the latest industry trends and technologies in data engineering and implementing them as relevant.

Abilities

• Ability to interact with customers one-on-one or in large groups
• Ability to work independently with remote supervision.
• Ability to build in receiving feedback as part of the development process, and seek consistent and constructive feedback.
• Ability to embrace accountability and ownership.

Peferred Qualifications:

• Education: Masters Degree in Computer Science, Data Science, Information Technology or other quantitative disciplines such as Science, Statistics, Economics, or Mathematics.
• Experience with the Microsoft Azure technology stack or similar technologies in competing platforms.
• Practical knowledge of data analytics and visualization tools to aid in data-driven decision making and reporting.

Certifications & Licenses:

• Professional certification in Agile and SCRUM methodologies (e.g., Certified ScrumMaster (CSM), SAFe Agilist).
• Certifications in Python and SQL programming (e.g., Microsoft Certified: Python Programming Specialist, Oracle SQL Certification).
• Certifications in cloud services relevant to the job (e.g., AWS Certified Solutions Architect, Google Professional Data Engineer, Microsoft Certified: Azure Data Engineer Associate).
• Big Data certifications (e.g., Cloudera Certified Professional (CCP): Data Engineer, Databricks Certified Professional Data Scientist).

Willing to attain certification, if not currently certified.

For applicants who will perform this position in New York City or Westchester County, the proposed annual salary is $150,000.00 p/yr. to $160,000.00 p/yr. For applicants who will perform this position outside of New York City or Westchester County, salary will reflect local market rates and be commensurate with the applicants skills, job-related knowledge, and experience.

Unless otherwise specified, all posted opportunities are located in the New York or Greater Tri-State office locations.