Senior Data Engineer

Senior Data Engineer

Job Locations

US-NY-New York

ID

2023-5206

Category

Information Technology

Work Location Type

Hybrid

Type

Regular Full-Time

Overview

At New York Blood Center Enterprises (NYBCe), one of the most comprehensive blood centers in the world, our focus is on cultivating excellence by merging cutting-edge innovation with diligent customer service, groundbreaking research, and comprehensive program and service development. Join us as we work towards meeting and exceeding the growing needs of our diverse communities, further our lifesaving strategic goals in a rapidly changing environment, and expand our impact on the local, national, and global communities we serve.

Responsibilities

As a Senior Data Engineer, you'll be a pivotal figure in defining and advancing our data infrastructure vision. Reporting to the Director of Data Engineering, your role will be crucial in designing, implementing, and refining databases, data pipelines, and data interfaces to ensure scalability and performance. Your proficiency in SQL, Python, and cloud environments (Azure, AWS, or Google Cloud) will empower you to develop solutions that are both robust and optimally aligned with our strategic goals. With a solid grasp of Big Data concepts, including Spark and Cloud ETL tools like Databricks, you will enhance our capabilities in handling complex data challenges. By adopting Agile/SCRUM methodologies, you'll drive innovative and timely project deliveries. You'll also mentor junior engineers, promoting a culture of excellence and continuous improvement. As a senior member of our team, you will work closely with data scientists, BI teams, software engineers, and other stakeholders to translate complex data requirements into practical and impactful engineering strategies.

Candidates must be able to report into one of the following NYBCe locations:New York City, NY; Kansas City, Missouri; St. Paul, Minnesota; Providence , RI and Newark, DE.

Responsibilities:

• Data Pipeline Design & Optimization: Design, implement, and optimize robust and scalable data pipelines using SQL, Python, and cloud-based ETL tools such as Databricks. Ensure efficient data flow and processing to support large-scale data handling.
• Data Modeling: Develop and refine data models to accurately represent business processes, ensuring they're scalable and fully integrate with our extensive data architecture, including Big Data frameworks like Spark.
• Data Architecture: Enhance our overarching data architecture strategy, assisting in decisions related to data storage, consumption, integration, and management within cloud environments (Azure, AWS, or Google Cloud).
• Agile/SCRUM: Lead and contribute within Agile/SCRUM frameworks to ensure timely and efficient project deliveries. Actively participate in sprints and stand-ups, applying these methodologies to streamline development.
• Collaboration: Partner with data scientists, BI teams, and other engineering teams to understand and translate complex data requirements into actionable engineering solutions.
• Mentorship: Guide and mentor junior data engineers, promoting best practices in SQL, Python, and cloud technologies, and fostering a culture of continuous learning and improvement.
• Quality & Governance: Uphold and champion data quality standards and governance policies, ensuring reliability and compliance in all data-related tasks.
• Performance Tuning: Monitor and enhance the performance of data infrastructure, proactively identifying and resolving bottlenecks or inefficiencies in cloud and Big Data environments.
• Innovation: Stay abreast of emerging data engineering and AI technologies and methodologies, recommending and implementing innovative tools or practices as appropriate.
• Documentation: Generate comprehensive documentation for data processes, pipelines, and architectures to ensure clarity and ease of maintenance for the team, including detailed descriptions of cloud and Big Data implementations.

Qualifications

Required Minimum Education & Experience:

Education:

Bachelor's Degree in Computer Science, Data Science, Information Technology or other quantitative disciplines such as Science, Statistics, Economics, or Mathematics.

Essential Experience:

• 7+ years of progressive experience in data engineering, with significant expertise in designing, implementing, and optimizing databases and data pipelines.
• Extensive hands-on experience with SQL Server, Oracle, or other relational database management systems (RDBMS).
• Proficiency in SQL and Python for advanced data manipulation and analytics.
• Demonstrated experience with data modeling and architecture for both analytics and transactional systems within large-scale environments.

Cloud and Big Data Experience:

• Proficient with at least one major cloud data platform (Azure, AWS, Google Cloud) with practical application in data engineering projects.
• Familiarity with Big Data technologies such as Spark and Cloud ETL tools like Databricks, focusing on scalability and real-time processing capabilities.

Methodology and Tools:

• Proven track record of using Agile and SCRUM methodologies to drive successful project delivery in a dynamic development environment.
• Experience in developing data models for integration and analysis that support business intelligence and data analytics initiatives.

Any combination of education, training, and experience that provides the required knowledge, skills, and abilities to perform the essential functions of the job.

Preferred Qualifications:

Education: Master's Degree in Computer Science, Data Science, Information Technology or other quantitative disciplines such as Science, Statistics, Economics, or Mathematics.

Experience with the Microsoft Azure technology stack or similar technologies in competing platforms.

Practical knowledge of data analytics and visualization tools to aid in data-driven decision making and reporting.

Certifications & Licenses:

Professional certification in Agile and SCRUM methodologies (e.g., Certified ScrumMaster (CSM), SAFe Agilist).

Certifications in Python and SQL programming (e.g., Microsoft Certified: Python Programming Specialist, Oracle SQL Certification).

Certifications in cloud services relevant to the job (e.g., AWS Certified Solutions Architect, Google Professional Data Engineer, Microsoft Certified: Azure Data Engineer Associate).

Big Data certifications (e.g., Cloudera Certified Professional (CCP): Data Engineer, Databricks Certified Professional Data Scientist).

Willing to attain certification, if not currently certified.

Required Knowledge, Skills & Abilities:

Knowledge

• Fluent Communication: Ability to articulate complex data concepts and project updates clearly to both technical and non-technical stakeholders.
• Strong Data Analysis Ability: Expertise in analyzing large datasets to derive insights and inform business decisions.
• Proficiency in SQL and Python: High level of skill in SQL and Python for data analysis, data manipulation and scripting.
• ETL/ELT Architecture: In-depth knowledge of developing and managing ETL and ELT architectures using various tools and frameworks.
• Cloud Experience: Experience with cloud platforms such as Azure, AWS, or Google Cloud, and their respective data services and tools.
• Big Data Concepts: Understanding of Big Data technologies and frameworks, including Spark and Cloud ETL tools such as Databricks.
• Agile and SCRUM Knowledge: Familiarity with Agile methodologies and SCRUM practices, capable of integrating these into project management and daily workflows.
• Quality Assurance and Data Governance: Knowledge of data quality standards and governance, ensuring data integrity and compliance across all processes.

Skills

• Collaboration: Ability to work effectively with cross-functional teams, including data scientists, BI analysts, and software engineers, to implement data solutions.
• Mentorship and Leadership: Skills in mentoring junior engineers and leading project teams to promote knowledge sharing and professional growth within the team.
• Innovation and Continuous Learning: Commitment to staying updated on the latest industry trends and technologies in data engineering and implementing them as relevant.

Abilities

• Ability to interact with customers one-on-one or in large groups
• Ability to work independently with remote supervision.
• Ability to build in receiving feedback as part of the development process, and seek consistent and constructive feedback.
• Ability to embrace accountability and ownership

For applicants who will perform this position in New York City or Westchester County, the proposed annual salary is $125,000 to $135,000 a year. For applicants who will perform this position outside of New York City or Westchester County, salary will reflect local market rates and be commensurate with the applicant's skills, job-related knowledge, and experience.

---