Data Engineer

Job Details
Job Location:    Albany or NYC Office - Albany or New York, NY
Position Type:    Full Time
Education Level:    4 Year Degree
Salary Range:    $ $ Salary
Travel Percentage:    Occasional
Job Shift:    M - F 9am - 5pm

Description

New York eHealth Collaborative – Data Engineer

New York eHealth Collaborative (NYeC) is a not-for-profit organization working in partnership with the New York State Department of Health to improve healthcare by collaboratively leading, connecting, and integrating health information exchange across the State.

Founded in 2006 by healthcare leaders, NYeC works to help New York State achieve the Triple Aim of improving the patient experience of care, delivering better health outcomes, and reducing costs. On behalf of the State, NYeC leads the Statewide Health Information Network for New York (SHIN-NY), a network connecting healthcare providers statewide, develops policies and standards that support the utilization of health technologies, and assists healthcare providers in adopting and effectively using electronic health records.

Position Summary:

NYeC is seeking a Data Engineer experience working with healthcare data in a NoSql environment. Candidate will have strong experience in big data, healthcare, data structures, complex queries, query performance, and ETL. The ideal candidate will also have exposure to web application programming/technologies on the perimeter of a NoSql database, such as taking in data from a web API and syncing data with external databases. This position will report to the Director of Product and can be operated out of the Albany, NY or Manhattan, NY office on a hybrid work schedule.

Primary Responsibilities:

Snowflake Query and Schema Creation

• Write queries for reporting, analysis, and data quality detection.
• Create new fields for ingesting new data or tracking data flow for reporting.
• Leverage existing data models and factor in ways that optimize sharing.
• Develop comprehensive schema designs that effectively address business needs and support data-driven decision-making.

Tune and Document Query Performance

• Minimize compute time of Snowflake queries, identify problem queries, and tune existing queries.
• Understand appropriate use cases for Snowflake queries and determine when to use high-speed registries.

Data Pipeline Development & Automation

• Build and maintain scalable ETL pipelines between systems.
• Build and maintain AWS Lambda APIs and integrations with DynamoDB and Snowflake.
• Develop new API integrations to support increasing data volume and complexity.

Data Quality & Fidelity Assurance

• Establish and maintain data fidelity standards for accurate, timely, and consistent data.
• Develop and maintain high data quality standards that comply with company guidelines, HITRUST requirements, and other regulatory requirements.
• Implement data verification, validation, and governance frameworks to ensure reliability and compliance.

Integration, Collaboration & Troubleshooting

• Act as a Subject Matter Expert (SME) on specific products, processes, or applications.
• Collaborate with stakeholders, other data teams, and software engineers to integrate systems and enable smooth data flow.
• Troubleshoot interoperability issues and document probable causes.

Process Improvement

• Serve as a thought leader to identify opportunities for modernization and process improvements.
• Coordinate and forecast work for self and others on owned solutions.

Other duties as assigned

Experience and Skills Required:

• Bachelor's degree in Engineering or Computer Science or equivalent experience.
• 5+ years of experience in cloud platforms such as AWS or Azure, with relevant industry certifications or demonstrated equivalent experience.
• Proven experience with high-volume data processing and designing solutions for reliability and performance at scale.
• Deep understanding of data architecture, data modeling, and ETL processes.
• Proficiency in SQL and Python; experience with Snowflake, Redshift, Databricks, and other cloud-based data platforms.
• Experience writing AWS Lambda, Airflow, AWS Glue; familiarity with serverless frameworks.
• Knowledge of NoSQL databases, especially Snowflake and DynamoDB Streams.
• Experience with APIs: Postman/API troubleshooting skills, API Gateway, AWS SDKs, OAuth 2.0/OpenID Connect.
• Knowledge of JavaScript, , or Java preferred.
• Experience with healthcare standards and technologies such as HL7, CDA, PIXV3, FHIR, HAPI FHIR, e-prescribing, and meaningful use.
• Understanding of HIPAA compliance and handling PHI in testing and development environments.
• Ability to write complex SQL for data analysis and troubleshooting anomalies.
• Demonstrated ability to seek out, analyze, and solve novel problems.
• Able to work independently and within a team environment.
• Strong organizational skills with exceptional attention to detail.

Expectations of Employees: 

• Employees work a hybrid in-office schedule (at either our Manhattan, NY or Albany, NY office and then remotely). Barring specific exemptions, team members are expected to work from the office on a regular schedule determined by the CCOO and on other days specified by their manager (no less than 1 day per week in the office). This schedule is subject to change. 
• NYeC supports work happening across New York State. From time to time our team members must visit other parts of the state. The most common requirement is for a New York City based team member to travel to Albany and vice versa.

We consider a wide range of factors when determining compensation, which may cause compensation to vary depending on your skills, experience, qualifications, and home office location (Manhattan, NY vs. Albany, NY). The annual base salary range for this role for an Albany based candidate is $80,000 -95,000. The annual base salary range for this role for a NYC based candidate is $100,000 – $120,000. The salary offer will not be based on a candidate's salary history at other jobs, and by law, NYeC will not seek information about salary history, and candidates should not share such information with NYeC. All compensation questions and comments should be directed to the HR Department representative during your application, interview, and hiring process.

For more information about NYeC and to apply for this position, visit our website at We accept online applications only.

Qualifications

---