Data Engineer
3 weeks ago
About Our Client:
Our healthcare client is committed to leveraging innovative data engineering solutions, particularly the Medallion architecture, to drive operational excellence and deliver transformative insights in patient care.
Must-Haves:
- Minimum of 3 years' experience with Databricks, PySpark, SQL, Spark clusters, and Jupyter Notebooks.
- Expertise in building data lakes using the Medallion architecture and working with delta tables in the delta file format.
- Familiarity with CI/CD pipelines and Agile methodologies, ensuring efficient and collaborative development practices.
- Strong understanding of ETL processes, data modeling, and data warehousing principles.
- Experience with data visualization tools like Power BI is a plus.
- Knowledge of cybersecurity data, particularly vulnerability scan data, is preferred.
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
Responsibilities:
- Develop, test, and maintain large-scale data processing systems using Databricks, PySpark, SQL, Spark clusters, delta tables, and the innovative Medallion architecture.
- Collaborate closely with cybersecurity analysts to gather data requirements and deliver effective solutions aligned with Medallion architecture principles.
- Ensure data quality and implement robust data governance standards, leveraging the scalability and efficiency offered by the Medallion architecture.
- Design and implement ETL processes, including data cleansing, transformation, and integration, optimizing performance within the delta file format framework.
- Build and manage data lakes based on Medallion architecture principles, ensuring scalability, reliability, and adherence to best practices.
- Monitor and optimize data pipelines, integrating CI/CD practices to streamline development and deployment processes.
- Collaborate with cross-functional team members to implement data analytics projects, utilizing Jupyter Notebooks and other tools to harness the power of the Medallion architecture.
- Embrace Agile methodologies throughout the development lifecycle to promote iterative and collaborative development practices, enhancing the effectiveness of Medallion-based solutions.
#J-18808-Ljbffr
-
Windows Platform Engineer
6 days ago
Boston, United States Data Affect Full timeAbout Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...
-
Windows Platform Engineer
6 days ago
Boston, United States Data Affect Full timeAbout Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...
-
Windows Platform Engineer
6 days ago
Boston, United States Data Affect Full timeAbout Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...
-
Data Engineer
2 weeks ago
Boston, United States Jellyfish Full timeAs a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You’ll also build...
-
Data Engineer
3 weeks ago
Boston, Massachusetts, United States Jellyfish Full timeAs a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You'll also build...
-
Data Engineer
4 weeks ago
Boston, Massachusetts, United States Northeastern University Full timeThe College of Social Sciences and Humanities is a leader in the Experiential Liberal Arts. The College is strongly committed to fostering excellence through diversity and enthusiastically welcomes applications from members of groups underrepresented in higher education administration.The College's schools, departments, and programs—with disciplines...
-
Data Engineer
3 weeks ago
Boston, United States Northeastern University Full timeThe College of Social Sciences and Humanities is a leader in the Experiential Liberal Arts. The College is strongly committed to fostering excellence through diversity and enthusiastically welcomes applications from members of groups underrepresented in higher education administration. The College’s schools, departments, and programs—with...
-
VP Data Engineering
4 weeks ago
Boston, United States Socium Recruitment Full timeVP Data Engineering Onsite – Boston – Unable to support Visa’s & C2C Hey there! Are you all about Big Data and building awesome teams? This Start-up is shaking things up in their world with some seriously cool data solutions. They aren’t afraid to do things differently, and they’re on a mission to make a real impact with our innovative approach.As...
-
Data Engineer
2 weeks ago
Boston, Massachusetts, United States firsthand Full timefirsthand is changing the way individuals living with Serious Mental Illness (SMI) get care. We are focusing on delivering real outcomes for a cohort that has historically been underserved, stigmatized, and deprioritized. By building a service focused on whole-person care, firsthand aims to find, establish a trusting relationship with, and support quality...
-
Data Engineer
2 weeks ago
Boston, United States OneSource Regulatory Full timeCompany Introduction OneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space. OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team! Job Description OneSource Regulatory is trying to identify a full-time contractor with at least 4+...
-
Head of Data Engineering
2 days ago
Boston, United States WiseHire Recruitment Full timeWiseHire are seeking a Head of Data Engineering to lead the Analytics, Data Science, and Data Engineering teams within a growing organization. You'll shape the data strategy, architecture, data platform, and collaborate with various stakeholders to address their data needs, all while aligning with the wider business goals. This hands-on role involves...
-
Head of Data Engineering
1 day ago
Boston, United States WiseHire Recruitment Full timeWiseHire are seeking a Head of Data Engineering to lead the Analytics, Data Science, and Data Engineering teams within a growing organization. You'll shape the data strategy, architecture, data platform, and collaborate with various stakeholders to address their data needs, all while aligning with the wider business goals. This hands-on role involves...
-
Data Engineer
3 weeks ago
Boston, Massachusetts, United States Selby Jennings Full timeWe have a current opportunity for a Data Engineer - Markit EDM on a permanent basis. The position will be based in Boston. For further information about this position please apply.
-
Senior Data Engineer
2 days ago
Boston, United States Storm2 Full time⚡ Senior Data Engineer🎯 Series B FinTech Hyper Growth🌎 Boston, MA Hybrid💸 Salary: $140-180k base + equityCompany:Join a dynamic and rapidly growing fintech company backed by $130M in funding. The company is revolutionizing the financial industry with cutting-edge technology and innovative solutions. They have been recognized by Forbes Fintech 50...
-
Senior Data Engineer
2 days ago
Boston, United States Storm2 Full time⚡ Senior Data Engineer🎯 Series B FinTech Hyper Growth🌎 Boston, MA Hybrid💸 Salary: $140-180k base + equityCompany:Join a dynamic and rapidly growing fintech company backed by $130M in funding. The company is revolutionizing the financial industry with cutting-edge technology and innovative solutions. They have been recognized by Forbes Fintech 50...
-
Data Engineer
3 weeks ago
Boston, United States OneSource Regulatory Full timeJob DescriptionJob DescriptionSalary: Company IntroductionOneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space. OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team! Job DescriptionOneSource Regulatory is trying to identify a...
-
Data Engineering Consultant
2 weeks ago
Boston, United States Ask Staffing Full timeJob Title: Data Engineering ConsultantDuration: 5 monthsWork Location: Open to 100% Remote Job Description: • As a Data Engineering Consultant specializing in Data Warehouse Migration, you will be at the forefront of our efforts to transition our data ecosystem to a more efficient and modern platform. • Your expertise in data engineering, AWS cloud...
-
Software Engineer
3 days ago
Boston, United States Vendelux LLC. Full timeVendelux is a Series A SaaS company and we provide the system of record for event marketing. Our software platform provides proprietary insights that helps high-growth companies find the highest ROI events, conferences and trade shows to attend and sponsor. We have built an AI-powered platform that customers describe as an event marketer’s dream. Vendelux...
-
Quantitative Data Engineer
4 weeks ago
Boston, United States Clickjobs.com Full timeQuantitative Data Engineer Quantitative Investment Platforms and Developer Operations Team Full-time Boston, MA, Springfield, MA or New York, NY The Opportunity As a Data Engineer you will work in a fast paced, innovative, and collaborative environment on exciting technology directives. A Data Engineer embodies a unique combination of both technical and...
-
Data Engineer
2 weeks ago
Boston, United States Tive Inc Full timeAt Tive, we imagine a fully-transparent world in which everything and everyone is connected. We innovate beyond what the world thought possible—so what people hold near and dear arrives on time and in full. We believe (and live!) four core values: • Create transparency first—everything else comes second: We believe that trust is critical, and we...