Data Engineer

3 weeks ago


Boston, United States Gardner Resources Consulting Full time

About Our Client: Our healthcare client is committed to leveraging innovative data engineering solutions, particularly the Medallion architecture, to drive operational excellence and deliver transformative insights in patient care.

Must-Haves: - Minimum of 3 years' experience with Databricks, PySpark, SQL, Spark clusters, and Jupyter Notebooks. - Expertise in building data lakes using the Medallion architecture and working with delta tables in the delta file format. - Familiarity with CI/CD pipelines and Agile methodologies, ensuring efficient and collaborative development practices. - Strong understanding of ETL processes, data modeling, and data warehousing principles. - Experience with data visualization tools like Power BI is a plus. - Knowledge of cybersecurity data, particularly vulnerability scan data, is preferred. - Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.

Responsibilities: - Develop, test, and maintain large-scale data processing systems using Databricks, PySpark, SQL, Spark clusters, delta tables, and the innovative Medallion architecture. - Collaborate closely with cybersecurity analysts to gather data requirements and deliver effective solutions aligned with Medallion architecture principles. - Ensure data quality and implement robust data governance standards, leveraging the scalability and efficiency offered by the Medallion architecture. - Design and implement ETL processes, including data cleansing, transformation, and integration, optimizing performance within the delta file format framework. - Build and manage data lakes based on Medallion architecture principles, ensuring scalability, reliability, and adherence to best practices. - Monitor and optimize data pipelines, integrating CI/CD practices to streamline development and deployment processes. - Collaborate with cross-functional team members to implement data analytics projects, utilizing Jupyter Notebooks and other tools to harness the power of the Medallion architecture. - Embrace Agile methodologies throughout the development lifecycle to promote iterative and collaborative development practices, enhancing the effectiveness of Medallion-based solutions.

#J-18808-Ljbffr



  • Boston, United States Data Affect Full time

    About Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...


  • Boston, United States Data Affect Full time

    About Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...


  • Boston, United States Data Affect Full time

    About Data Affect: We are a boutique data/service management firm specializing in the delivery of data governance, enterprise data strategy, solutions architecture, data warehousing, data integrations, data security & privacy management, IT service management, business analysis (and) agile project management services to diverse clients across multiple...

  • Data Engineer

    2 weeks ago


    Boston, United States Jellyfish Full time

    As a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You’ll also build...

  • Data Engineer

    3 weeks ago


    Boston, Massachusetts, United States Jellyfish Full time

    As a member of Jellyfish Research, you will work closely with data scientists to understand and facilitate data needs for high-impact research, analysis and reporting. You will optimize, expand or redesign our current data platforms, including storage and pipeline architecture, to ingest, combine, and aggregate data from multiple sources. You'll also build...

  • Data Engineer

    4 weeks ago


    Boston, Massachusetts, United States Northeastern University Full time

    The College of Social Sciences and Humanities is a leader in the Experiential Liberal Arts. The College is strongly committed to fostering excellence through diversity and enthusiastically welcomes applications from members of groups underrepresented in higher education administration.The College's schools, departments, and programs—with disciplines...

  • Data Engineer

    3 weeks ago


    Boston, United States Northeastern University Full time

    The College of Social Sciences and Humanities is a leader in the Experiential Liberal Arts. The College is strongly committed to fostering excellence through diversity and enthusiastically welcomes applications from members of groups underrepresented in higher education administration. The College’s schools, departments, and programs—with...

  • VP Data Engineering

    4 weeks ago


    Boston, United States Socium Recruitment Full time

    VP Data Engineering Onsite – Boston – Unable to support Visa’s & C2C Hey there! Are you all about Big Data and building awesome teams? This Start-up is shaking things up in their world with some seriously cool data solutions. They aren’t afraid to do things differently, and they’re on a mission to make a real impact with our innovative approach.As...

  • Data Engineer

    2 weeks ago


    Boston, Massachusetts, United States firsthand Full time

    firsthand is changing the way individuals living with Serious Mental Illness (SMI) get care. We are focusing on delivering real outcomes for a cohort that has historically been underserved, stigmatized, and deprioritized. By building a service focused on whole-person care, firsthand aims to find, establish a trusting relationship with, and support quality...

  • Data Engineer

    2 weeks ago


    Boston, United States OneSource Regulatory Full time

    Company Introduction OneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space. OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team! Job Description OneSource Regulatory is trying to identify a full-time contractor with at least 4+...


  • Boston, United States WiseHire Recruitment Full time

    WiseHire are seeking a Head of Data Engineering to lead the Analytics, Data Science, and Data Engineering teams within a growing organization. You'll shape the data strategy, architecture, data platform, and collaborate with various stakeholders to address their data needs, all while aligning with the wider business goals. This hands-on role involves...


  • Boston, United States WiseHire Recruitment Full time

    WiseHire are seeking a Head of Data Engineering to lead the Analytics, Data Science, and Data Engineering teams within a growing organization. You'll shape the data strategy, architecture, data platform, and collaborate with various stakeholders to address their data needs, all while aligning with the wider business goals. This hands-on role involves...

  • Data Engineer

    3 weeks ago


    Boston, Massachusetts, United States Selby Jennings Full time

    We have a current opportunity for a Data Engineer - Markit EDM on a permanent basis. The position will be based in Boston. For further information about this position please apply.


  • Boston, United States Storm2 Full time

    ⚡ Senior Data Engineer🎯 Series B FinTech Hyper Growth🌎 Boston, MA Hybrid💸 Salary: $140-180k base + equityCompany:Join a dynamic and rapidly growing fintech company backed by $130M in funding. The company is revolutionizing the financial industry with cutting-edge technology and innovative solutions. They have been recognized by Forbes Fintech 50...


  • Boston, United States Storm2 Full time

    ⚡ Senior Data Engineer🎯 Series B FinTech Hyper Growth🌎 Boston, MA Hybrid💸 Salary: $140-180k base + equityCompany:Join a dynamic and rapidly growing fintech company backed by $130M in funding. The company is revolutionizing the financial industry with cutting-edge technology and innovative solutions. They have been recognized by Forbes Fintech 50...

  • Data Engineer

    3 weeks ago


    Boston, United States OneSource Regulatory Full time

    Job DescriptionJob DescriptionSalary: Company IntroductionOneSource Regulatory Technology hosts a number of innovative solutions to enhance job performance in the Pharmaceutical space.  OSR Technology is looking for an experienced and dedicated data engineer to join our product solutions team!  Job DescriptionOneSource Regulatory is trying to identify a...


  • Boston, United States Ask Staffing Full time

    Job Title: Data Engineering ConsultantDuration: 5 monthsWork Location: Open to 100% Remote Job Description: • As a Data Engineering Consultant specializing in Data Warehouse Migration, you will be at the forefront of our efforts to transition our data ecosystem to a more efficient and modern platform. • Your expertise in data engineering, AWS cloud...

  • Software Engineer

    3 days ago


    Boston, United States Vendelux LLC. Full time

    Vendelux is a Series A SaaS company and we provide the system of record for event marketing. Our software platform provides proprietary insights that helps high-growth companies find the highest ROI events, conferences and trade shows to attend and sponsor. We have built an AI-powered platform that customers describe as an event marketer’s dream. Vendelux...


  • Boston, United States Clickjobs.com Full time

    Quantitative Data Engineer Quantitative Investment Platforms and Developer Operations Team Full-time Boston, MA, Springfield, MA or New York, NY The Opportunity As a Data Engineer you will work in a fast paced, innovative, and collaborative environment on exciting technology directives. A Data Engineer embodies a unique combination of both technical and...

  • Data Engineer

    2 weeks ago


    Boston, United States Tive Inc Full time

    At Tive, we imagine a fully-transparent world in which everything and everyone is connected. We innovate beyond what the world thought possible—so what people hold near and dear arrives on time and in full. We believe (and live!) four core values: • Create transparency first—everything else comes second: We believe that trust is critical, and we...