Data Engineer

7 days ago


Blue Bell, United States PMA Companies Full time
Data Engineer #4399 Multiple Locations Apply for this job now Job Description:

We are seeking a Data Engineer to join our IT department. This role will focus on leveraging Databricks, Azure Data Factory and related tool sets to enhance our systems integration capabilities within the commercial insurance domain. The ideal candidate will have strong technical problem-solving skills, a deep understanding of Databricks, and a strong understanding of data.

Data Pipeline Development:

    • Design, develop, and maintain scalable data pipelines using Databricks.
    • Ensure data quality, integrity, and consistency across all data pipelines.

Data Integration:

    • Integrate data from multiple sources, including Azure Data Lake, SQL databases, and any APIs.
    • Leveraging Databricks to integrate with different Large and Small Language Models.
    • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.

ETL Processes:

    • Develop and maintain ETL (Extract, Transform, Load) processes using Azure Databricks.
    • Implement Data Quality and Audit, Balance and Control to the data workflows.
    • Maintain and optimize data workflow performance.

Collaboration and Communication:

    • Work closely with cross-functional teams to understand business requirements and translate them into technical solutions.
    • Provide technical support and guidance to team members and stakeholders.

Performance Optimization:

    • Optimize data processing performance on Azure Databricks.
    • Implement best practices for data storage, processing, and retrieval.
  • Security and Compliance:
    • Ensure data security and compliance with industry standards and regulations.
    • Implement data governance policies and procedures.
  • Documentation and Reporting:
    • Document data engineering processes, workflows, and best practices.
    • Generate reports and dashboards to provide insights into data pipeline performance and data quality.
  • Continuous Improvement:
    • Stay updated with the latest industry trends and technologies in data engineering and Azure Databricks.
    • Identify opportunities for process improvements and implement best practices
    • Demonstrate commitment to Company's Code of Business Conduct and Ethics, and apply knowledge of compliance policies and procedures, standards and laws applicable to job responsibilities in the performance of work.

Requirements:
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Proven experience as a Data Engineer with a focus on Azure Databricks.
  • Strong knowledge of Azure Databricks and its ecosystem.
  • Experience with data pipeline and workflow management tools.
  • Proficiency in programming languages such as PySpark, Python, Scala, or SQL.
  • Familiarity with big data technologies (e.g., Spark).
  • Experience with Azure services such as Azure Data Lake, Azure SQL Database, and analytics capabilities available in Azure.
  • Knowledge of data warehousing concepts and best practices.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills.
  • Ability to work in a fast-paced, dynamic environment.
Locations Blue Bell Philadelphia

  • Blue Bell, Pennsylvania, United States IPS-Integrated Project Services, LLC Full time

    **About the Role**We are seeking an experienced Data Center Commissioning Project Manager to join our team at IPS-Integrated Project Services, LLC.As a key member of our commissioning team, you will be responsible for overseeing and managing the commissioning process for data center facilities. Your primary goal will be to ensure that all systems and...


  • Blue Bell, Pennsylvania, United States IPS-Integrated Project Services, LLC Full time

    At IPS, we're committed to delivering exceptional services in the life sciences industry. We're seeking an experienced Electrical Commissioning Expert to join our Mission Critical Data Center Team.About the RoleThis is a safety-sensitive position that requires frequent travel (up to 75% of the time) to project locations. As an Electrical Commissioning...


  • Blue Bell, United States IPS-Integrated Project Services, LLC Full time

    Reference #: 8970 Job Description At IPS, we apply our knowledge, skills, and passion to make a difference in the lives of people, solving complex challenges related to life sciences. We are looking for a dedicated Electrical Commissioning Engineer to join our talented Mission Critical Data Center Team. The Data Center Electrical Commissioning Engineer...

  • Solutions Engineer

    1 month ago


    Blue Bell, United States Kane Partners LLC Full time

    Job DescriptionJob DescriptionLocation: 100% On-Site with occasional travel to client sitesPosition Overview:We are seeking a skilled Solutions Engineer with a strong command of T-SQL and proven analytical skills to join our team in supporting customized deployments of our FOIA application across various government agencies. This role is critical in...


  • Blue Ash, Ohio, United States LeadStack Inc. Full time

    Data Catalog EngineerLeadStack Inc., a certified minority-owned staffing services provider, seeks a highly skilled Data Catalog Engineer to contribute to the development and maintenance of its data catalog platform.Key Responsibilities:Configure and customize the data catalog platform to accurately represent the organization's data landscape.Implement and...

  • Solutions Engineer

    1 month ago


    Blue Bell, United States Kane Partners LLC Full time

    Location: 100% On-Site with occasional travel to client sitesPosition Overview:We are seeking a skilled Solutions Engineer with a strong command of T-SQL and proven analytical skills to join our team in supporting customized deployments of our FOIA application across various government agencies. This role is critical in tailoring our document management...


  • Blue Bell, United States CV Library Full time

    Overview:Do you want to build an impactful career to change the world for the better? Geosyntec has an exciting opportunity for an Early-Career Environmental Engineer or Scientist as part of our contaminated site assessment and remediation practice in our Blue Bell, PA, Pennington or Lyndhurst, NJ office.  The position will have the opportunity to apply...


  • Blue Bell, United States Innova Full time

    A client of Innova Solutions is immediately hiring for a Lab Automation Engineer (Onsite).   Position type: Full-time Contract, potential contract to hire. Duration: 12+ months Location: West Point PA 19486 USA Position: Lab Automation Engineer (Onsite) Comments/Special Instructions: 100% on-site position Programming focused.  File transformations,...


  • Blue Ash, Ohio, United States Eliassen Group Full time

    Eliassen Group is a leading strategic consulting company for human-powered solutions, with a strong presence in the US and Europe.We are seeking an experienced Data Scientist/ML Engineer to join our innovative team.Job OverviewThis role will involve designing, building, and deploying scalable cloud-based solution architectures using public cloud platforms...


  • Blue Bell, United States JSAT AUTOMATION INC Full time

    Job DescriptionJob DescriptionAbout JSat AutomationAt JSat Automation we believe that by providing our customers with the resources they need to overcome obstacles in their quest for scientific discovery and change, we can more broadly improve the daily lives of everyone in our world. It is our mission to provide our clients with a strategic, efficient...


  • Blue Bell, United States IPS-Integrated Project Services, LLC Full time

    Job Description At IPS, you will apply your knowledge, skills, and passion to make a difference in the lives of people, solving complex challenges related to pharmaceutical, biotech, and animal health research and manufacturing facilities. We are looking for a talented Sustainability Engineer / Consultant II to join our team at our office located in...


  • Blue Bell, Pennsylvania, United States Verinext Full time

    We're Verinext, a tech company that's not just keeping pace with the future, but actively shaping it. At our core, we believe work should be as enjoyable as it is rewarding.As a Data Project Manager, you'll join an environment that thrives on innovation and collaboration. Our team-oriented culture is more than just a buzzword – it's the cornerstone of our...


  • Blue Bell, United States IPS-Integrated Project Services, LLC Full time

    Reference #: 8622 Job Description At IPS, we apply our knowledge, skills, and passion to make a difference in the lives of people, solving complex challenges related to life sciences. We are looking for a dedicated Mechanical Engineer III / Senior Mechanical (HVAC) to join our talented design team based in our Blue Bell, PA headquarters. In this role, you...


  • Blue Ash, United States CoStrategix Full time

    SummaryWe are seeking an experienced Data Quality Assurance Professional to join our dynamic team. The ideal candidate will be responsible for ensuring the accuracy, consistency, and reliability of our organization's data assets. As a Data Quality Assurance Professional, you will play a critical role in developing and implementing data quality standards,...

  • Data Engineer

    3 weeks ago


    Blue Ash, United States Tata Consultancy Services Full time

    Job Description6+ plus years of experience in solutioning data pipeline for large enterprise data Warehouse applications using Azure Data Factory, Data bricks, Data Ingestion, Data Transformation /Computation, Orchestration, Reporting & Data AnalyticsKnowledge of Azure Services such as ADF, Databricks, Event Hub, Delta Lake, Data LakeStrong command on...

  • Data Engineer

    2 weeks ago


    Blue Ash, United States Tata Consultancy Services Full time

    Job Description6+ plus years of experience in solutioning data pipeline for large enterprise data Warehouse applications using Azure Data Factory, Data bricks, Data Ingestion, Data Transformation /Computation, Orchestration, Reporting & Data AnalyticsKnowledge of Azure Services such as ADF, Databricks, Event Hub, Delta Lake, Data LakeStrong command on...


  • Blue Ash, United States ACL Digital Full time

    Job Title: Data Engineer Location: Blue Ash, OH, 45242, USAJob Duration: 6 Month(s) - contract to hireThey will be working a hybrid schedule, in the office 2-3 day/week.Soft Skills Needed - Self-starter, good communicator, willing to take ownership Project person will be supporting - SeamlessTeam details i.e. size, dynamics, locations - Majority of this team...


  • Blue Ash, United States ACL Digital Full time

    Job Title: Data Engineer Location: Blue Ash, OH, 45242, USAJob Duration: 6 Month(s) – contract to hireThey will be working a hybrid schedule, in the office 2-3 day/week.Soft Skills Needed - Self-starter, good communicator, willing to take ownership Project person will be supporting - SeamlessTeam details i.e. size, dynamics, locations - Majority of this...


  • Blue Bell, United States IPS-Integrated Project Services, LLC Full time

    Reference #: 8972 Job Description The Commissioning Project Manager for our Mission Critical Facilities groups is responsible for overseeing and managing the commissioning process for data center facilities, ensuring that all systems and infrastructure meet industry standards and operational requirements. This role involves project management, coordination,...

  • AutoCAD Civil Designer

    2 months ago


    Blue Bell, United States The Judge Group Full time

    Our firm is currently seeking an experienced Civil Designer. Candidate must be proficient in AutoCAD Land Desktop and Civil 3D. Candidate must possess a full range of drafting skills, including drafting of survey field data and notes, conceptual design development, sketch plans, and full land development approval and permitting sets. Proven ability to design...