Data Engineer

4 weeks ago

Montpelier, Vermont, United States Public Partnerships Full time
Job Summary

As a Data Engineer at Public Partnerships, you will design, build, and manage Azure Data Factory pipelines for data ingestion, handling both batch and near real-time data ingestion using Azure Service Bus. You will integrate structured, semi-structured, and unstructured data sources, utilizing formats such as Parquet, Avro, JSON, etc., and manage loading into the Bronze layer. Additionally, you will build and manage data engineering pipelines using Databricks (PySpark, SparkSQL), implement Spark-based data transformation jobs from Bronze to Silver and Gold layers, and establish and manage Data Quality Framework using Databricks and Data Factory. You will also set up data security using RBAC and ACLs to manage permissions on sensitive datasets, implement automation frameworks for pipeline execution using Databricks Workflows, and monitor pipelines using Azure Monitor, set up alerts, and ensure SLAs are met. Furthermore, you will optimize storage in ADLS Gen2 and use Azure Key Vault to secure credentials, and optimize costs using Azure Cost Management.

Key Responsibilities:
Design, build, and manage Azure Data Factory pipelines for data ingestion
Handle both batch and near real-time data ingestion using Azure Service Bus
Integrate structured, semi-structured, and unstructured data sources
Utilize formats such as Parquet, Avro, JSON, etc., and manage loading into the Bronze layer
Build and manage data engineering pipelines using Databricks (PySpark, SparkSQL)
Implement Spark-based data transformation jobs from Bronze to Silver and Gold layers
Establish and manage Data Quality Framework using Databricks and Data Factory
Set up data security using RBAC and ACLs to manage permissions on sensitive datasets
Implement automation frameworks for pipeline execution using Databricks Workflows
Monitor pipelines using Azure Monitor, set up alerts, and ensure SLAs are met
Optimize storage in ADLS Gen2 and use Azure Key Vault to secure credentials
Optimize costs using Azure Cost Management

Technical Skills:
Azure Data Factory for ETL/ELT operations
Databricks (PySpark, SparkSQL) for data transformations
Azure Data Lake Storage (ADLS Gen2)
SQL Server, PostgreSQL, Cosmos DB, CRM, and ERP systems
Data Governance, RBAC, and ACLs for managing permissions
CI/CD tools: Azure DevOps, GitHub Actions, Terraform
Azure Monitoring, Alerting, and Logging tools

Equal Opportunity Employer:
Public Partnerships is an Equal Opportunity Employer dedicated to celebrating diversity and intentionally creating a culture of inclusion. We believe that we work best when our employees feel empowered and accepted, and that starts by honoring each of our unique life experiences. We do not discriminate on the basis of race, color, religion or belief, national, social, or ethnic origin, sex, gender identity and/or expression, age, physical, mental, or sensory disability, sexual orientation, marital, civil union, or domestic partnership status, past or present military service, citizenship status, family medical history or genetic information, family or parental status, or any other status protected under federal, state, or local law. We will not tolerate discrimination or harassment based on any of these characteristics.

  • Data Engineer

    3 weeks ago

    Montpelier, Vermont, United States Bellavance Trucking Inc Full time

    Job SummaryData Engineer - Centralized Data SolutionsWe are seeking a highly skilled Data Engineer to join our team at Bellavance Trucking Inc. As a Data Engineer, you will be responsible for designing, implementing, and maintaining centralized data storage solutions. Your expertise in data pipelines, data visualization, and business intelligence will enable...

  • Data Engineer, Product Analytics

    4 weeks ago

    Montpelier, Vermont, United States META Full time

    Job Summary:We are seeking a highly skilled Data Engineer, Product Analytics to join our team at Meta. As a key member of our data engineering team, you will be responsible for designing and implementing large-scale data architectures, collaborating with cross-functional teams to understand data needs, and representing key data insights in a meaningful...

  • Data Governance Manager

    4 weeks ago

    Montpelier, Vermont, United States Ford Motor Company Full time

    Job SummaryAs the Data Governance Manager at Ford Motor Company, you will lead the development and delivery of Privacy, Compliance, and Consent capabilities for our Enterprise Data Platform (EDP). This role requires a strong understanding of data platforms, product management, and data minimization and privacy concepts to ensure the platform meets the needs...

  • Senior Data Analyst

    4 weeks ago

    Montpelier, Vermont, United States National Life Group Full time

    About the RoleWe are seeking a highly skilled Senior Data Solutions Analyst to join our Analytics Center of Excellence at National Life Group. As a key member of our team, you will play a critical role in enabling our data transformation and driving business insights.Key ResponsibilitiesSupport the facilitation of enterprise reporting and analytics by...

  • Data Governance Analyst

    4 weeks ago

    Montpelier, Vermont, United States National Life Group Full time

    Data Governance Internship OpportunityAt National Life Group, we are seeking a highly motivated and detail-oriented Data Governance Intern to join our team. As a Data Governance Intern, you will play a critical role in supporting the implementation of our data governance framework and facilitating enterprise reporting and analytics.Key...

  • Data Solution Architect

    4 weeks ago

    Montpelier, Vermont, United States National Life Group Full time

    Job SummaryThe Data Solution Architect role is a critical position at National Life Group, responsible for architecting the full data supply chain solution. This includes platform selection, technical architecture design, data model design for business solutions, deployment approaches, and implementation methods.This role will provide data architecture...

  • Data Scientist

    3 weeks ago

    Montpelier, Vermont, United States Norstella Full time

    About the RoleWe are seeking a highly skilled Data Scientist to join our team at Norstella. As a Data Scientist, you will play a key role in revolutionizing pharma intelligence and driving informed decision making.Key ResponsibilitiesCollaborate with product partners and other scientists to identify new opportunities to apply AI/ML to our content and...

  • Data Sales Support Expert

    3 weeks ago

    Montpelier, Vermont, United States Acxiom Full time

    Job SummaryWe are seeking an accomplished data sales support expert to join our team. The ideal candidate will exhibit a high level of autonomy, proficiency, and leadership, mentoring less experienced team members while continually expanding their own expertise. As a pivotal member of our sales support team, you will play a crucial role in enhancing our...

  • Solution and Data Architect

    4 weeks ago

    Montpelier, Vermont, United States Marriott Full time

    Job SummaryMarriott International is seeking a highly skilled Technical Architect to join our Global Technology - Enterprise Architecture team. As a key member of our team, you will be responsible for defining, advancing, and executing the technology future of our Loyalty platform.Key ResponsibilitiesLead and define Loyalty projects architecture solutions to...

  • Electrical Engineer Internship Opportunity

    4 weeks ago

    Montpelier, Vermont, United States Wieland Full time

    OverviewWieland - Creating Value for GenerationsWieland is a global leader in copper and copper alloy manufacturing, processing, and distribution. We provide a large product and service portfolio including power, automotive, ammunition, coinage, construction materials, refrigeration, electronics, and many other industries. To us, metal is more than...

  • Site Reliability Engineer

    3 weeks ago

    Montpelier, Vermont, United States Oracle Full time

    Job SummarySolve complex problems related to infrastructure cloud services and build automation to prevent problem recurrence. Design, write, and deploy software to improve the availability, scalability, and efficiency of Oracle products and services. Design and develop designs, architectures, standards, and methods for large-scale distributed systems....

  • Senior Oracle APEX Engineer

    4 weeks ago

    Montpelier, Vermont, United States Oracle Full time

    Job DescriptionAs a member of the Infrastructure Delivery Engineering (IDE) team at Oracle, you will play a key role in the automation and acceleration of infrastructure delivery. We are seeking a highly skilled and experienced APEX engineer to lead the design, delivery, and ongoing support of Oracle APEX Applications.ResponsibilitiesDesign and develop...

  • Principal Technical Support Engineer

    3 weeks ago

    Montpelier, Vermont, United States Oracle Full time

    Job DescriptionThis position is for a Principal Technical Support Engineer to join an established Oracle Database Exadata & Cloud support team. The ideal candidate will be fluent in Japanese language (verbal and written) as well as English.The team's main responsibility is to troubleshoot and resolve highly complex technical issues on Oracle Cloud and...

  • Senior Cloud Software Development Engineer

    4 weeks ago

    Montpelier, Vermont, United States Oracle Full time

    Job Title: Senior Cloud Software Development EngineerAbout the Role:We are seeking a highly skilled Senior Cloud Software Development Engineer to join our team at Oracle. As a member of our software engineering division, you will assist in defining and developing software for tasks associated with the developing, debugging or designing of software...

  • Principal Technical Support Engineer

    3 weeks ago

    Montpelier, Vermont, United States Oracle Full time

    Job DescriptionThis position is for a highly skilled Principal Technical Support Engineer to join our established Oracle Database Exadata & Cloud support team.The team's main responsibility is to solve and resolve highly complex technical issues on Oracle Cloud and Exadata On-Prem platforms, spanning across database footprint including Exadata architecture,...

  • Senior Director of Product Management

    4 weeks ago

    Montpelier, Vermont, United States EDB Full time

    About EDBEDB is a leading provider of data and AI platforms that empower organizations to harness the full power of Postgres for transactional, analytical, and AI workloads across any cloud, anywhere.Job SummaryWe are seeking a highly experienced Senior Director of Product Management to lead the strategic direction, development, and deployment of our new...

  • AI/ML Scientist Manager

    4 weeks ago

    Montpelier, Vermont, United States General Motors Full time

    Job DescriptionThe RoleWe are seeking a highly skilled and experienced Technical Lead to join our AI Center. The ideal candidate is a Data Scientist with a proven track record of deploying real-world AI applications at scale in large organizations.This role requires an individual who is deeply passionate about the operationalization of AI models within both...

  • Senior Software Development Engineer

    3 weeks ago

    East Montpelier, Vermont, United States Trellix Full time

    About TrellixTrellix is a global company redefining the future of cybersecurity and soulful work. The company's comprehensive, open and native cybersecurity platform helps organizations confronted by today's most advanced threats gain confidence in the protection and resilience of their operations.Role OverviewThe Endpoint Sustaining Engineering team focuses...

  • Senior Software Architect

    4 weeks ago

    Montpelier, Vermont, United States Marriott Full time

    Job SummaryMarriott International is seeking a highly skilled Senior Software Engineer to join our team. As a key member of our technology team, you will be responsible for designing, developing, and implementing technology models for our entire Marriott Associate App ecosystem and stack.You will work closely with our Product Squads and receive direction...

  • Database Systems Specialist

    3 weeks ago

    Montpelier, Vermont, United States Humana Full time

    Job SummaryThe Database Administrator is a key member of our team, responsible for the management and maintenance of both production and non-production databases. This role contributes to the development and implementation of standards and practices for physical data storage, maintenance, access, and security administration. The successful candidate will...