Data Engineer

2 days ago

Saint Paul, Minnesota, United States Beacon Hill Staffing Group Full time
Job Summary:

The Data Engineer is responsible for designing, developing, and maintaining data pipelines and infrastructure to support data-driven decision-making within the organization.

This role requires strong proficiency in Python, SQL, cloud technologies, and Kubernetes to ensure the efficient processing, storage, and retrieval of data.


Key Responsibilities:

Data Pipeline Development:
Design, build, and maintain scalable and reliable data pipelines using Python and SQL.
Develop ETL (Extract, Transform, Load) processes to integrate data from various sources into data warehouses and databases.
Ensure data quality and consistency across different data sources and systems.

Cloud Data Infrastructure:
Implement and manage data storage and processing solutions on cloud platforms (e.g., AWS, Azure, Google Cloud).
Utilize cloud data services such as BigQuery, Snowflake, Redshift, or similar to store and analyze large datasets.
Configure and manage cloud resources for optimal performance and cost-efficiency.

Containerization and Orchestration:
Deploy and manage data applications and services using Docker and Kubernetes.
Develop and maintain Kubernetes manifests, Helm charts, and CI/CD pipelines to automate data workflows.
Monitor and troubleshoot containerized applications to ensure high availability and reliability.

Performance Optimization:
Optimize data processing pipelines for performance and scalability.
Tune SQL queries and data storage configurations to handle large volumes of data efficiently.
Implement monitoring and logging solutions to track data pipeline performance and identify issues.

Collaboration and Communication:
Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide support.
Collaborate with DevOps and infrastructure teams to integrate data solutions with existing systems.
Document data processes, workflows, and configurations for transparency and knowledge sharing.

Qualifications:

Experience:

Experience as a Data Engineer or in a similar role with a strong focus on Python, SQL, and cloud technologies.


Technical Skills:
Proficiency in Python for data engineering tasks, including scripting and automation.
Advanced SQL skills for querying and manipulating data.
Hands-on experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and related data services.
Experience with containerization and orchestration technologies such as Docker and Kubernetes.

Beacon Hill is an Equal Opportunity Employer that values the strength diversity brings to the workplace. Individuals with Disabilities and Protected Veterans are encouraged to apply.

If you would like to complete our voluntary self-identification form, please click here or copy and paste the following link into an open window in your browser: Completion of this form is voluntary and will not affect your opportunity for employment, or the terms or conditions of your employment.

This form will be used for reporting purposes only and will be kept separate from all other records.

Company Profile:

Beacon Hill Technologies, a premier National Information Technology Staffing Group, provides world-class technology talent across all industries utilizing a complete suite of staffing services.

Beacon Hill Technologies' dedicated team of recruiting and staffing experts consistently delivers quality IT professionals to solve our customers' technical and business needs.

Beacon Hill Technologies covers a broad spectrum of IT positions, including Project Management and Business Analysis, Programming/Development, Database, Infrastructure, Quality Assurance, Production/Support, and ERP roles.

Learn more about Beacon Hill and our specialty divisions, Beacon Hill Associates, Beacon Hill Financial, Beacon Hill HR, Beacon Hill Legal, Beacon Hill Life Sciences, and Beacon Hill Technologies by visiting .

We look forward to working with you. Beacon Hill. Employing the FutureTM

  • Data Engineer

    3 weeks ago

    Saint Paul, Minnesota, United States IP Corporation Full time

    {"h1": "Data Engineer", "h2": "Job Description", "p": "At IP Corporation, we are seeking a talented Data Engineer to join our team and help us build robust data solutions that power our business insights. As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines that support our data-driven initiatives. You will work...

  • Data Engineer

    4 days ago

    Saint Paul, Minnesota, United States State of Minnesota Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our team at the State of Minnesota. As a Data Engineer, you will be responsible for designing, building, and maintaining the data infrastructure for our organization.Key Responsibilities:Design and implement data pipelines using AWS servicesDevelop and maintain data analytics tools using Python...

  • Data Engineer

    6 days ago

    Saint Paul, Minnesota, United States Beacon Hill Staffing Group Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Beacon Hill Technologies. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to support data-driven decision-making within the organization.Key Responsibilities:Data Pipeline Development:Design,...

  • Data Engineer

    4 weeks ago

    Saint Paul, Minnesota, United States IP Corporation Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at IP Corporation. As a Data Engineer, you will play a pivotal role in evolving our data architecture to support data-intensive initiatives and technologies.Key Responsibilities:Design, build, and maintain data pipelines that support our data-driven initiatives.Work...

  • Data Engineer

    3 weeks ago

    Saint Paul, Minnesota, United States ABOUT HEALTHCARE INC Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at ABOUT HEALTHCARE INC. As a Data Engineer, you will play a critical role in designing, building, and maintaining our data infrastructure, ensuring the accuracy and integrity of our data.Key ResponsibilitiesDesign and implement scalable data models, tables, views, and ETL pipelines...

  • Data Engineer

    6 days ago

    Saint Paul, Minnesota, United States IP Corporation Full time

    About UsIP Corporation is a leading manufacturer and distributor of composites, leveraging data to drive strategic decisions and enhance operations.Job DescriptionWe are seeking a talented Data Engineer to design, build, and maintain data pipelines that support our data-driven initiatives. As a key member of our team, you will work closely with technical and...

  • Data Engineer

    11 hours ago

    Saint Paul, Minnesota, United States ABOUT HEALTHCARE INC Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at ABOUT HEALTHCARE INC. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data models and ETL pipelines.Key Responsibilities:Design and build data pipelines from various data sources to a target data warehouse using batch data load...

  • Data Engineer

    2 days ago

    Saint Paul, Minnesota, United States ABOUT HEALTHCARE, Inc. Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at ABOUT HEALTHCARE, Inc. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data models, tables, views, and ETL pipelines.Key Responsibilities:Design and build data pipelines from various data sources to a target data warehouse using batch...

  • Data Solutions Engineer

    2 weeks ago

    Saint Paul, Minnesota, United States Highmark Health Full time

    Job SummaryThis role is responsible for designing and implementing data solutions for the enterprise, working closely with business, analytic, and IT stakeholders to develop and maintain these solutions. The incumbent will assist with the development and maintenance of data ingestion pipelines, transformations, and delivery programs/logic for people and...

  • Data Engineer I

    5 days ago

    Saint Paul, Minnesota, United States 020 Travelers Indemnity Co Full time

    About UsAt Travelers Indemnity Co, we're committed to delivering exceptional products and services to our customers. Our team is dedicated to fostering a culture of innovation, collaboration, and inclusivity.Job SummaryWe're seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you'll play a key role in designing, building, and...

  • Data Engineer I

    23 hours ago

    Saint Paul, Minnesota, United States 020 Travelers Indemnity Co Full time

    About UsAt 020 Travelers Indemnity Co, we're committed to delivering exceptional products and services to our customers. Our Data Engineering team plays a critical role in achieving this goal by constructing pipelines that provide easy access to data across the enterprise.Job SummaryWe're seeking a skilled Data Engineer to join our team. As a Data Engineer,...

  • Data Engineer I

    4 days ago

    Saint Paul, Minnesota, United States Travelers Insurance Full time

    Transforming Data for a Better TomorrowAt Travelers Insurance, we're committed to delivering exceptional products and services to our customers. As a Data Engineer, you'll play a crucial role in building and operationalizing complex data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning, and...

  • Data Engineer II

    1 week ago

    Saint Paul, Minnesota, United States The Travelers Companies, Inc Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at The Travelers Companies, Inc. As a Data Engineer, you will play a key role in designing, building, and deploying data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning, and business intelligence/insights.Key...

  • Data Engineer II

    2 days ago

    Saint Paul, Minnesota, United States Travelers Insurance Full time

    Transforming Data for a Better FutureAt Travelers Insurance, we're committed to delivering exceptional products and services to our customers. As a Data Engineer, you'll play a critical role in constructing pipelines that contextualize and provide easy access to data across the enterprise.As a key member of our Data Engineering team, you'll leverage your...

  • Data Engineer II

    3 weeks ago

    Saint Paul, Minnesota, United States 020 Travelers Indemnity Co Full time

    Job Title: Data Engineer IIAt Travelers Indemnity Co, we are seeking a highly skilled Data Engineer II to join our team. As a Data Engineer II, you will play a key role in designing, building, and deploying data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning, and business...

  • Data Engineer I

    4 weeks ago

    Saint Paul, Minnesota, United States 020 Travelers Indemnity Co Full time

    Job Title: Data Engineer IAt Travelers Indemnity Co, we are seeking a highly skilled Data Engineer I to join our team. As a Data Engineer, you will play a key role in growing and transforming our analytics landscape.Job SummaryWe are looking for a talented Data Engineer to design, build, and deploy data solutions using cloud-based compute and warehouse...

  • Data Engineer II

    4 days ago

    Saint Paul, Minnesota, United States Beacon Hill Staffing Group Full time

    Job Summary:Beacon Hill Technologies is seeking a highly skilled Data Engineer II to lead the design, development, and implementation of ETL pipelines using Apache NiFi and Java Spring Boot microservices. The ideal candidate will possess deep expertise in Java programming, Spring Boot, and AWS big data technologies, including Redshift and Athena.Key...

  • Data Engineer

    4 weeks ago

    Saint Paul, Minnesota, United States General Motors Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Marketing Applied Sciences team at General Motors. As a Data Engineer, you will play a critical role in building analytical data sets in support of Advanced Analytics projects.Key ResponsibilitiesCollaborate with internal and external stakeholders to deliver on the portfolio.Drive the...

  • Data Engineering Director

    4 weeks ago

    Saint Paul, Minnesota, United States Travelers Full time

    Travelers Data Engineering TeamWe are seeking a highly skilled Data Engineering Director to join our team at Travelers. As a key member of our organization, you will play a critical role in accelerating the growth and transformation of our analytics landscape.Key ResponsibilitiesLead the development of analytic data products, including pilots and proof of...

  • Domo Data Engineer

    4 days ago

    Saint Paul, Minnesota, United States Beacon Hill Staffing Group Full time

    Job Summary:We are seeking a highly skilled Domo Data Engineer / Administrator to join our team at Beacon Hill Staffing Group. As a key member of our team, you will be responsible for creating and maintaining Domo dashboards, apps, and reports. You will work closely with internal teams to define data requirements and structures for Domo BI applications. Your...