Data Pipeline Engineer Mid
2 weeks ago
As a mid-level Data Pipeline Engineer, you will play a crucial role in designing, building, and maintaining robust data pipelines for our customers. Your expertise will drive the efficient collection, storage, processing, and transformation of large-scale data sets. Here are the key responsibilities and qualifications for this role:
Pipeline Development:
- Design, develop, and optimize end-to-end data pipelines for efficient data extraction, transformation, and loading (ETL) processes.
- Collaborate with cross-functional teams to understand data requirements and translate them into scalable pipeline solutions.
- Implement best practices for data integration, ensuring high performance, reliability, and scalability.
Data Transformation and Quality:
- Transform raw data into usable formats, ensuring data quality, consistency, and accuracy.
- Handle data validation, cleansing, and error handling to maintain data integrity.
- Monitor and proactively maintain data pipelines to ensure high service availability.
Performance Optimization:
- Continuously improve pipeline performance by identifying bottlenecks and implementing optimizations.
- Work with cloud-based technologies (e.g., AWS, GCP, Azure) to enhance scalability and efficiency.
Collaboration and Leadership:
- Partner with Data Scientists, Analysts, and other stakeholders to understand their data needs.
- Lead discussions on system enhancements, process improvements, and data governance.
- Mentor junior engineers and contribute to the growth of the data engineering team.
This role will be performed 80% remote with 20% onsite.
#qf #pmf
Requirements:- Bachelor's degree in Computer Science, Engineering, or a related field.
- 3+ years of experience in data engineering, with a focus on building and maintaining data pipelines.
- 1-2 years experience with building NiFI data flows or similar for Kafka and Hadoop-based NoSQL databases.
- Proficiency in ETL tools, SQL, and scripting languages (Python, Scala, etc.).
- Experience with Data Catalog and Accumulo indexes for information retrieval and discovery.
- Experience with API-led design.
- Experience with ELK stack a plus.
- Experience with cloud-based data platforms (e.g., AWS S3, Redshift, Google BigQuery).
- Strong problem-solving skills and attention to detail.
- Excellent communication and collaboration abilities.
- Security+ Certification
- Active US Government Clearance at Secret level or higher
- Ability to sit for extended periods of time.
- Ability to regularly lift at least 25 pounds.
- Ability to commute to the designated onsite work location as required.
QBE is an equal opportunity/affirmative action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender, gender-identity and/or expression, age, disability, Veteran status, genetic information, pregnancy (including childbirth, lactation, or other related medical conditions), marital-status, neurodivergence, ethnicity, ancestry, caste, military/uniformed service-member status, or any other characteristic protected by applicable federal, state, local, or international law.
PI33ac7c8d2bf
-
Mid-Level Infrastructure Engineer
2 weeks ago
Tampa, Florida, United States Carollo Engineers Full timeOverview Carollo Engineers is an internationally recognized water engineering firm that specializes in the planning, design, and construction of water and wastewater facilities and infrastructure. We are seeking an Mid Level Infrastructure Engineer to support our vision of becoming the best water consulting firm. As a member of our team, you will work on...
-
Data Engineer
4 weeks ago
Tampa, United States Alta Mira Full timeDescription Data Engineer Altamira delivers a variety of analytic and engineering capabilities to the US National Security community, but the tech culture and the caliber of the individuals that bring these capabilities to fruition are what really set us apart. We’re a curious, responsive, dedicated bunch spread across many corporate cultures. Dayton, OH...
-
Sr. Data Engineer
4 weeks ago
Tampa, United States Virtual Full timeENVIRONMENT: Data Modeling experience ETL pipelines Azure data factory SSIS experience Reporting data in the data mart Team Size: 2 other senior data engineers Overview: The Senior Data Engineer is primarily responsible for designing, building, and maintaining the infrastructure that supports data storage, flow, and retrieval. The Senior Data...
-
Early Career Pipeline Engineer
2 weeks ago
Tampa, Florida, United States WSP Full timeThis Opportunity WSP USA, Inc. is currently seeking a talented and motivated Early Career Pipeline Engineer to join us. This position is based in either our Tampa, FL or Houston, TX office. We are looking for an individual who is passionate about their work, eager to learn, and driven to excel in a dynamic and collaborative environment Your Impact Perform...
-
Data Engineer
4 weeks ago
Tampa, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION?Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology...
-
Data Engineer
1 month ago
Tampa, Florida, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION?Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology...
-
Data Engineer
2 weeks ago
Tampa, Florida, United States Amgen Full time**HOW MIGHT YOU DEFY IMAGINATION?** Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading...
-
Data Engineer
2 weeks ago
Tampa, Florida, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION?Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology...
-
Data Engineer
4 weeks ago
Tampa, United States RIT Solutions, Inc. Full timeData Engineer Duration: 6 months Needed for submittal: Birthday (Month/Day) City/State Email / Phone Education and dates on resume Full Legal name (including middle) Skills: 7+ years of development experience building data pipelines using Cloud technologies. 5+ years of experience in architecture of modern data warehousing platforms...
-
Data Engineer
2 weeks ago
Tampa, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION?Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology...
-
Data Engineer
3 weeks ago
Tampa, United States BioSpace, Inc. Full timeJob Details HOW MIGHT YOU DEFY IMAGINATION? Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading...
-
Senior Data Engineer
2 days ago
Tampa, United States Take2 Consulting, LLC Full timeAre you an experienced Data Engineer on the Azure platform? Take2 Consulting is looking for a Senior Data Consultant to come join the team with one of our clients here in the Tampa Bay Area. This role is 100% remote, but will operate on EST. If you have great communciation and people skills, as well as the technical background listed in the summary below,...
-
Data Engineer
4 weeks ago
Tampa, United States General Dynamics Information Technology Full timeClearance Level Secret Category Data Science Location Tampa, Florida Onsite Workplace Public Trust: None Requisition Type: Regular Your Impact Own your opportunity to work with the largest government agency in the nation. Make an impact by advancing the Department of Defense’s mission to keep our country safe and secure. Job Description Deliver insights to...
-
Lead Data Engineer
5 days ago
Tampa, United States E2 MEDIA LLC Full timeJob DescriptionJob Description18 month contract to hireRate: $65.97 W-2 with Medical benefitsClient: Major entertainment clientLocation: ONE DAY per week in Orlando, 32819 Primary Job Duties: ETL Development, Dashboard / Report Development, Quality Assurance Top Skills: SQL, Tableau, PowerBI, SnowflakeJob Summary:We are in search of a Lead Data Engineer to...
-
Associate Software/Data Engineer
1 month ago
Tampa, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION? You’ve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared mission—to serve patients—drives all that we do. It is key to our becoming one of the world’s leading...
-
Data Systems Engineer
2 weeks ago
Tampa, Florida, United States Leidos Full timeDescriptionExciting Opportunity at LeidosLeidos is known for delivering cutting-edge solutions thanks to our diverse and talented team members who are committed to ensuring the success of our clients. Our focus on doing what's best for our customers, employees, and communities is unwavering. Our Mission, Vision, and Values underpin our business...
-
Associate Software/Data Engineer
1 month ago
Tampa, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION?Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology...
-
Associate Software/Data Engineer
1 month ago
Tampa, Florida, United States Amgen Full timeHOW MIGHT YOU DEFY IMAGINATION?Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading biotechnology...
-
Associate Software/Data Engineer
4 weeks ago
Tampa, United States Amgen Full time**HOW MIGHT YOU DEFY IMAGINATION?** Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading...
-
Associate Software/Data Engineer
1 month ago
Tampa, United States BioSpace, Inc. Full timeJob Details HOW MIGHT YOU DEFY IMAGINATION? Youve worked hard to become the professional you are today and are now ready to take the next step in your career. How will you put your skills, experience and passion to work toward your goals? At Amgen, our shared missionto serve patientsdrives all that we do. It is key to our becoming one of the worlds leading...
