Data Pipeline Developer
2 weeks ago
Company Overview: Welcome to the forefront of data-driven innovation At our company, we're committed to harnessing the power of data to drive insights and empower decision-making. Our mission is to develop scalable and efficient data pipeline solutions that enable organizations to streamline data processing and analysis. Join us and be part of a dynamic team dedicated to shaping the future of data pipeline development.
Position Overview: As a Data Pipeline Developer, you'll play a crucial role in designing, building, and maintaining our data pipeline infrastructure and workflows. Working closely with cross-functional teams of data engineers, analysts, and business stakeholders, you'll ensure the reliability, scalability, and efficiency of our data pipeline solutions. If you're passionate about data engineering and eager to drive innovation through scalable data solutions, we want you on our team.
Requirements
Key Responsibilities:
- Data Pipeline Design: Design and architect scalable and reliable data pipeline solutions to automate data ingestion, processing, and transformation tasks.
- Data Integration: Integrate data from diverse sources, including databases, files, APIs, and streaming platforms, into our data pipeline ecosystem, ensuring data consistency and coherence.
- Data Processing: Develop and maintain data processing workflows to cleanse, enrich, and transform raw data into formats suitable for analysis and reporting.
- Data Orchestration: Orchestrate and schedule data pipeline workflows using workflow management tools and frameworks (e.g., Apache Airflow, Luigi, Prefect).
- Performance Optimization: Optimize data pipeline performance for speed, efficiency, and scalability, tuning data processing tasks and optimizing resource utilization.
- Data Quality and Governance: Implement data quality checks, validation rules, and error handling mechanisms to ensure the accuracy, completeness, and consistency of data in the pipeline.
- Monitoring and Maintenance: Monitor data pipeline performance and health, proactively identifying and addressing issues to minimize downtime and optimize resource utilization.
- Documentation and Collaboration: Document data pipeline architecture, processes, and best practices, and collaborate with cross-functional teams to ensure alignment and transparency.
Qualifications:
- Bachelor's degree or higher in Computer Science, Engineering, or related field.
- Strong background in data engineering, with hands-on experience in designing, building, and optimizing data pipeline solutions.
- Proficiency in programming languages such as Python, Java, or Scala, and experience with data processing frameworks such as Apache Spark, Apache Beam, or Apache Flink.
- Experience with workflow management tools and frameworks such as Apache Airflow, Luigi, or Prefect.
- Familiarity with data integration techniques and technologies, including ETL (Extract, Transform, Load) processes and streaming data processing.
- Strong problem-solving abilities and analytical thinking, with a keen attention to detail and a passion for tackling complex technical challenges.
- Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams and communicate technical concepts to non-technical stakeholders.
Benefits
- Competitive salary: The industry standard salary for Data Pipeline Developers typically ranges from $140,000 to $210,000 per year, depending on experience and qualifications. Exceptional candidates may be eligible for higher compensation packages.
- Comprehensive health, dental, and vision insurance plans.
- Flexible work hours and remote work options.
- Generous vacation and paid time off.
- Professional development opportunities, including access to training programs, conferences, and workshops.
- State-of-the-art technology environment with access to cutting-edge tools and resources.
- Vibrant and inclusive company culture with team-building activities and social events.
- Opportunities for career growth and advancement within the company.
- Exciting projects with real-world impact across diverse industries.
- Chance to work alongside top talent and industry experts in the field of data engineering.
Join Us: Ready to shape the future of data pipeline development? Apply now to join our team and be part of an exciting journey of innovation and discovery
-
Lead Data Pipeline Developer
2 weeks ago
San Francisco, United States Unreal Gigs Full timeJob DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data pipeline solutions that enable efficient data ingestion, processing, and...
-
Lead Data Pipeline Developer
7 days ago
San Francisco, United States Unreal Gigs Full timeJob DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data pipeline solutions that enable efficient data ingestion, processing, and...
-
Senior Data Pipeline Developer
2 weeks ago
San Francisco, United States Unreal Gigs Full timeJob DescriptionJob DescriptionCompany Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data pipeline solutions that enable efficient data ingestion, processing, and...
-
Senior Data Pipeline Developer
3 days ago
San Francisco, CA, United States Unreal Gigs Full timeJob Description Job Description Company Overview: Welcome to the forefront of data-driven innovation! Our company is dedicated to leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge data pipeline solutions that enable efficient data ingestion, processing,...
-
Software Developer, Data Pipelines
6 days ago
San Francisco, California, United States Autodesk Full timeJob Requisition ID #24WD77850Job Title: Software Engineer, Data Pipelines - Generative AIPosition OverviewAutodesk is leading the transformation of the AEC industry, integrating AI technology into our products. We're enhancing our applications with cloud-native capabilities, including data at scale, edge computing, AI-based solutions, and advanced 3D...
-
Senior Data Engineer
7 days ago
San Francisco, United States People Data Labs Full timeJob DescriptionJob DescriptionAbout UsAt People Data Labs, we're committed to democratizing access to high-quality B2B data and leading the emerging DaaS economy. We empower developers, engineers, and data scientists to create innovative, compliant data products at scale with our clean, easy-to-use datasets of resume, company, location, and education...
-
Data Analyst
6 days ago
San Francisco, United States People Data Labs Full timeJob DescriptionJob DescriptionAbout UsAt People Data Labs, we're committed to democratizing access to high-quality B2B data and leading the emerging DaaS economy. We empower developers, engineers, and data scientists to create innovative, compliant data products at scale with our clean, easy-to-use datasets of resume, company, location, and education...
-
Sales Development Representative
4 days ago
San Francisco, United States Pipeline Gurus Full time**Role Overview**: As a Sales Development Representative at Pipeline Gurus, you will be a pivotal part of our sales team, responsible for generating new business opportunities and driving our growth strategy. Your role will focus on identifying potential clients, engaging them through targeted outreach, and nurturing leads to create sales-ready...
-
San Francisco, United States Stripe Full timeWho we are About Stripe Stripe is a financial infrastructure platform for businesses. Millions of companies - from the world's largest enterprises to the most ambitious startups - use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount...
-
Sales Development Executive
7 days ago
San Francisco, United States NTT DATA Full timeReq ID: 277353 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sales Development Executive (Remote) to join our team in Plano, Texas (US-TX), United States (US). At NTT...
-
Dir. of Philanthropy, Pipeline Development
2 months ago
San Diego, CA, United States St. Vincent De Paul Village Full timeDirector of Philanthropy, Pipeline Development Senior Director of Philanthropy The director of philanthropy, pipeline development (DOP, PD) plays a key strategic role in elevating the office of philanthropy into a disciplined, data driven operation, advising on every phase of the donor cycle, broadly and on the individual donor level, to include...
-
Sales Development Representative
1 day ago
San Francisco, United States Pipeline Gurus Full timeRole Overview:As a Sales Development Representative at Pipeline Gurus, you will be a pivotal part of our sales team, responsible for generating new business opportunities and driving our growth strategy. Your role will focus on identifying potential clients, engaging them through targeted outreach, and nurturing leads to create sales-ready opportunities. You...
-
Sales Development Representative
3 days ago
San Francisco, United States Pipeline Gurus Full timeJob DescriptionJob DescriptionRole Overview:As a Sales Development Representative at Pipeline Gurus, you will be a pivotal part of our sales team, responsible for generating new business opportunities and driving our growth strategy. Your role will focus on identifying potential clients, engaging them through targeted outreach, and nurturing leads to create...
-
Sr. Data Analyst
3 days ago
San Francisco, CA, United States Windfall Data, Inc. Full timeAt Windfall, we aim to democratize access to people data and insights. We use a vast and varied data asset to help non-profits exceed their donation goals, and commercial companies achieve higher ROI on their marketing spend. As a Sr. Data Analyst at Windfall, your role will be to contribute to a deeper understanding of these data assets, and how they can...
-
Machine Learning Engineer
2 months ago
San Rafael, CA, United States Karkidi Full time24WD75344 Machine Learning Engineer (Pipeline Developer) Position presentation Autodesk designs software for creators. If you've ever driven a high-performance car, looked at a towering skyscraper, used a smartphone, or seen a great sci-fi movie, chances are you've admired what millions of Autodesk customers are designing. help of our software....
-
Dir. of Philanthropy, Pipeline Development
2 months ago
San Diego, CA, United States St. Vincent De Paul Village Full timeJob DescriptionJob Description Job Details Job Title Director of Philanthropy, Pipeline Development Reports To Senior Director of Philanthropy Salary Grade E8 FLSA Status Exempt Effective Date January 2024 Summary The director of philanthropy, pipeline development (DOP, PD) plays a key strategic role in elevating the office of philanthropy into a...
-
Data Engineer
1 month ago
San Francisco, United States Legalist Full timeJob DescriptionJob DescriptionIntro description:Legalist is an institutional alternative asset management firm. Founded in 2016 and incubated at Y Combinator, the firm uses data-driven technology to invest in credit assets at scale. We are always looking for talented people to join our team.As a highly collaborative organization, our data engineers work...
-
Machine Learning Engineer
2 months ago
San Rafael, CA, United States Karkidi Full time24WD75344 Machine Learning Engineer (Pipeline Developer) Position presentation Autodesk designs software for creators. If you've ever driven a high-performance car, looked at a towering skyscraper, used a smartphone, or seen a great sci-fi movie, chances are you've admired what millions of Autodesk customers are designing. help of our software....
-
Data Engineer
7 days ago
San Francisco, United States Rigil Corporation Full timeJob DescriptionJob DescriptionRole: Data EngineerAbout Rigil:Rigil is an award-winning, woman-owned, small business that specializes in technology consulting, strategy consulting and product development. We value teamwork and strive to build strong leaders.Location: San Francisco, CAJob Type: Full TimeData Engineer is integral to evolving Vision and its...
-
Data Engineer
7 days ago
San Francisco, United States Rigil Corporation Full timeRole: Data Engineer About Rigil: Rigil is an award-winning, woman-owned, small business that specializes in technology consulting, strategy consulting and product development. We value teamwork and strive to build strong leaders.Location: San Francisco, CAJob Type: Full TimeData Engineer is integral to evolving Vision and its underlying data and AI...
