Data Engineer for Scalable Data Pipelines
1 month ago
We're on a mission to empower software organizations to deliver high-quality projects quickly. Our unique approach enables engineering teams to stay focused on designing and implementing software, leading to the creation of magical projects and happier teams.
Our team is growing rapidly, with our founders leveraging their experience from some of the world's largest tech companies. We've secured significant funding and are looking for entrepreneurial engineers who are passionate about solving problems in software development.
As a founding member of our Data Engineering team, you'll build data pipelines for processing and storing customer data generated by CI/CD and automated tests. You'll experiment with integrating AI models to drive analytics and insights for customers. We tackle challenging problems and need engineers who can operate well in ambiguity.
Our data stack is built on Python, PostgreSQL, Spark, TimescaleDB, AWS, Kubernetes, and AWS Glue. You'll work closely with customers to understand their use cases and develop solutions that meet their needs.
Key Responsibilities- Design fault-tolerant and scalable data pipelines
- Create efficient data storage systems with product engineers
- Optimize distributed data-intensive applications for latency, accuracy, resource consumption, and throughput
- Develop observability of data quality and accuracy
- Integrate machine learning models like Llama to analyze data
- 10-12+ years of experience as a software engineer with strong understanding of distributed systems
- Experience building and deploying data applications with track record of shipping new features regularly
- Fluency in at least two programming languages: Java, Scala, Kotlin, Python, Go, Rust, or C++
- Good understanding of partitioning, replication, map-reduce, indexing, and CAP theorem
- Experience with distributed storage systems, processing engines, and message queues
- Passion for building large-scale ML applications and improving software engineers' productivity
- Understanding of natural language processing, machine learning, or statistical analysis
- Competitive salary range $200K - $245K
- Up to .5% equity
- Unlimited PTO
- Work-life balance
- Flexibility to be fully or partly remote
- $200/month stipend for coworking space for remote folks
- Paid parental leave (up to 12 weeks)
- Top-notch medical, dental, vision, short-term disability, long-term disability, and life insurance
- Frontend: Typescript, React, Redux, Next.js
- Backend: Typescript, Node, AWS, CDK, k8s, gRPC
- Observability: Prometheus, Grafana, Kiali, Jaeger
- CI/CD: GitHub Actions
- CLI/Daemon/LSP: C++20, Bazel
-
Scalable Data Pipeline Specialist
2 weeks ago
San Francisco, California, United States Airtable Full timeJob OverviewThe successful candidate will design and own mission-critical data pipelines to enable decision-making. They'll partner with company leaders to create scalable data solutions and launch innovative alerting and visualization tools. With a focus on collaboration, they'll work between our engineering organization and stakeholders from data science,...
-
Data Engineer for Scalable Data Platforms
2 weeks ago
San Francisco, California, United States TEKsystems Full timeAs a Data Engineer at TEKsystems, you will be responsible for designing and implementing scalable data platforms that meet the needs of our clients. In this role, you will work with our team to develop and maintain ETL pipelines, data warehouses, and other data systems that are used by businesses to make informed decisions.The ideal candidate will have...
-
Data Engineer for Scalable Data Systems
3 weeks ago
San Francisco, California, United States Grow Full timeAbout RockerboxRockerbox empowers marketing executives to make informed decisions with confidence. We help companies like Tula, Figs, and Burton drive growth through strategic decision-making.Our Integrations team plays a crucial role in our success, managing and scaling data pipelines that ingest marketing data from third-party APIs. This data is validated,...
-
San Francisco, California, United States Recruiting From Scratch Full timeCompany OverviewRecruiting from Scratch is a talent firm that focuses on placing the best candidates for our clients. Our team is 100% remote, working with teams across North America, South America, and Europe to help them hire.SalaryThe estimated salary range for this position is $130,000-$180,000 per year.Job DescriptionWe are seeking a Technical Lead to...
-
Data Architect for Scalable Cloud Solutions
1 month ago
San Francisco, California, United States Unreal Gigs Full timeWe are seeking a seasoned Data Architect to lead the design and implementation of scalable cloud-based data solutions at Unreal Gigs. In this role, you will be responsible for architecting and building data pipelines that support ETL processes in cloud platforms such as AWS, GCP, or Azure.Key Responsibilities:Data Pipeline Architecture: Design and build...
-
Data Engineer for Scalable Data Platforms
4 weeks ago
San Francisco, California, United States Faire Full timeData Engineer for Scalable Data PlatformsFaire is an online wholesale marketplace that empowers entrepreneurs to grow their businesses. Our mission is to level the playing field by leveraging technology and data.As a Data Engineer on our Core Data Infrastructure team, you will design and build data capabilities that inform product launches and roadmaps. You...
-
Data Pipeline Specialist
2 weeks ago
San Francisco, California, United States UnitedHealth Group Full timeAbout the Job\This role involves working with distributed systems, designing and implementing data pipelines, and optimizing data delivery. The successful candidate will have 3+ years of experience in applied ML/AI engineering, 3+ years of experience in designing and managing distributed systems, and 3+ years of experience using Python or similar programming...
-
San Francisco, California, United States AirTree Ventures Pty Full timeAt Linktree, we are seeking a skilled Senior Data Platform Architect to join our team as we continue to drive innovation in the data platform space. As a key member of our engineering team, you will play a crucial role in building and maintaining a robust, scalable data platform that supports our mission to empower anyone to curate, grow and monetize their...
-
Lead Data Pipeline Developer
2 weeks ago
San Francisco, California, United States ZipRecruiter Full timeJob Overview: As a Lead Machine Learning Infrastructure Engineer, you will play a pivotal role in leading our machine learning infrastructure initiatives and driving the design, development, and optimization of our infrastructure solutions. You will lead a team of skilled engineers, collaborating closely with cross-functional teams to deliver high-quality,...
-
Data Engineer for Cloud Data Platforms
2 weeks ago
San Francisco, California, United States ZipRecruiter Full timeAt ZipRecruiter, we are seeking a highly skilled Data Engineer for Cloud Data Platforms to join our team. This role will be responsible for designing and building scalable data pipelines that support ETL processes in cloud platforms such as AWS, GCP, or Azure.With a strong background in cloud data engineering and experience with data pipeline orchestration...
-
Data Engineer
3 weeks ago
San Francisco, California, United States Perplexity AI Full timePerplexity AI is rapidly scaling both in number of use cases and users. We're seeking an experienced Data Engineer to help build our end-to-end data stack and flywheel.The successful candidate will collaborate closely with Product, Backend, and Data Science teams to design, build, and maintain scalable data pipelines and infrastructure. Key responsibilities...
-
Data Engineering Lead
2 weeks ago
San Francisco, California, United States eTeam Inc. Full timeeTeam Inc. is seeking a talented Data Engineering Lead to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and managing large-scale data pipelines and workflows using Trino SQL/Spark SQL warehoused in HDFS datasets.We offer a competitive salary of $120,000 - $150,000 per year, depending on...
-
Cloud Data Engineer
2 weeks ago
San Francisco, California, United States eTeam Inc. Full timeeTeam Inc. is seeking a highly skilled Cloud Data Engineer to join our team. As a key member of our cloud data engineering team, you will be responsible for designing, developing, and managing large-scale data pipelines and workflows using Trino SQL/Spark SQL warehoused in HDFS datasets in a cloud environment.We offer a competitive salary of $110,000 -...
-
Data Engineer Lead
2 weeks ago
San Francisco, California, United States Unity Technologies Full timeJob DescriptionThis is an exciting opportunity to work with our global and cross-functional teams to create meaningful data products and drive technical solutions. As a Senior Data Engineer, you will be responsible for designing and developing data pipelines and services to enable data-driven insights and power BI, ML, experimentation, and user-facing...
-
Data Engineer: Scalable Solutions
2 weeks ago
San Francisco, California, United States Akraya Full timeCompany OverviewAkraya is a leading IT staffing firm recognized for its commitment to excellence and a thriving work environment. As a seasoned Data Engineer, you will play a pivotal role in developing high-scalable web and cloud solutions, focusing on data warehousing, analytics, and database management to support our marketing strategies.">Salary and...
-
Data Engineer Leader
2 weeks ago
San Francisco, California, United States Baton Full timeWe are seeking a skilled Data Engineer Leader to lead data engineering efforts and develop scalable data pipelines. At Baton, we believe that data is the backbone of any successful organization, and we need someone who can build and maintain high-quality data systems.">The ideal candidate will have experience with cloud platforms like AWS, strong SQL skills,...
-
Data Engineering Lead
3 weeks ago
San Francisco, California, United States Pendulum Full timePioneering a Health RevolutionPendulum is at the forefront of a global movement to improve physical and mental health through advanced microbiome research and innovative probiotic solutions.The company's cutting-edge probiotic pipelines and discovery platform have disrupted the consumer probiotics market, offering therapeutic products that bridge the gap...
-
Data Engineering Innovator
4 weeks ago
San Francisco, California, United States ZipRecruiter Full timeData Engineer Position OverviewZipRecruiter is seeking a highly skilled Data Engineer to join our team. This role involves designing and implementing scalable data pipelines using big data technologies such as Apache Spark, Hadoop, and Kafka.Job Responsibilities:Design and Build Scalable Data Pipelines: Create robust and efficient data workflows that handle...
-
Cloud Data Engineer
3 weeks ago
San Francisco, California, United States Amazon Full time**What You'll Do**Collaborate with data scientists and engineers to develop scalable data pipelines using Spark, EMR, Python, Redshift, Glue, and S3.Simplify complex datasets by creating data cubes and sharing solutions, enhancing accessibility and usability.
-
Data Engineering Lead
3 weeks ago
San Francisco, California, United States WEX Full timeAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our dynamic team at WEX. The ideal candidate will have extensive experience in designing and implementing cloud-based data solutions, with a strong focus on scalability, performance, and data governance.The Data Engineering Lead will be responsible for leading the design and...