Real-Time Data Engineer
4 weeks ago
**Key Responsibilities:**
* Design and develop real-time data pipelines to ingest, process, and analyze streaming data from various sources, ensuring low-latency and high-throughput.
* Implement stream processing applications using technologies such as Apache Kafka, Apache Flink, Apache Spark Streaming, or AWS Kinesis to process and analyze real-time data streams.
* Integrate real-time data streams with existing data systems and applications, ensuring data consistency and coherence across systems.
* Transform and enrich real-time data streams to derive meaningful insights and enable real-time decision-making, using techniques such as complex event processing (CEP) and real-time analytics.
* Implement data quality checks, validation rules, and error handling mechanisms to ensure the accuracy, completeness, and consistency of real-time data.
* Optimize real-time data processing pipelines for speed, scalability, and efficiency, tuning streaming applications and leveraging distributed computing technologies.
* Monitor real-time data infrastructure and systems for performance, reliability, and availability, proactively identifying and addressing issues to minimize downtime and ensure data integrity.
* Document real-time data architecture, processes, and best practices, and collaborate with cross-functional teams to ensure alignment and transparency.
**Requirements:**
* Bachelor's degree or higher in Computer Science, Engineering, or related field.
* Strong background in real-time data engineering, with hands-on experience in designing, building, and optimizing real-time data solutions.
* Proficiency in programming languages such as Python, Java, or Scala, and experience with stream processing frameworks such as Apache Kafka, Apache Flink, or Apache Spark Streaming.
* Experience with cloud platforms such as AWS, Google Cloud Platform, or Microsoft Azure, and familiarity with cloud services for real-time data processing (e.g., AWS Kinesis, Google Cloud Dataflow, Azure Stream Analytics).
* Knowledge of distributed computing technologies and concepts, including message brokers, distributed databases, and microservices architecture.
* Strong problem-solving abilities and analytical thinking, with a keen attention to detail and a passion for tackling complex technical challenges.
* Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams and communicate technical concepts to non-technical stakeholders.
**Benefits:**
* Competitive salary: The industry standard salary for Real-Time Data Engineers typically ranges from $140,000 to $210,000 per year, depending on experience and qualifications.
* Comprehensive health, dental, and vision insurance plans.
* Flexible work hours and remote work options.
* Generous vacation and paid time off.
* Professional development opportunities, including access to training programs, conferences, and workshops.
* State-of-the-art technology environment with access to cutting-edge tools and resources.
* Vibrant and inclusive company culture with team-building activities and social events.
* Opportunities for career growth and advancement within the company.
* Exciting projects with real-world impact in the field of real-time data engineering.
* Chance to work alongside top talent and industry experts in real-time data engineering.
**About Unreal Gigs:**
Unreal Gigs is a leading company in the field of real-time data engineering, dedicated to harnessing the power of real-time data to drive insights and enable real-time decision-making. We are committed to developing scalable and efficient real-time data solutions that empower organizations to leverage data in real-time.
If you are passionate about real-time data engineering and eager to drive innovation through scalable data solutions, we want you on our team. Apply now to join our team and be part of an exciting journey of innovation and discovery.
-
Real-Time Data Engineer
3 weeks ago
San Francisco, California, United States Unreal Gigs Full timeJob OverviewAt Unreal Gigs, we're pushing the boundaries of real-time data engineering. As a key member of our team, you'll play a pivotal role in designing, building, and optimizing our real-time data infrastructure and systems.As a Real-Time Data Engineer, you'll work closely with cross-functional teams of data scientists, software engineers, and business...
-
Lead Real-Time Data Architect
4 weeks ago
San Francisco, California, United States Unreal Gigs Full timeJob OverviewWelcome to Unreal Gigs, where we're pushing the boundaries of data-driven innovation. As the Lead Real-Time Data Architect, you'll be at the forefront of designing, building, and optimizing real-time data infrastructure. Your expertise in real-time data technologies will enable timely insights and actions, driving transformative change across...
-
Data Engineer
3 weeks ago
San Francisco, California, United States Fintool Full timeAbout FintoolFintool is an AI Equity Research Copilot for institutional investors. We're building the AI Warren Buffett. Our team is on the fastest growing LLM vertical applications, with thousands of investors signed up for Fintool. We're backed by Y Combinator and entrepreneurs such as the co-founders of Datadog, Vercel, HuggingFace, or domain experts from...
-
Senior Data Engineer
3 weeks ago
San Jose, California, United States Tik Tok Full timeJob Title: Senior Data Engineer - Real-Time Data WarehouseAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our commercial real-time data warehouse team. As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing our real-time data warehouse solution.Key Responsibilities:Design and develop a...
-
San Francisco, California, United States Cisco Full timeAbout the RoleCisco is seeking a highly skilled Lead Software Engineer to join our Events team. As a key member of this team, you will be responsible for building a highly scalable stream processing platform to detect anomalies and provide actionable insights to our customers.We are looking for a talented engineer with expertise in designing and building...
-
Data Engineer, Real Estate and Workplace
3 weeks ago
San Francisco, California, United States OpenAI Full timeAbout the TeamAt OpenAI, our Real Estate and Workplace team is responsible for planning our Real Estate needs and elevating the employee experience through different delineations of the industry (Facilities, Operations, Experience) to support our growing company in its mission to ensure that AGI benefits all of humanity.We're looking for individuals who are...
-
Data Engineer
4 weeks ago
San Francisco, California, United States OpenAI Full timeAbout the TeamOpenAI's Real Estate and Workplace team is responsible for planning and elevating the employee experience through data-driven insights.The team's mission is to ensure that artificial intelligence benefits all of humanity, and they strive to achieve this by creating a workplace that is both innovative and inclusive.About the RoleAs a Data...
-
Senior Data Engineer
3 weeks ago
San Jose, California, United States Tik Tok Full timeJob SummaryAt TikTok, we're looking for a skilled Senior Data Engineer - Real-Time Data Warehouse to join our team. As a key member of our commercial data platform, you will be responsible for the development and optimization of our real-time data warehouse. This is a unique opportunity to work on a large-scale streaming data computing project, leveraging...
-
Senior Data Engineer
4 weeks ago
San Jose, California, United States Tik Tok Full timeJob DescriptionTikTok is a leading destination for short-form mobile video, and our mission is to inspire creativity and bring joy. We're looking for a talented Data Streaming Engineer to join our team and help us achieve this goal.As a Data Streaming Engineer, you will be responsible for developing and optimizing TikTok's commercial real-time data...
-
Lead Software Engineer, Real-Time Data Platform
3 weeks ago
San Francisco, California, United States Cisco Full timeAbout the RoleThe Events team at Cisco is responsible for building a highly scalable stream processing platform to detect anomalies and provide actionable insights to our customers.We work on problems such as identifying timely automated events, LLM/GenAI-based summarization and remediation, and building and deploying multiple stream processing platforms.We...
-
San Francisco, California, United States Openai Full timeAbout the TeamOpenAI has a global presence with a custom-built headquarters in San Francisco and multiple offices worldwide where our teams collaborate.The Real Estate and Workplace team is responsible for planning our Real Estate needs and elevating the employee experience through different delineations of the industry (Facilities, Operations, Experience)...
-
Software Engineer, Real-time Data Ingestion
3 weeks ago
San Francisco, California, United States Orb Full timeAbout OrbOrb is a modern billing infrastructure that powers revenue workflows for the next wave of AI and software companies. We believe that pricing and billing shouldn't be a blocker to building and shipping great products.We power billing for highly innovative companies like Vercel, Pinecone, Perplexity, Replit, and work with our customers on some of...
-
Senior Data Engineer
4 weeks ago
San Francisco, California, United States Gridware Full timeJob Title: Senior Data EngineerGridware is a leading provider of advanced sensing systems for the electrical grid. Our team is responsible for designing, building, and optimizing data pipelines and infrastructure to support real-time grid monitoring systems.The Senior Data Engineer will be responsible for handling large-scale sensor data, ensuring data...
-
Senior Data Engineer
4 weeks ago
San Francisco, California, United States Gridware Full timeJob OverviewGridware is a pioneering company that aims to enhance and protect the electrical grid, a critical infrastructure that underpins modern society. As a Senior Data Engineer, you will play a crucial role in designing, building, and optimizing data pipelines and infrastructure to support real-time grid monitoring systems.The ideal candidate will have...
-
Senior Data Engineer
4 weeks ago
San Francisco, California, United States Unreal Gigs Full timeUnreal Gigs is seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining our data infrastructure and pipelines.Key Responsibilities:* Design and implement scalable and efficient data pipelines for ingesting, processing, and transforming large...
-
San Francisco, California, United States Discord Full timeAt Discord, we're building a platform that enables people to create spaces for belonging in their lives. We're looking for a skilled Staff Software Engineer to join our Real Time Infrastructure team, responsible for building, maintaining, and scaling the systems that power chat, push notifications, presence, and more for over 150 million active users.This...
-
Data Engineer
3 weeks ago
San Francisco, California, United States Bayone Full timeJob Title: Data EngineerCompany Overview:At Bayone, we strive to create an environment where people are valued, and differences are celebrated. Our purpose is to empower our teams and inspire our customers to become the best versions of themselves.The Opportunity:Our technology team works fast and smart, developing the latest in mobile technologies, scalable...
-
Data Visualization Engineer
4 weeks ago
San Francisco, California, United States kbit LLC Full timeJob DescriptionWe are seeking a skilled Data Visualization Engineer to join our team at kbit LLC. As a systematic trading firm, our success depends on the expertise of our engineers and researchers who collaborate to develop and implement profitable trading strategies.The ideal candidate is adaptable, takes initiative with projects, and consistently delivers...
-
Senior Data Engineer
3 weeks ago
San Francisco, California, United States EliseAI Full timeAbout EliseAIEliseAI is a cutting-edge conversational AI technology company that focuses on developing innovative solutions for industries fundamental to our lives: housing and healthcare. Our mission is to leverage data as a strategic asset to drive decision-making processes. We're seeking a skilled Senior Data Engineer to join our Engineering team and play...
-
Data Visualization Engineer
3 weeks ago
San Francisco, California, United States kbit LLC Full timeJob Title: Data Visualization EngineerWe are a successful crypto hedge fund with a strong track record of delivering returns for our investors. Our success depends on the expertise of our engineers and researchers who collaborate to develop and implement profitable trading strategies.We're seeking a skilled Data Visualization Engineer to join our team and...