Data Engineering Specialist

3 days ago

Palo Alto, California, United States Georgia IT Inc Full time
Data Engineer Role Overview
Georgia IT Inc is seeking a skilled Data Engineer to join our team in Palo Alto, CA. This long-term role offers a competitive $140,000 - $180,000 per year salary for individuals with experience in data orchestration and workflow management.

Key Responsibilities:
  • Design and implement data workflows using Airflow to automate and orchestrate data processing tasks.
  • Optimize Airflow DAGs (Directed Acyclic Graphs) for performance and scalability.
Task Management:
  • Develop and maintain distributed task processing using Celery and ensure robust task queue management with Redis or RabbitMQ.
Database Management:
  • Design and manage databases using Cosmos DB, MongoDB, and PostgreSQL.
  • Develop and maintain efficient data models and ensure data consistency and integrity.
API and Webhooks:
  • Implement and manage FastAPI webhooks to handle data ingestion and integration tasks.
  • Develop and maintain Azure Functions to support webhook operations and integrate with cloud services.
Streaming Data:
  • Implement and manage Kafka Streams to handle real-time data processing and streaming requirements.
Data Lake Management:
  • Work with Iceberg to manage and optimize large-scale data lake storage and querying.
Collaboration and Communication:
  • Collaborate with data scientists, engineers, and business analysts to understand data requirements and provide technical solutions.
  • Document processes, architectures, and configurations to ensure knowledge sharing and compliance with best practices.
Required Skills and Qualifications:

Technical Requirements:
  • Strong understanding of data pipelines, ETL processes, and data integration.
  • Proficient in Python, with experience in building and maintaining data-oriented applications.
  • Ability to work with large datasets and optimize performance across distributed systems.
Soft Skills:
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.
  • Ability to work independently and manage multiple priorities in a fast-paced environment.

  • Data Engineer Specialist

    3 days ago

    Palo Alto, California, United States Tesla Full time

    Job SummaryWe are seeking an experienced Data Engineer Specialist to join our Self-Driving Data and Analytics team at Tesla. This is a high-impact role that requires strong analytical skills, expertise in data engineering, and the ability to work with large datasets.In this position, you will be responsible for designing, developing, and maintaining...

  • Senior Data Protection Engineer

    1 week ago

    Palo Alto, California, United States InDepth Engineering Solutions, LLC Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Protection Engineer to join our team at InDepth Engineering Solutions, LLC. This role will play a critical part in enhancing our cybersecurity posture by focusing on tactical data protection operations, automation, and tool development.The ideal candidate will have a strong technical background, a...

  • Data Science Specialist

    3 days ago

    Palo Alto, California, United States Amazon Full time

    Job DetailsWe are looking for a seasoned Data Science Specialist to join our team working on a revolutionary LLM project for Amazon Stores. In this role, you will be responsible for analyzing complex data sets, designing and developing machine learning models, and collaborating with cross-functional teams to drive business outcomes. You will work closely...

  • Data Science and Engineering Specialist

    2 days ago

    Palo Alto, California, United States Rivian Full time

    Rivian is a leading electric vehicle manufacturer on a mission to keep the world adventurous forever. We are seeking highly motivated and talented students to join our team as interns in data science and engineering.Role SummaryThis internship is an exciting opportunity for students to gain hands-on experience in data analysis, machine learning, and software...

  • Data Science Lead Specialist

    2 days ago

    Palo Alto, California, United States HP Development Company, L.P. Full time

    **Job Title:** Data Science Lead Specialist**Location:** United States of America**Overview:HP Development Company, L.P. is a leading provider of innovative technology solutions. We are seeking an experienced Data Science Lead Specialist to join our team.**Job Description:** As a Data Science Lead Specialist, you will oversee a team of machine learning...

  • Data Science Specialist in Systems Biology

    3 days ago

    Palo Alto, California, United States SoftWash Systems Full time

    CZ Biohub SF seeks a highly motivated Data Science Specialist in Systems Biology to join our dynamic team of researchers and engineers. In this role, you will work closely with data scientists, software engineers, and experts in genomics and cell biology to develop cutting-edge data science solutions for systems biology.Key Responsibilities:Develop and...

  • Data Science Engineer

    3 days ago

    Palo Alto, California, United States Mistral AI Full time

    About Mistral AI">Mistral AI is a leading innovator in the field of artificial intelligence, dedicated to delivering cutting-edge solutions that meet and exceed client expectations. ">Job Summary">We are seeking an exceptional Data Science Engineer to join our team. As a key member of our Applied AI Engineering team, you will work closely with customers from...

  • Cyber Security Specialist

    3 weeks ago

    Palo Alto, California, United States InDepth Engineering Solutions, LLC Full time

    At InDepth Engineering Solutions, LLC, we are seeking a highly skilled Cyber Security Specialist to join our team.About the RoleThis is a challenging and rewarding opportunity for an experienced professional to enhance our cybersecurity posture by focusing on tactical data protection operations, automation, and tool development.The successful candidate will...

  • Data Engineering Specialist

    3 weeks ago

    Palo Alto, California, United States xAI Full time

    xAI is revolutionizing the field of artificial intelligence with a mission to create systems that accurately understand the universe and aid humanity in its pursuit of knowledge.About xAIWe are a small, highly motivated team focused on engineering excellence. Our organization is for individuals who appreciate challenging themselves and thrive on curiosity....

  • Cloud Data Engineer

    3 days ago

    Palo Alto, California, United States Navan Full time

    Job OverviewWe are seeking a highly skilled Cloud Data Engineer to join our team at Navan in Palo Alto, CA. As a key member of our data organization, you will play a critical role in modernizing and scaling our data warehouse to power our data-driven organization.About the Role:You will work closely with our data analyst, product, and engineering teams to...

  • Fleet Data Engineering Specialist

    1 week ago

    Palo Alto, California, United States Tesla Full time

    Role Description">We're looking for a talented Vehicle Product Metrics Engineer to join our Fleet Analytics team. In this role, you'll focus on identifying, computing, predicting, and communicating core metrics about our vehicle products. This will involve collaborating with cross-functional teams to drive improvements in these metrics.">Main...

  • Mobility Data Engineer

    3 days ago

    Palo Alto, California, United States Woven by Toyota Full time

    At Woven by Toyota, we're shaping the future of mobility through innovative technologies. As a Senior Software Engineer on our Simulation Data and Analysis team, you'll play a pivotal role in developing the metrics and evaluation frameworks that assess the quality of our autonomy and validation stacks.About the RoleYou'll be responsible for designing and...

  • Electronics Engineer Specialist

    3 weeks ago

    Palo Alto, California, United States Ultimate Staffing Full time

    **About the Job:**We are seeking a skilled Electronics Engineer Specialist to join our team in Palo Alto, CA. This is a full-time contract position with competitive pay.**Estimated Salary:** $32-$37 per hour.**Job Summary:**The Electronics Engineer Specialist will be responsible for conducting functional and environmental tests on electronic systems and...

  • Data Engineer at Tesla

    2 days ago

    Palo Alto, California, United States Tesla Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Tesla. As a Data Engineer, you will be responsible for designing and developing scalable distributed systems that can handle large amounts of data.Our ideal candidate has 5+ years of software development experience and expertise in languages such as Rust, Java, Scala, Python, or...

  • Cloud Data Architect

    3 weeks ago

    Palo Alto, California, United States Prophecy Full time

    Transformative Role in Cloud Data EngineeringWe are seeking a highly skilled Cloud Data Architect - Data Pipeline Optimization Specialist to join our team at Prophecy. This role offers an unparalleled opportunity to leverage your expertise in cloud data engineering, driving innovation and efficiency in our cutting-edge data pipelines.About ProphecyProphecy...

  • Data Engineer

    2 days ago

    Palo Alto, California, United States Tesla Full time

    **Job Overview**We are seeking a skilled Data Engineer to drive the development of data pipelines and analytics tooling to support the growth of our energy business.ResponsibilitiesDevelop an overall strategy for data management within our established information architecture.Gather and process raw, structured, semi-structured, and unstructured data using...

  • Data-Driven Innovations Specialist

    3 days ago

    Palo Alto, California, United States Tesla Full time

    Tesla is seeking a talented Data-Driven Innovations Specialist to join our team. As a key member of our organization, you will play a critical role in developing cutting-edge AI solutions that drive business growth and transformation.SalaryThe estimated annual salary for this position is $140,000 - $300,000, plus a comprehensive benefits package and...

  • Data Security Engineer

    3 days ago

    Palo Alto, California, United States Palantir Technologies Full time

    About Palantir TechnologiesPalantir builds the world's leading software for data-driven decisions and operations. We empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more. Our products support some of the most important and impactful work in the world.The RoleWe are trusted by our customers to...

  • Systems Biologist and Data Engineer

    3 days ago

    Palo Alto, California, United States SoftWash Systems Full time

    We are seeking a skilled Systems Biologist and Data Engineer to join our team at CZ Biohub SF. In this role, you will develop and apply advanced data science and engineering skills to drive innovative research and discovery in systems biology.Key Responsibilities:Design and implement high-performance data processing pipelines for large-scale biological...

  • Senior Data Engineer Lead

    3 weeks ago

    Palo Alto, California, United States Machinify, Inc. Full time

    Machinify, Inc. is revolutionizing the healthcare industry with its AI-powered software products that transform claims and payment operations.About MachinifyWe are seeking a highly skilled Senior Data Engineer Lead to join our team and help us achieve our mission.Salary RangeThe estimated base salary for this position is $220k-$250k per year, which is...