Cloud Data Engineer

1 month ago

San Francisco, United States ZipRecruiter Full time

Job Description

Are you passionate about building and optimizing data pipelines that drive data-driven decision-making in cloud environments? Do you have the technical expertise to design robust data architectures that manage large-scale data processing and ensure seamless data flow? If you’re ready to harness the power of cloud technology to create scalable data solutions, our client has the ideal role for you. We’re seeking a Cloud Data Engineer (aka The Data Pipeline Architect) to design, develop, and manage cloud-based data infrastructures that support analytical and operational needs across the organization.

As a Cloud Data Engineer at our client, you’ll collaborate with data scientists, analysts, and software engineers to build data pipelines and storage solutions that are secure, efficient, and scalable. Your role will be vital in ensuring that data systems are optimized for performance and capable of supporting a range of data-driven initiatives.

Key Responsibilities:

  1. Design and Build Data Pipelines: Create and manage scalable data pipelines that support ETL (Extract, Transform, Load) processes. You’ll develop automated solutions to handle data ingestion, transformation, and integration in cloud platforms such as AWS, GCP, or Azure.
  2. Develop and Maintain Data Architectures: Architect and maintain cloud-based data storage solutions, such as data lakes and warehouses, ensuring they align with business needs and best practices. You’ll optimize data structures for performance and reliability.
  3. Collaborate on Data Strategy and Solutions: Work closely with data scientists, business analysts, and application developers to understand data requirements and implement solutions that support analytics and machine learning models.
  4. Ensure Data Security and Compliance: Implement and monitor security measures that protect data privacy and comply with industry standards (e.g., GDPR, HIPAA). You’ll manage data access controls and audit logs to safeguard sensitive information.
  5. Optimize Data Processing Workflows: Improve the performance of data processing workflows by optimizing resource allocation and parallel processing techniques. You’ll work to reduce latency and enhance the scalability of data systems.
  6. Automate and Streamline Data Operations: Develop automation scripts and tools using Python or Java to streamline data processes and reduce manual intervention. You’ll contribute to the efficiency and reliability of data operations.
  7. Monitor and Troubleshoot Data Pipelines: Use monitoring tools and techniques to ensure the health and performance of data pipelines. You’ll identify and troubleshoot issues promptly to maintain smooth data flow and system uptime.

Requirements

Required Skills:

  • Cloud Data Expertise: Strong experience with cloud data platforms (AWS, GCP, Azure) and data processing tools like Apache Spark, Apache Beam, or AWS Glue. You’re proficient in building scalable data solutions.
  • Programming and Scripting: Proficiency in Python, Java, or Scala for data manipulation and pipeline automation. You can create scripts that handle data transformation and integration tasks.
  • ETL and Data Pipeline Management: Experience in developing and managing complex ETL workflows that handle large data volumes. You’re skilled at building data pipelines that support real-time and batch processing.
  • Data Security and Compliance Knowledge: Familiarity with implementing data security measures and ensuring compliance with industry standards. You can configure access controls and monitor data usage.
  • Collaboration and Problem-Solving: Strong ability to work with cross-functional teams and troubleshoot data-related issues. You can clearly communicate technical concepts and solutions.

Educational Requirements:

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field. Equivalent experience in data engineering and cloud environments may be considered.
  • Certifications such as Google Professional Data Engineer, AWS Certified Big Data – Specialty, or Microsoft Certified: Azure Data Engineer Associate are highly desirable.

Experience Requirements:

  • 5+ years of experience in data engineering, with at least 3+ years focusing on cloud-based data solutions.
  • Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow, Dataflow, Step Functions).
  • Familiarity with database technologies such as SQL, NoSQL, and data warehousing solutions (e.g., BigQuery, Redshift, Snowflake).

Benefits

  • Health and Wellness: Comprehensive medical, dental, and vision insurance plans with low co-pays and premiums.
  • Paid Time Off: Competitive vacation, sick leave, and 20 paid holidays per year.
  • Work-Life Balance: Flexible work schedules and telecommuting options.
  • Professional Development: Opportunities for training, certification reimbursement, and career advancement programs.
  • Wellness Programs: Access to wellness programs, including gym memberships, health screenings, and mental health resources.
  • Life and Insurance: Life insurance and short-term/long-term coverage.
  • Employee Assistance Program (EAP): Confidential counseling and support services for personal and professional challenges.
  • Tuition Reimbursement: Financial assistance for continuing education and professional development.
  • Community Engagement: Opportunities to participate in community service and volunteer activities.
  • Recognition Programs: Employee recognition programs to celebrate achievements and milestones.
#J-18808-Ljbffr

  • Cloud Data Engineer

    2 weeks ago

    San Francisco, California, United States eTeam Inc. Full time

    eTeam Inc. is seeking a highly skilled Cloud Data Engineer to join our team. As a key member of our cloud data engineering team, you will be responsible for designing, developing, and managing large-scale data pipelines and workflows using Trino SQL/Spark SQL warehoused in HDFS datasets in a cloud environment.We offer a competitive salary of $110,000 -...

  • Cloud Data Engineer

    1 week ago

    San Francisco, California, United States Syntricate Technologies Full time

    Job Title: Cloud Data EngineerAbout Syntricate TechnologiesWe are a rapidly growing company that specializes in data solutions, and we're looking for a skilled Cloud Data Engineer to join our team. As a Cloud Data Engineer, you will play a key role in designing and implementing data architectures on the cloud using Snowflake. You'll work closely with our...

  • Cloud Data Engineering Specialist

    2 weeks ago

    San Francisco, California, United States Diverse Lynx Full time

    About the Role:We are looking for a highly motivated and experienced Cloud Data Engineering Specialist to join our team at Diverse Lynx LLC. As a key member of our cloud engineering team, you will be responsible for designing, building, and maintaining scalable and efficient cloud-based data platforms using various cloud data engineering technologies such as...

  • Cloud Data Engineer

    2 months ago

    San Francisco, United States Unreal Gigs Full time

    Are you passionate about building and optimizing data pipelines that drive data-driven decision-making in cloud environments? Do you have the technical expertise to design robust data architectures that manage large-scale data processing and ensure seamless data flow? If you’re ready to harness the power of cloud technology to create scalable data...

  • Data Engineer for Cloud Platforms

    1 week ago

    San Francisco, California, United States Early Warning Services LLC Full time

    Early Warning Services LLC is a leading provider of cutting-edge solutions for the U.S. financial system, empowering institutions to increase access to financial services and protect transactions for millions of consumers and small businesses.The company's hybrid work model allows for a collaborative working environment, with positions located in Scottsdale,...

  • Cloud Data Analytics Engineer

    4 weeks ago

    San Francisco, California, United States Tbwa ChiatDay Inc Full time

    About the Role:We are seeking an experienced Cloud Data Analytics Engineer to join our team. As a key member of our engineering group, you will be responsible for designing and building scalable data analytics solutions using cloud-based technologies.The ideal candidate will have a strong background in computer science, experience with cloud platforms such...

  • Data Engineer for Cloud Data Platforms

    2 weeks ago

    San Francisco, California, United States ZipRecruiter Full time

    At ZipRecruiter, we are seeking a highly skilled Data Engineer for Cloud Data Platforms to join our team. This role will be responsible for designing and building scalable data pipelines that support ETL processes in cloud platforms such as AWS, GCP, or Azure.With a strong background in cloud data engineering and experience with data pipeline orchestration...

  • Cloud Data Engineer

    2 weeks ago

    San Francisco, United States ZipRecruiter Full time

    Job DescriptionJob Description Are you passionate about building and optimizing data pipelines that drive data-driven decision-making in cloud environments? Do you have the technical expertise to design robust data architectures that manage large-scale data processing and ensure seamless data flow? If you’re ready to harness the power of cloud technology...

  • Cloud and Big Data Engineer

    2 weeks ago

    San Francisco, California, United States Eventual Computing Full time

    We are looking for a skilled Cloud and Big Data Engineer to join our team at Eventual Computing. You will be working on core features of our open-sourced data engine Daft across the entire stack from the Daft frontend API and dashboard to its distributed scheduler and low-level Rust execution engine.Your responsibilities will include tinkering with...

  • Cloud Engineer

    2 weeks ago

    San Francisco, California, United States WEX Full time

    Your Role at WEXWe're seeking a skilled Cloud Engineer to join our Data, AI, and Risk Technology (DART) organization. As a key member of this dynamic team, you'll harness the power of cloud technologies to deliver scalable, secure, and user-friendly platforms and tools for data, AI, and risk applications.

  • Data Engineer

    1 week ago

    San Francisco, California, United States MongoDB Full time

    MongoDB empowers innovators to create, transform, and disrupt industries by unleashing the power of software and data. Our industry-leading developer data platform, MongoDB Atlas, is a globally distributed, multi-cloud database available in over 115 regions across AWS, Google Cloud, and Microsoft Azure. We enable organizations of all sizes to easily build,...

  • Cloud Data Engineering Specialist

    2 weeks ago

    San Francisco, California, United States SS&C Technologies Full time

    SS&C Technologies is a leading global provider of investment and financial services and software for the financial services and healthcare industries. With a strong presence in over 90 offices in 35 countries, our company has been named to the Fortune 1000 list as a top U.S. company based on revenue. As a Senior Data Engineer at SS&C, you will be part of a...

  • Big Cloud | Machine Learning Engineer

    2 days ago

    san francisco, United States Big Cloud Full time

    Are you passionate about building generative AI products with state-of-the art technologies?Do you thrive in an environment that combines research with hands-on product-building?Ready to work with a leading fintech platform for Silicon Valley?We’re working with a 40-person Series A start-up that’s transforming how startups manage their capital and...

  • Cloud Data Engineering Specialist

    1 week ago

    San Francisco, California, United States CyberTec Full time

    Job Summary:CyberTec is seeking an experienced Cloud Data Engineering Specialist to join our team. In this role, you will be responsible for designing and developing cloud-based data pipelines using Kafka, DynamoDB, and related AWS technologies.The ideal candidate will have more than 7 years of experience in owning the design and development activities of...

  • Cloud Data Engineer Manager

    5 days ago

    San Francisco, California, United States Cruise Full time

    About the JobWe are looking for an experienced Cloud Data Engineer Manager to lead our data engineering team at Cruise. As a manager, you will be responsible for overseeing the design, development, and implementation of our data platform on GCP. You will work closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • Cloud Data Engineer

    1 week ago

    San Francisco, California, United States Parabola Full time

    Company OverviewAt Parabola, we're revolutionizing the way businesses manage their data. Our unique flow-based calculation engine enables high-throughput and availability, making us a leader in the industry. We're proud to serve companies like Flexport, Sonos, Uber Freight, Brooklinen, and Chubbies, and backed by top investors such as OpenView Partners,...

  • Data Cloud Engineer

    2 weeks ago

    San Diego, California, United States Apple Full time

    About the RoleWe're looking for a highly motivated Software Engineer to join our Data Cloud team and contribute to the development of our analytics platform. As a Senior Software Engineer, you'll work closely with cross-functional teams to design and implement scalable data systems and cloud services that drive business value.ResponsibilitiesDesign and...

  • Senior Cloud Data Engineer

    3 weeks ago

    San Francisco, California, United States Instrinsic Full time

    Intrinsic is at the forefront of harnessing innovative technologies to address some of the digital age's most complex problems. As a Senior Data Engineer, you'll have the opportunity to contribute to the development of our data platform, which enables Trust & Safety teams to focus on high-priority tasks and safeguard their communities from abuse.You'll be...

  • Cloud Data Engineer

    2 weeks ago

    San Francisco Bay Area, United States Compunnel Inc. Full time

    Job: Cloud Data EngineerLocation: San Francisco CA - Hybrid (2+ days in office) Interview process: Phone/teams/Webex interview with HM & 2nd round: Panel interviewTop Must have skills:Min 10 years of relevant experience and 5+ with AWS CloudNeed strong on SQL and cloud and able to build data pipeline.Migration to Cloud from SQL server.hands-on exp in SQL and...

  • Cloud-Based Data Engineering Lead

    2 weeks ago

    San Francisco, California, United States WEX Full time

    Job SummaryWe are seeking a Cloud-Based Data Engineering Lead to join our team at WEX. As a key member of our Data Engineering organization, you will be responsible for designing, developing, and implementing scalable data architectures and systems.Your primary focus will be on leading the technical architecture and development of our unified, scalable data...