Cloud Data Engineer

2 weeks ago

San Francisco, United States Unreal Gigs Full time

Are you passionate about building and optimizing data pipelines that drive data-driven decision-making in cloud environments? Do you have the technical expertise to design robust data architectures that manage large-scale data processing and ensure seamless data flow? If you’re ready to harness the power of cloud technology to create scalable data solutions, our client has the ideal role for you. We’re seeking a Cloud Data Engineer (aka The Data Pipeline Architect) to design, develop, and manage cloud-based data infrastructures that support analytical and operational needs across the organization.

As a Cloud Data Engineer at our client, you’ll collaborate with data scientists, analysts, and software engineers to build data pipelines and storage solutions that are secure, efficient, and scalable. Your role will be vital in ensuring that data systems are optimized for performance and capable of supporting a range of data-driven initiatives.

Key Responsibilities:

  1. Design and Build Data Pipelines: Create and manage scalable data pipelines that support ETL (Extract, Transform, Load) processes. You’ll develop automated solutions to handle data ingestion, transformation, and integration in cloud platforms such as AWS, GCP, or Azure.
  2. Develop and Maintain Data Architectures: Architect and maintain cloud-based data storage solutions, such as data lakes and warehouses, ensuring they align with business needs and best practices. You’ll optimize data structures for performance and reliability.
  3. Collaborate on Data Strategy and Solutions: Work closely with data scientists, business analysts, and application developers to understand data requirements and implement solutions that support analytics and machine learning models.
  4. Ensure Data Security and Compliance: Implement and monitor security measures that protect data privacy and comply with industry standards (e.g., GDPR, HIPAA). You’ll manage data access controls and audit logs to safeguard sensitive information.
  5. Optimize Data Processing Workflows: Improve the performance of data processing workflows by optimizing resource allocation and parallel processing techniques. You’ll work to reduce latency and enhance the scalability of data systems.
  6. Automate and Streamline Data Operations: Develop automation scripts and tools using languages such as Python or Java to streamline data processes and reduce manual intervention. You’ll contribute to the efficiency and reliability of data operations.
  7. Monitor and Troubleshoot Data Pipelines: Use monitoring tools and techniques to ensure the health and performance of data pipelines. You’ll identify and troubleshoot issues promptly to maintain smooth data flow and system uptime.

Required Skills:

  • Cloud Data Expertise: Strong experience with cloud data platforms (AWS, GCP, Azure) and data processing tools like Apache Spark, Apache Beam, or AWS Glue. You’re proficient in building scalable data solutions.
  • Programming and Scripting: Proficiency in Python, Java, or Scala for data manipulation and pipeline automation. You can create scripts that handle data transformation and integration tasks.
  • ETL and Data Pipeline Management: Experience in developing and managing complex ETL workflows that handle large data volumes. You’re skilled at building data pipelines that support real-time and batch processing.
  • Data Security and Compliance Knowledge: Familiarity with implementing data security measures and ensuring compliance with industry standards. You can configure access controls and monitor data usage.
  • Collaboration and Problem-Solving: Strong ability to work with cross-functional teams and troubleshoot data-related issues. You can clearly communicate technical concepts and solutions.

Educational Requirements:

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field. Equivalent experience in data engineering and cloud environments may be considered.
  • Certifications such as Google Professional Data Engineer, AWS Certified Big Data – Specialty, or Microsoft Certified: Azure Data Engineer Associate are highly desirable.

Experience Requirements:

  • 5+ years of experience in data engineering, with at least 3+ years focusing on cloud-based data solutions.
  • Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow, Dataflow, Step Functions).
  • Familiarity with database technologies such as SQL, NoSQL, and data warehousing solutions (e.g., BigQuery, Redshift, Snowflake).
#J-18808-Ljbffr

  • Cloud Data Engineer

    6 months ago

    San Francisco, United States Unreal Gigs Full time

    Company Overview: Welcome to the forefront of cloud data engineering! At our company, we're dedicated to leveraging the power of cloud technology to drive data-driven innovation. Our mission is to develop scalable and robust cloud data solutions that enable organizations to harness the full potential of their data. Join us and be part of a dynamic team...

  • Lead Cloud Data Engineer

    6 months ago

    San Francisco, United States Unreal Gigs Full time

    Company Overview: Welcome to the forefront of data-driven innovation! Our company is a trailblazer in leveraging the power of data to drive transformative change and solve complex problems across industries. We're committed to building cutting-edge cloud data solutions that harness the scalability, flexibility, and power of cloud platforms. Join us and lead...

  • Senior Cloud Data Engineer

    4 weeks ago

    San Francisco, California, United States DigitalOcean Full time

    Cloud Data EngineerAt DigitalOcean, we're simplifying cloud computing to empower builders to create software that changes the world. As a Cloud Data Engineer, you'll play a key role in designing and implementing data models and architectures using dbt that support efficient data storage, retrieval, availability, analysis, and BI reporting. You'll work...

  • Cloud Data Engineer

    4 days ago

    San Francisco, United States ZipRecruiter Full time

    Job DescriptionAre you passionate about building and optimizing data pipelines that drive data-driven decision-making in cloud environments? Do you have the technical expertise to design robust data architectures that manage large-scale data processing and ensure seamless data flow? If you’re ready to harness the power of cloud technology to create...

  • Cloud Data Engineer

    4 weeks ago

    San Francisco, California, United States Varite Full time

    Job SummaryVarite is seeking a highly skilled Cloud Data Engineer to join our team. The ideal candidate will have expertise in developing and analyzing complex SQL on various RDBMS, as well as experience with ETL tools and AWS Cloud Data Warehousing Technologies.Key Responsibilities Contributes to the design, development, testing, implementation, and review...

  • Sr/Data Engineer

    2 weeks ago

    San Francisco, United States Federal Reserve Bank of New York Full time

    CompanyFederal Reserve Bank of San Francisco We are the Federal Reserve Bank of San Francisco—public servants with a mission to advance the nation’s monetary, financial, and payment systems to build a stronger economy for all Americans. We are a community-engaged bank, and are committed to understanding and serving the vibrant, expansive communities of...

  • Cloud Data Engineer

    1 month ago

    San Francisco, California, United States Saxon Global Full time

    Job Title: Cloud Data Engineer - Pharmaceutical IndustryWe are seeking a highly skilled Cloud Data Engineer to join our team at Saxon Global in the pharmaceutical industry. The successful candidate will have a strong background in data engineering, with expertise in data ingestion, ETL, analytics, and other data-related services, specifically using AWS...

  • Cloud Data Engineer Position

    4 weeks ago

    San Francisco, California, United States Stefanini Full time

    Job DetailsStefanini is seeking a skilled Cloud Data Engineer to join our team in San Francisco, CA. As a Cloud Data Engineer, you will contribute to the design, development, and implementation of complex data warehouse and business intelligence solutions.Key ResponsibilitiesDesign and develop complex data warehouse applicationsDevelop software from...

  • Cloud Data Engineer

    4 weeks ago

    San Francisco, California, United States Software Guidance & Assistance Full time

    Job Title: Cloud Data Engineer - Data Warehouse SolutionsJob Summary:Software Guidance & Assistance, Inc. is seeking a Cloud Data Engineer to design, develop, and implement data warehouse solutions using AWS Cloud Data Warehousing Technologies. The ideal candidate will have expertise in developing and analyzing complex SQL, data modeling, and ETL tools, as...

  • Cloud Data Solutions Architect

    4 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Cloud Data Engineer Role at Unreal GigsAt Unreal Gigs, we're seeking a skilled Cloud Data Engineer to join our team. As a Cloud Data Engineer, you'll play a pivotal role in designing, building, and maintaining our cloud-based data infrastructure and systems.Key Responsibilities:Cloud Data Infrastructure Design:Design and architect scalable and resilient...

  • Data Engineer

    4 weeks ago

    San Francisco, California, United States Figma Full time

    About the RoleFigma is seeking an experienced Data Engineer to partner with our Data Science and Data Infrastructure teams to own and scale our data pipelines. As a Data Engineer at Figma, you will play a critical role in building the metrics and self-serve reporting capabilities to unlock Figma's next phase of growth.Key Responsibilities- Own, build, and...

  • Cloud Data Engineer

    4 weeks ago

    San Francisco, California, United States Stefanini Group Full time

    Job Summary:Stefanini Group is seeking a skilled Cloud Data Engineer to join our team. As a Cloud Data Engineer, you will be responsible for designing, developing, and implementing complex data warehouse and business intelligence solutions using AWS Cloud Data Warehousing Technologies.Key Responsibilities:• Contribute to the design, development, testing,...

  • Senior Cloud Data Engineer

    4 weeks ago

    San Antonio, Texas, United States Rackspace Technology, Inc. Full time

    About the Role:We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have expertise in HADOOP, Big Data, and Data Engineer, with a strong focus on Processing and Technology.Key Responsibilities:Design and implement cloud data services and solutions.Collaborate with cross-functional teams...

  • Senior Data Engineer

    4 weeks ago

    San Francisco, California, United States Akraya Full time

    Job Summary:We are seeking a seasoned Data Engineer with a robust background in programming, SQL, and cloud services to drive our data infrastructure forward.The ideal candidate will have a deep understanding of the marketing domain, data warehousing, and analytics, and will thrive in a hybrid work environment.This role involves designing scalable web and...

  • Senior Cloud Engineer

    1 month ago

    San Francisco, California, United States Saxon Global Full time

    Job SummaryAs a Senior Cloud Engineer at Saxon Global, you will be responsible for designing, implementing, and managing cloud-based infrastructure and data platforms. Your expertise in cloud services, automation, and data processing will enable you to create scalable and secure solutions for our clients.Key Responsibilities:* Design and implement...

  • GCP Data Engineer

    4 weeks ago

    San Francisco, California, United States Tekfortune Inc Full time

    Job Description:At Tekfortune Inc, we are seeking a highly skilled GCP Data Engineer - Cloud Computing Expert to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems on Google Cloud Platform.Key Responsibilities: Provide technical guidance to a team of...

  • Hybrid Cloud Engineer

    2 weeks ago

    San Francisco, United States Unreal Gigs Full time

    Are you an expert in designing and managing hybrid cloud environments that seamlessly integrate on-premises infrastructure with public cloud services? Do you have the technical skills and strategic vision to create solutions that optimize performance, security, and scalability across multiple platforms? If you’re ready to bridge the gap between on-premises...

  • Cloud Engineer

    2 weeks ago

    San Francisco, United States Cornerstone Research Full time

    Overview CLOUD ENGINEER (HYBRID, ANY US OFFICE) If you are a technical professional looking for an opportunity to showcase your exceptional cloud expertise, then we would like to meet with you! The Cloud Engineer designs, deploys, manages, and supports cloud solutions running on AWS, Azure, and Google cloud platforms. Working with various functions in the IT...

  • Senior Data Engineer

    4 weeks ago

    San Antonio, Texas, United States Omni Inclusive Full time

    Job SummaryWe are seeking a highly skilled Senior Data Engineer - Cloud Solutions to join our team at Omni Inclusive. As a key member of our data engineering team, you will be responsible for designing, developing, and delivering cloud-based data solutions that meet the needs of our clients.Key ResponsibilitiesDesign and implement data warehousing concepts,...

  • Principal Platform Engineer

    2 weeks ago

    San Francisco, United States Autodesk Full time

    Job Requisition ID # 24WD77185Autodesk's Platform Services and Emerging Technology (PSET) team is hiring a Principal Platform Engineer with experience automating large-scale cloud infrastructure services. In this exciting role, you will help us automate a robust and scalable data platform used by numerous teams across the company. Reporting to our Software...