Big Data Engineer

1 day ago

San Francisco, California, United States Unreal Gigs Full time
Unreal Gigs

Company Overview:

Welcome to Unreal Gigs, a leading innovator in harnessing the power of big data to drive insights and innovation. Our mission is to develop scalable and robust big data solutions that empower organizations to extract value from their data at scale.

Position Overview:

As a Big Data Engineer at Unreal Gigs, you'll play a pivotal role in designing, building, and maintaining our big data infrastructure and platforms. Working closely with cross-functional teams of data scientists, analysts, and software engineers, you'll ensure the reliability, scalability, and efficiency of our big data pipelines and systems.

Key Responsibilities:
  • Big Data Infrastructure Design: Design and architect scalable and reliable big data infrastructure and platforms, including data lakes, data warehouses, and streaming data pipelines.
  • Data Ingestion and Processing: Develop and maintain data ingestion pipelines to collect and process large volumes of data from diverse sources, ensuring data quality and integrity.
  • Data Storage and Management: Implement and manage big data storage solutions, including distributed file systems (e.g., HDFS), NoSQL databases (e.g., Cassandra, MongoDB), and cloud storage services (e.g., Amazon S3, Google Cloud Storage).
  • Data Transformation and ETL: Design and implement data transformation and ETL (Extract, Transform, Load) processes to standardize, cleanse, and enrich data for analysis and modeling.
  • Data Querying and Analysis: Develop and optimize data querying and analysis tools and frameworks, enabling data scientists and analysts to extract insights from large datasets.
  • Data Governance and Security: Implement data governance policies and security controls to ensure compliance with regulatory requirements and protect sensitive data.
  • Monitoring and Optimization: Monitor big data pipelines and systems for performance, reliability, and efficiency, proactively identifying and addressing issues to minimize downtime and optimize resource utilization.
  • Documentation and Collaboration: Document big data infrastructure and processes, including architecture diagrams, data lineage, and workflow documentation, and collaborate with cross-functional teams to ensure alignment and transparency.
Qualifications:
  • Bachelor's degree or higher in Computer Science, Engineering, or related field.
  • Strong background in big data engineering, with hands-on experience in designing, building, and maintaining big data infrastructure and platforms.
  • Proficiency in programming languages such as Java, Python, or Scala, and experience with big data processing frameworks such as Apache Hadoop, Apache Spark, or Apache Flink.
  • Experience with distributed computing technologies and concepts, including MapReduce, Spark RDDs, and streaming data processing.
  • Familiarity with big data storage solutions and databases, including HDFS, Cassandra, MongoDB, Amazon S3, and Google Cloud Storage.
  • Strong problem-solving abilities and analytical thinking, with a keen attention to detail and a passion for tackling complex technical challenges.
  • Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams and communicate technical concepts to non-technical stakeholders.
Benefits:
  • Competitive salary: The industry standard salary for Big Data Engineers typically ranges from $150,000 to $230,000 per year, depending on experience and qualifications.
  • Comprehensive health, dental, and vision insurance plans.
  • Flexible work hours and remote work options.
  • Generous vacation and paid time off.
  • Professional development opportunities, including access to training programs, conferences, and workshops.
  • State-of-the-art technology environment with access to cutting-edge tools and resources.
  • Vibrant and inclusive company culture with team-building activities and social events.
  • Opportunities for career growth and advancement within the company.
  • Exciting projects with real-world impact across diverse industries.
  • Chance to work alongside top talent and industry experts in the field of big data engineering.

  • Big Data Engineer

    3 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Unlock the Power of Data with Unreal GigsWe're on a mission to harness the transformative power of data and turn it into actionable insights that can change the world. As a Big Data Engineer at Unreal Gigs, you'll play a pivotal role in our journey to navigate, analyze, and innovate in an ocean of bytes.What You'll Be Doing:Design and Deploy Scalable Big...

  • Lead Big Data Engineer

    3 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Title: Lead Big Data EngineerAt Unreal Gigs, we're seeking a highly skilled Lead Big Data Engineer to join our team. As a key member of our data engineering team, you'll be responsible for designing, building, and maintaining our big data infrastructure and pipelines.Key Responsibilities:Technical Leadership: Provide technical guidance and leadership to...

  • Big Data Architect

    23 hours ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Title: Big Data ArchitectAt Unreal Gigs, we're seeking a highly skilled Big Data Architect to join our team. As a key member of our data engineering team, you'll be responsible for designing and building scalable and reliable big data infrastructure and platforms.Key Responsibilities:Design and architect big data infrastructure and platforms, including...

  • Big Data Architect

    13 hours ago

    San Francisco, California, United States Unreal Gigs Full time

    Job OverviewUnreal Gigs is seeking a visionary Big Data Architect to spearhead the design and construction of our cutting-edge data solutions. As a key member of our team, you will be responsible for crafting the blueprint for tomorrow's innovations, leveraging colossal data sets to solve real-world problems.Key Responsibilities:Architect and refine our big...

  • Senior Big Data Architect

    14 hours ago

    San Francisco, California, United States Unreal Gigs Full time

    Unlock the Power of Big DataWelcome to Unreal Gigs, where we're harnessing the potential of big data to drive innovation and solve complex problems. As a Senior Big Data Engineer, you'll play a pivotal role in designing, building, and maintaining our big data infrastructure and pipelines.Key Responsibilities:Big Data Infrastructure Design: Design, architect,...

  • Software Engineer

    3 weeks ago

    San Diego, California, United States ServiceNow Full time

    Job Title: Software Engineer - Big DataAt ServiceNow, we're building the next-generation platform for big data, and we need talented engineers like you to help us make it happen.About the RoleWe're looking for a skilled software engineer to join our Cloud Services Big Data Team. As a member of this team, you'll be responsible for building real-time analytic...

  • Big Data Architect

    11 hours ago

    San Francisco, California, United States Unreal Gigs Full time

    Shape the Future of DataWe're not just collecting data; we're crafting the blueprint for tomorrow's innovations. As a Big Data Architect, you'll spearhead our efforts to design and construct the infrastructure that underpins our cutting-edge data solutions.Your vision will pave the way for breakthroughs that leverage colossal data sets to solve real-world...

  • Big Data Developer

    1 month ago

    San Francisco, California, United States Sigmaways Inc Full time

    About Sigmaways Inc.Sigmaways Inc. is a leading provider of innovative data solutions, and we're seeking a highly skilled Big Data Developer to join our team.Key Responsibilities:Design and Develop Data Solutions: Utilize open-source, modern data lake, and enterprise data warehouse technologies (Hadoop, Spark, Cloud, etc.) to create end-to-end data...

  • Big Data Solutions Architect

    2 days ago

    San Francisco, California, United States Sigmaways Inc Full time

    Job Title: Big Data DeveloperAt Sigmaways Inc, we are seeking a highly skilled Big Data Developer to join our team. As a key member of our data solutions team, you will be responsible for designing, developing, and maintaining end-to-end data solutions using open source, modern data lake, and enterprise data warehouse technologies.Key Responsibilities:Design...

  • Big Data Infrastructure Specialist

    3 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Unlock the Power of Big DataWe're on a mission to harness the potential of colossal data sets to drive real-world innovation. As a Big Data Architect at Unreal Gigs, you'll play a pivotal role in shaping our cutting-edge data solutions. Your expertise will pave the way for breakthroughs that transform industries and improve lives.Your Key...

  • Big Data Architect

    1 day ago

    San Jose, California, United States TEPHRA Full time

    Job DescriptionAt TEPHRA, we are seeking a highly skilled Big Data Engineer-Assurance to join our team.Key Responsibilities:Design and develop scalable big data solutions using HDFS and Apache Spark.Manage the data lake and prepare data for applications, including video and image data.Optimize data storage and enable fast retrieval.Implement metadata...

  • Big Data Developer

    4 weeks ago

    San Francisco, California, United States Sigmaways Inc Full time

    About the RoleWe are seeking a highly skilled Big Data Developer to contribute to the development of high-quality technology solutions that address business needs. As a key member of our team, you will be responsible for designing and implementing large-scale data applications using Big Data technologies.Key ResponsibilitiesDesign and develop data...

  • Big Data Engineer-Assurance Specialist

    12 hours ago

    San Jose, California, United States TEPHRA Full time

    Job Summary: We are seeking a highly skilled Big Data Engineer-Assurance Specialist to join our team at TEPHRA. The ideal candidate will have expertise in designing and building Big Data solutions, including hands-on experience with HDFS, Apache Spark, and Python/Scala programming languages.Key Responsibilities:Manage the data lake and prepare data for all...

  • Lead Data Engineer

    3 weeks ago

    San Francisco, California, United States Unreal Gigs Full time

    Job Title: Lead Big Data EngineerAt Unreal Gigs, we're seeking a highly skilled Lead Big Data Engineer to join our team. As a key member of our data engineering team, you'll be responsible for designing, building, and maintaining our big data infrastructure and pipelines.Key Responsibilities:Lead a team of big data engineers, providing guidance, mentorship,...

  • Software Engineer

    3 weeks ago

    San Diego, California, United States ServiceNow Full time

    About the RoleAt ServiceNow, we're building the next-generation platform for collecting, storing, and providing real-time access to large amounts of data. As a member of our Cloud Services Big Data Team, you'll play a key role in designing and developing the Observability platform, ensuring it's scalable, clean, and reusable.Key ResponsibilitiesDesign and...

  • Senior Data Engineer

    2 weeks ago

    San Francisco, California, United States SS&C Technologies Full time

    Senior Data EngineerSS&C Technologies is a leading provider of investment and financial services and software for the financial services and healthcare industries. As a Senior Data Engineer, you will be part of a team responsible for creating a cutting-edge cloud platform using the latest big data, cloud native, and machine learning technologies.Key...

  • Azure Data Engineer

    4 weeks ago

    San Francisco, California, United States Outdefine Full time

    About the Role:We are seeking a skilled professional to join our dynamic team as an Azure Databricks Engineer. In this role, you will be responsible for designing, developing, and implementing big data solutions using Azure Databricks. You will collaborate with data scientists, analysts, and other stakeholders to ensure efficient data processing,...

  • Senior NLP Engineer

    4 weeks ago

    San Francisco, California, United States Big Cloud Full time

    Unlock the Power of NLPWe're seeking a highly skilled Senior NLP Engineer to join our team at Big Cloud. As a key member of our ML team, you'll be responsible for building and leading the development of state-of-the-art NLP features.With a strong background in deep learning and cloud services, you'll be working on cutting-edge projects that drive innovation...

  • Senior Data Engineer

    2 days ago

    San Francisco, California, United States Webologix Ltd INC Full time

    Role: Senior Data EngineerLocation: Dallas, TX / San Francisco, CA / Seattle, WAType of hiring: FTEJob Overview:We are seeking a highly skilled Senior Data Engineer to join our team at Webologix Ltd/ INC. As a key member of our data engineering team, you will be responsible for designing, developing, and implementing large-scale data engineering solutions on...

  • Senior Data Engineer

    2 weeks ago

    San Francisco, California, United States SS&C Technologies Full time

    Senior Data EngineerSS&C Technologies is a leading provider of investment and financial services and software for the financial services and healthcare industries. As a Senior Data Engineer, you will be part of a team responsible for creating a cutting-edge cloud platform using the latest big data, cloud native, and machine learning technologies.Key...