Big Data Software Engineer

2 days ago


Palo Alto, California, United States Predactiv Full time
About Us

Predactiv, a company that owns online behavior data of 1 billion+ users globally, is developing an audience intelligence platform with cutting-edge big data technologies.

We are seeking an innovative software engineer to join our exciting projects and platform development.

Job Description

As a big data software engineer, you will be responsible for designing and developing scalable and efficient big data software and platform architecture.

You will work with our team to build large-scale data processing pipelines, leveraging technologies such as Spark and AWS EMR.

Requirements
  • 5+ years of experience in big data software and platform development
  • Fluent in two or more programming languages such as Python, Scala, Java, GoLang
  • Knowledge of microservices & serverless architectures
  • Experience building large CI/CD pipelines
  • Hands-on experience with Spark and AWS EMR
  • Strong experience with AWS (S3, SQS, EMR etc)
  • Strong automation skills with Python and/or Bash
  • Experience working in fast-paced Agile teams
  • Ability to learn new tools and workflows with little or no supervision
  • Education: BS/MS in Computer Science or Software Engineering
Nice to Haves
  • Experience with GCS
  • Experience with SQL/NoSQL Database
  • Experience with Databricks is a plus
  • Working knowledge of Security and Privacy in big data platform
  • Experience with Azure, Snowflake is a plus
  • Experience with Airflow


  • Palo Alto, California, United States Kohler Full time

    About the RoleWe are seeking a highly skilled Software Engineering Manager to lead our cloud data systems team at Kohler Ventures. As a key member of our technology team, you will be responsible for designing and implementing data systems that adhere to modern high-throughput architectures while prioritizing privacy.Key ResponsibilitiesGuide the team in...


  • Palo Alto, California, United States Kohler Full time

    About the RoleWe are seeking a highly skilled Software Engineering Manager to join our team at Kohler Ventures. As a key member of our software engineering team, you will be responsible for guiding the design and implementation of data systems that adhere to modern high-throughput architectures while prioritizing privacy.Key ResponsibilitiesStrategic...


  • Palo Alto, California, United States Kohler Full time

    Software Engineering ManagerKohler Ventures is seeking a highly skilled Software Engineering Manager to lead our cloud engineering team. As a key member of our organization, you will be responsible for designing and implementing data systems that adhere to modern high throughput architectures while prioritizing privacy.Key Responsibilities:Guide the team in...

  • Software Engineer

    7 days ago


    Palo Alto, California, United States Palantir Technologies Full time

    About Palantir TechnologiesPalantir Technologies is a world-changing company that builds software for data-driven decisions and operations. Our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.The RoleWe're seeking a skilled Software Engineer to join our Developer Productivity...


  • Palo Alto, California, United States General Motors Full time

    About the RoleWe are seeking a highly skilled Site Reliability Engineer to join our team at General Motors. As a Site Reliability Engineer, you will play a critical role in ensuring the stability, scalability, and reliability of our software-defined vehicle solutions.Key ResponsibilitiesLead the Site Reliability engineering effort to improve anomaly...

  • Reliability Engineer

    3 weeks ago


    Palo Alto, California, United States General Motors Full time

    About the RoleWe are seeking a highly skilled Site Reliability Engineer to join our team at General Motors. As a Site Reliability Engineer, you will play a critical role in ensuring the stability, scalability, and reliability of our software-defined vehicle solutions.Key ResponsibilitiesLead the Site Reliability engineering effort to improve anomaly...


  • Palo Alto, California, United States Amazon Full time

    **About the Role**We are seeking a highly skilled Senior Software Engineer to join our Amazon Redshift team. As a Senior Software Engineer, you will be responsible for designing, developing, and deploying large-scale distributed systems that power our cloud data warehouse service.**Key Responsibilities**Design and implement scalable and highly available...


  • Palo Alto, California, United States Amazon Full time

    **About the Role**Amazon is seeking a highly skilled Senior Software Engineer to join our Cloud Data Warehouse team. As a key member of our team, you will be responsible for designing, developing, and deploying large-scale distributed systems that power our cloud-based data warehouse service.**Key Responsibilities**Design and implement scalable and highly...


  • Palo Alto, California, United States SA TECHNOLOGIES Full time

    Company OverviewSA Technologies is committed to delivering exceptional software solutions tailored to meet the demands of our clients.Position OverviewRole: Senior Software EngineerDuration: 6 Months+The Data Computing Appliance and Platform Software team plays a pivotal role in crafting the software that deploys, manages, monitors, and maintains both the...


  • Palo Alto, California, United States Kohler Full time

    Position: Software Engineering ManagerWork Mode: HybridOverview:Kohler Ventures, a subsidiary of Kohler Co., a prominent name in kitchen and bath products, tile, and home interiors, is on a mission to innovate digital solutions that promote healthier lifestyles. We are committed to enhancing everyday experiences through cutting-edge technology, design, and...


  • Palo Alto, California, United States Palantir Technologies Full time

    About Palantir TechnologiesPalantir Technologies is a world-changing company that builds software for data-driven decisions and operations. Our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.The RoleAs a Backend Software Engineer at Palantir, you will build software at scale to...


  • Palo Alto, California, United States Palantir Technologies Full time

    About the RolePalantir Technologies is seeking a skilled Backend Software Engineer to join our team. As a Backend Software Engineer, you will play a critical role in building software at scale to transform how organizations use data.Key ResponsibilitiesDesign and develop high-performance search and indexing ecosystems for complex, granularly permissioned...


  • Palo Alto, California, United States SA TECHNOLOGIES Full time

    Company Overview: SA Technologies is dedicated to providing exceptional technology solutions tailored to meet the needs of our clients.Position OverviewRole: Senior Software EngineerContract Duration: 6 Months+The Software Development team is tasked with creating and managing the software that oversees the deployment, monitoring, and maintenance of our...


  • Palo Alto, California, United States Ganymede Full time

    Job DescriptionGanymede is building a cutting-edge platform for the sciences, with a mission of improving the world through better healthcare outcomes. We believe that the highest leverage approach to speed up science is through better software and data connectivity.As a Scientific Software Engineer, you'll work directly with scientists and engineers to...


  • Palo Alto, California, United States Luma AI Full time

    Job SummaryWe are seeking a highly skilled Senior Software Engineer to join our Data Infrastructure team at Luma AI. As a key member of our applied research team, you will be responsible for designing, building, and automating infrastructure for processing large-scale data across multiple clusters of thousands of GPUs.Key ResponsibilitiesDesign and implement...


  • Palo Alto, California, United States Palantir Technologies Full time

    About Palantir TechnologiesPalantir Technologies is a world-changing company that builds software for data-driven decisions and operations. Our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.The RoleAs a Backend Software Engineer at Palantir, you will build software at scale to...


  • Palo Alto, California, United States Palantir Technologies Full time

    About the RoleWe are seeking a highly skilled Backend Software Engineer to join our team at Palantir Technologies. As a key member of our infrastructure team, you will be responsible for designing and developing scalable, secure, and high-performance software systems that power our products.Key ResponsibilitiesDesign and implement performant search and...


  • Palo Alto, California, United States SA TECHNOLOGIES Full time

    Company OverviewSA Technologies is dedicated to delivering exceptional IT solutions tailored to our clients' needs. We pride ourselves on our commitment to quality and innovation.Position OverviewTitle: Senior Software EngineerDuration: 6 Months+The Data Computing Appliance and Platform Software team is tasked with the development of software that deploys,...


  • Palo Alto, California, United States Palantir Technologies Full time

    About Palantir TechnologiesPalantir Technologies is a world-changing company that builds software for data-driven decisions and operations. Our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more.The RoleAs a Backend Software Engineer at Palantir, you will build software at scale to...

  • Senior Data Engineer

    2 weeks ago


    Palo Alto, California, United States Penn State University Talent Acquisition Full time

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Penn State University Talent Acquisition. As a Senior Data Engineer, you will play a pivotal role in designing, developing, and maintaining our data ecosystem.Responsibilities:Collaborate with cross-functional teams to support data-driven...