Software Engineer

3 weeks ago


San Francisco, United States Magic, Inc Full time

Magic’s mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pre-training, domain-specific RL, ultra-long context, and inference-time compute to achieve this goal.

About the role:

As a Software Engineer working on our pretraining data, you write efficient and robust pipelines for giant, multimodal datasets. You will develop and optimize web scraping techniques to harvest and maintain data at internet-scale.

What you might work on:
  • Design & implement multimodal (video, audio, text etc) web crawlers for scraping and indexing petabytes of data

  • Create large scale data processing pipelines using tools like Ray, Apache Spark, Apache Flink, Google BigQuery etc.

  • Implement and scale deduplication techniques across modalities and apply heuristic and model-based techniques for parsing and filtering crawled data

  • Identify new data sources for inclusion in pre/post-training datasets

What we’re looking for:
  • Strong proficiency in distributed computing and parallel processing techniques

  • Obsession with details, reliability, and good testing to ensure data quality and integrity

  • Experience with designing and maintaining high-performance, scalable data architectures

  • Ability to design, develop and operate an LLM data pipeline from web scraping to data loading

Magic strives to be the place where high-potential individuals can do their best work. We value quick learning and grit just as much as skill and experience.

Our culture:
  • Integrity. Words and actions should be aligned

  • Hands-on. At Magic, everyone is building

  • Teamwork. We move as one team, not N individuals

  • Focus. Safely deploy AGI. Everything else is noise

  • Quality. Magic should feel like magic

Compensation, benefits and perks (US):
  • Annual salary range: $100K - $550K

  • Equity is a significant part of total compensation, in addition to salary

  • 401(k) plan with 6% salary matching

  • Generous health, dental and vision insurance for you and your dependents

  • Unlimited paid time off

  • Visa sponsorship and relocation stipend to bring you to SF, if possible

  • A small, fast-paced, highly focused team

#J-18808-Ljbffr
  • Software Engineer

    6 days ago


    San Francisco, United States Sara Software Systems Full time

    bem Share this job San Francisco, CA, USA Full Time Role Highlights Computer Science Integrations Software Systems LLMs AI Tools, Libraries and Frameworks Description The company leverages LLMs to create a new interface between humans and systems to build robust and scalable software solutions. The role involves working directly with founders to...


  • San Francisco, California, United States ONE INC SOFTWARE CORPORATION Full time

    Job OverviewWe are seeking a highly skilled and experienced Senior Software Engineering Team Lead to join our team at ONE INC SOFTWARE CORPORATION. This is an exciting opportunity to lead a team of developers in designing and implementing innovative solutions for the insurance industry.Key ResponsibilitiesLead and manage a global team of software engineers...


  • San Antonio, Texas, United States Captivation Software Full time

    Job Title: Software Engineer Lead PositionWe are seeking a skilled and experienced Software Engineer to join our team at Captivation Software. As a Software Engineer, you will be responsible for designing, developing, and implementing complex software capabilities for computer-based systems.About the Role:You will work closely with other talented engineers...

  • Software Engineer

    2 weeks ago


    San Francisco, United States Itump Full time

    itump is seeking a talented Software Engineer to join our dynamic team and contribute to the development of high-quality software solutions. As a Software Engineer at itump, you will play a key role in designing, implementing, and maintaining software applications that meet the needs of our clients and exceed industry standards.Key...


  • San Francisco, United States Nceinfotech Full time

    At TechCorp, we're more than just a technology company; we're pioneers committed to crafting groundbreaking software solutions that revolutionize industries and enhance lives. Our enthusiasm for technology fuels our drive to leverage software as a catalyst for positive change. Job Overview Are you a passionate software engineer eager to make an impact? Join...


  • San Francisco, United States Nceinfotech Full time

    At TechCorp, we're more than just a technology company; we're pioneers committed to crafting groundbreaking software solutions that revolutionize industries and enhance lives. Our enthusiasm for technology fuels our drive to leverage software as a catalyst for positive change.Job OverviewAre you a passionate software engineer eager to make an impact? Join...


  • San Francisco, United States Nceinfotech Full time

    At TechCorp, we're more than just a technology company; we're pioneers committed to crafting groundbreaking software solutions that revolutionize industries and enhance lives. Our enthusiasm for technology fuels our drive to leverage software as a catalyst for positive change.Job OverviewAre you a passionate software engineer eager to make an impact? Join...

  • Software Engineer

    4 weeks ago


    San Francisco, California, United States Salvation Army Full time

    We are seeking a skilled Software Engineer to join our team.As a Software Engineer, you will be responsible for designing, developing, and testing software applications.Key responsibilities include:Designing and developing software applicationsCollaborating with cross-functional teamsTesting and debugging software applications


  • San Jose, California, United States Lynx Software Technologies Full time

    Job OverviewLynx Software Technologies is a leading technology firm specializing in the development of mission-critical software used in the aerospace and military sectors.We are seeking a talented Software Quality Assurance Engineer to champion all aspects of software quality, particularly related to test automation using scripting languages like Python....

  • Software Engineer

    1 month ago


    San Francisco, United States HonorVet Technologies Full time

    Title: Software Engineer Location: Remote Duration: 09+ months; possible extensions Job ID: 33841929Job Description:SOFTWARE ENGINEER | REMOTE (CST) The Select Group is looking for a talented Software Engineer to join one of our top Healthcare Technology partners to support their Reconciliation ledger Project, part of the company's strategic application...

  • Software Engineer A

    2 days ago


    San Francisco, United States Yaskawa America, Inc. Motoman Full time

    Create and develop support Windows desktop application software for motion controllers and subsystems, and the completion of current development projects. Be a key member of next generation controller and subsystem development tools, and will create work plans, detailed specifications, and schedules in conjunction with Development Manager and then execute...

  • Software Engineer

    3 days ago


    San Francisco, United States Workflow86 (YC W22) Full time

    About Us Workflow86 is revolutionizing the way businesses handle workflow automation. Our AI-powered software enables the autonomous design, construction, modification, and operation of complex business workflows with just natural language inputs. This groundbreaking approach simplifies workflow automation, making it more accessible and efficient for...

  • Software Engineer

    4 months ago


    San Francisco, United States MERU Full time

    This is a founding role. You will work directly with Venkat (the CTO) to build deployment configurations and platform features for our customers. You will be expected to help with engineering across the stack, including model architecture research, latency optimization at both the software and hardware level, cloud infrastructure, and product roadmap. You...


  • San Francisco, United States SingleStore Full time

    Position Overview MemSQL Software Engineers are at the forefront of real-time technology that’s revolutionizing how companies run their businesses every day. With release cycles throughout the year, our engineers are accustomed to moving quickly in a rapid production environment. To do this successfully, we work collaboratively and creatively across teams...

  • Software Engineer A

    3 weeks ago


    San Francisco, United States Yaskawa America, Inc. Motoman Full time

    Create and develop support Windows desktop application software for motion controllers and subsystems, and the completion of current development projects. Be a key member of next generation controller and subsystem development tools, and will create work plans, detailed specifications, and schedules in conjunction with Development Manager and then execute...

  • Software Engineer

    1 week ago


    San Francisco, United States Diverse Lynx Full time

    Job Title: Software Engineer Location: San Francisco , CA (Onsite) Type: Contract Skills: Strong knowledge of Linux subsystems and networking protocols (TCP/IP, UDP, IPsec etc.) Exposure to common security protocols and their design (SSH, IPsec, TLS etc.), and ability to understand security standard requirements and apply them to products. Proficiency in...

  • Software Engineer

    1 month ago


    San Francisco, United States Diverse Lynx Full time

    Job Title: Software Engineer Location: San Francisco , CA (Onsite) Type: Contract Skills: Strong knowledge of Linux subsystems and networking protocols (TCP/IP, UDP, IPsec etc.) Exposure to common security protocols and their design (SSH, IPsec, TLS etc.), and ability to understand security standard requirements and apply them to products. Proficiency in...

  • Software Engineer

    2 months ago


    san francisco, United States Alpha Search Advisors Full time

    Our Software Engineers are responsible for building the systems that power every aspect of their investment process, from research and analysis to trading, risk management, funding and settlement. Working side by side with investors and quantitative researchers, they use their technical skills to drive critical commercial outcomes. If you’re interested in...

  • Software Engineer

    2 months ago


    San Francisco, United States Alpha Search Advisors Full time

    Our Software Engineers are responsible for building the systems that power every aspect of their investment process, from research and analysis to trading, risk management, funding and settlement. Working side by side with investors and quantitative researchers, they use their technical skills to drive critical commercial outcomes. If you’re interested in...


  • San Jose, United States Lynx Software Technologies Full time

    Must be a U.S. citizen by law, regulation, executive order, or government contract for ITAR compliance. Must be within a commutable distance from the San Jose headquarters. About Us: We are a leading technology firm specializing in RTOSs, Hypervisors, and other system-level software for the military and aerospace sectors. We meet stringent certification...