LLM Data Quality Analyst

2 months ago


sunnyvale, United States US Tech Solutions Full time

Duration: 5 Months Contract



Job Description:

  • We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.
  • In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM modeling teams, and ensure that the data produced by expert content creators meets the highest quality standards.
  • You’ll need a strong ability to understand LLM use cases, have a nuanced understanding of good versus great content, and be able to use quantitative and qualitative techniques to analyze LLM datasets.



Responsibilities:

  • Ensure high quality during task development:
  • Define data requirements based on a deep understanding of the modeling team goals and study of model loss patterns.
  • Design detailed instructions for content creator teams, illustrating nuanced differences with clear examples.
  • Rapidly iterate with content-production vendors on small batches of data to arrive at desired data quality.
  • Measure and demonstrate data quality during data collection:
  • Drive processes to deliver benchmark datasets for assessing data quality.
  • Benchmark vendor data quality against competitor models.
  • Use quantitative techniques to analyze vendor-produced data to assure high quality and drive rater pool optimization.
  • Audit datasets for quality issues and develop tools for accelerating qualitative analysis.
  • Use modeling and experimentation techniques to demonstrate data impact and identify blind spots.
  • Communicate on data quality:
  • Send out regular reports to project stakeholders on data quality.
  • Track key metrics such as iteration time and vendor quality audit results.
  • Author best practices and guideline documents (e.g. guidelines for teams wanting to work with vendors for human data, best practices for instruction design, etc).



Experience:

  • 3+ years of experience in Technical Writing, Quality Assurance or a related field.
  • Excellent writing and editing skills, with the ability to adapt to different styles and tones.
  • Strong analytical and problem-solving skills.
  • Ability to work independently and as part of a team.



Skills:

  • Data Quality
  • LLM processes like Pre-training, RLHF, SFT, Evals


Education:

  • Bachelor's degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.



About US Tech Solutions:

US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit www.ustechsolutions.com.


US Tech Solutions is an Equal Opportunity Employer.All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.


Recruiter Details:

Name: Ashik

Email: ashik.a@ustechsolutionsinc.com

Internal Id: 24-26342



  • Sunnyvale, United States US Tech Solutions Full time

    Duration: 5 Months ContractJob Description:We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM...


  • Sunnyvale, United States US Tech Solutions Full time

    Duration: 5 Months ContractJob Description:We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM...


  • Sunnyvale, California, United States US Tech Solutions Full time

    About US Tech Solutions:We are a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions.Job Description:We are looking for a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.About the Role:The project...


  • Sunnyvale, United States US Tech Solutions Full time

    Duration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...


  • Sunnyvale, United States US Tech Solutions Full time

    Duration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...


  • Sunnyvale, California, United States US Tech Solutions Full time

    About the Role:We are seeking a highly skilled Data Quality Expert to join our team at US Tech Solutions. The ideal candidate will have a strong understanding of Large Language Models (LLMs) and be able to ensure the highest quality standards in data production.Key Responsibilities:Collaborate with technical modeling teams and vendor leads to understand data...

  • LLM Content Expert

    3 days ago


    Sunnyvale, California, United States US Tech Solutions Full time

    Data Quality Analyst Job Description">We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.About the Job">This role involves ensuring that the data produced by expert content creators meets the highest quality standards. You will work...

  • Human Data Analyst

    3 days ago


    Sunnyvale, California, United States US Tech Solutions Full time

    Human Data Analyst Position">We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.About the Role">In this role, you will work closely with highly technical modeling teams and vendor leads to understand the data needs of LLM modeling...


  • sunnyvale, United States US Tech Solutions Full time

    Duration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...


  • sunnyvale, United States US Tech Solutions Full time

    Duration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...


  • Sunnyvale, California, United States US Tech Solutions Full time

    Job Overview:We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.About the Role:The project involves analyzing the results of Human Data in the coding domain.Work closely with technical modeling teams and vendor leads to understand data...


  • Sunnyvale, California, United States US Tech Solutions Full time

    Job OverviewWe are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.About the RoleIn this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM modeling teams, and...


  • Sunnyvale, California, United States US Tech Solutions Full time

    Job Title: Data Modeler EngineerWe are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.About the Role:The project involves analyzing the results of Human Data in the coding domain.Work closely with technical modeling teams and vendor leads...


  • Sunnyvale, California, United States US Tech Solutions Full time

    About the Role:US Tech Solutions is seeking a skilled Data Quality Expert to join our team in providing high-quality human-created data for Large Language Models (LLMs). The successful candidate will have a strong understanding of LLM processes and be able to ensure that the data meets the highest quality standards.Key Responsibilities:Collaborate with...

  • Quality Data Analyst

    2 months ago


    Sunnyvale, United States META Full time

    Summary: Meta’s Reality Labs Quality and Compliance team is seeking an experienced Data Analyst to provide technical Analytics solutions for the millions of Quest and Ray Ban Meta devices we ship.You’ll be responsible for developing data products such as statistical models and forecasts, complex queries, and Tableau dashboards. You will also play a...


  • Sunnyvale, California, United States US Tech Solutions Full time

    Job DescriptionWe are seeking a highly skilled Data Quality Expert to join our team at US Tech Solutions. As a key member of our team, you will play a critical role in ensuring the highest quality data is produced for our Large Language Model (LLM) projects.The ideal candidate will have a strong understanding of LLM processes, including Pre-training, RLHF,...

  • Data Analyst

    1 month ago


    Sunnyvale, United States GForce Life Sciences Full time

    Business Intelligence Data Analyst Must be able to work on a W2 Primary Function of Position:The Data Analyst will be a member of the RA/QA Quality Data Analytics team and use technical expertise with the understanding of business needs to turn data into useful and actionable information. In this role, the analyst will support the RA/QA business needs for...

  • LLM Prompt Writer

    4 weeks ago


    Sunnyvale, United States US Tech Solutions Full time

    Duration: 6 months contract Job Description:The primary focus is on enhancing customer prompt rewrites and contributing to tasks that drive improvements in generative AI models.These roles are essential for advancing the quality and functionality of content outputs for client's cross-functional projects, including those linked to AI-driven platforms....


  • Sunnyvale, California, United States Futran Tech Solutions Pvt. Ltd. Full time

    **About Futran Tech Solutions Pvt. Ltd.**">We are seeking a skilled Data Analyst Developer to join our team in Sunnyvale, CA.The estimated salary for this role is $160,000 - $200,000 per year, depending on experience.As a Data Analyst Developer, you will perform computational analysis using Snowflake, mySQL, Teradata, Python, Tableau and develop and maintain...

  • Data Analyst III

    4 weeks ago


    Sunnyvale, United States Walmart Full time

    Position Summary It's an exciting time to join our Walmart Technology journey. We are seeking for a Data Analyst. This position is part of the Walmart Global Tech SEM Engineering team. SEM team is in charge of optimizing paid and free search for Walmart. We are tasked with optimizing ad spend on RoAS while making sure as many ads for the items from our...