LLM Data Quality Analyst
2 months ago
Duration: 5 Months Contract
Job Description:
- We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.
- In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM modeling teams, and ensure that the data produced by expert content creators meets the highest quality standards.
- You’ll need a strong ability to understand LLM use cases, have a nuanced understanding of good versus great content, and be able to use quantitative and qualitative techniques to analyze LLM datasets.
Responsibilities:
- Ensure high quality during task development:
- Define data requirements based on a deep understanding of the modeling team goals and study of model loss patterns.
- Design detailed instructions for content creator teams, illustrating nuanced differences with clear examples.
- Rapidly iterate with content-production vendors on small batches of data to arrive at desired data quality.
- Measure and demonstrate data quality during data collection:
- Drive processes to deliver benchmark datasets for assessing data quality.
- Benchmark vendor data quality against competitor models.
- Use quantitative techniques to analyze vendor-produced data to assure high quality and drive rater pool optimization.
- Audit datasets for quality issues and develop tools for accelerating qualitative analysis.
- Use modeling and experimentation techniques to demonstrate data impact and identify blind spots.
- Communicate on data quality:
- Send out regular reports to project stakeholders on data quality.
- Track key metrics such as iteration time and vendor quality audit results.
- Author best practices and guideline documents (e.g. guidelines for teams wanting to work with vendors for human data, best practices for instruction design, etc).
Experience:
- 3+ years of experience in Technical Writing, Quality Assurance or a related field.
- Excellent writing and editing skills, with the ability to adapt to different styles and tones.
- Strong analytical and problem-solving skills.
- Ability to work independently and as part of a team.
Skills:
- Data Quality
- LLM processes like Pre-training, RLHF, SFT, Evals
Education:
- Bachelor's degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
About US Tech Solutions:
US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit www.ustechsolutions.com.
US Tech Solutions is an Equal Opportunity Employer.All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
Recruiter Details:
Name: Ashik
Email: ashik.a@ustechsolutionsinc.com
Internal Id: 24-26342
-
LLM Data Quality Analyst
4 weeks ago
Sunnyvale, United States US Tech Solutions Full timeDuration: 5 Months ContractJob Description:We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM...
-
LLM Data Quality Analyst
2 months ago
Sunnyvale, United States US Tech Solutions Full timeDuration: 5 Months ContractJob Description:We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM...
-
Senior LLM Data Specialist
2 weeks ago
Sunnyvale, California, United States US Tech Solutions Full timeAbout US Tech Solutions:We are a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions.Job Description:We are looking for a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.About the Role:The project...
-
LLM/Prompt Engineering Data Modeler
2 days ago
Sunnyvale, United States US Tech Solutions Full timeDuration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...
-
LLM/Prompt Engineering Data Modeler
2 weeks ago
Sunnyvale, United States US Tech Solutions Full timeDuration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...
-
Data Quality Expert for LLM Modeling Teams
4 weeks ago
Sunnyvale, California, United States US Tech Solutions Full timeAbout the Role:We are seeking a highly skilled Data Quality Expert to join our team at US Tech Solutions. The ideal candidate will have a strong understanding of Large Language Models (LLMs) and be able to ensure the highest quality standards in data production.Key Responsibilities:Collaborate with technical modeling teams and vendor leads to understand data...
-
LLM Content Expert
3 days ago
Sunnyvale, California, United States US Tech Solutions Full timeData Quality Analyst Job Description">We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.About the Job">This role involves ensuring that the data produced by expert content creators meets the highest quality standards. You will work...
-
Human Data Analyst
3 days ago
Sunnyvale, California, United States US Tech Solutions Full timeHuman Data Analyst Position">We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.About the Role">In this role, you will work closely with highly technical modeling teams and vendor leads to understand the data needs of LLM modeling...
-
sunnyvale, United States US Tech Solutions Full timeDuration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...
-
sunnyvale, United States US Tech Solutions Full timeDuration: 6 Months Contract Job Description:The project involves analyzing the results of Human Data in the coding domain.We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.In this role, you will work closely with highly technical...
-
Large Language Model Data Analyst
2 weeks ago
Sunnyvale, California, United States US Tech Solutions Full timeJob Overview:We are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.About the Role:The project involves analyzing the results of Human Data in the coding domain.Work closely with technical modeling teams and vendor leads to understand data...
-
Data Quality Expert for Large Language Models
2 weeks ago
Sunnyvale, California, United States US Tech Solutions Full timeJob OverviewWe are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data.About the RoleIn this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM modeling teams, and...
-
Data Modeler Engineer
1 week ago
Sunnyvale, California, United States US Tech Solutions Full timeJob Title: Data Modeler EngineerWe are seeking a highly skilled Data Modeling Engineer with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-Created Data.About the Role:The project involves analyzing the results of Human Data in the coding domain.Work closely with technical modeling teams and vendor leads...
-
Data Quality Expert for Large Language Models
2 weeks ago
Sunnyvale, California, United States US Tech Solutions Full timeAbout the Role:US Tech Solutions is seeking a skilled Data Quality Expert to join our team in providing high-quality human-created data for Large Language Models (LLMs). The successful candidate will have a strong understanding of LLM processes and be able to ensure that the data meets the highest quality standards.Key Responsibilities:Collaborate with...
-
Quality Data Analyst
2 months ago
Sunnyvale, United States META Full timeSummary: Meta’s Reality Labs Quality and Compliance team is seeking an experienced Data Analyst to provide technical Analytics solutions for the millions of Quest and Ray Ban Meta devices we ship.You’ll be responsible for developing data products such as statistical models and forecasts, complex queries, and Tableau dashboards. You will also play a...
-
Data Quality Expert for Large Language Models
4 weeks ago
Sunnyvale, California, United States US Tech Solutions Full timeJob DescriptionWe are seeking a highly skilled Data Quality Expert to join our team at US Tech Solutions. As a key member of our team, you will play a critical role in ensuring the highest quality data is produced for our Large Language Model (LLM) projects.The ideal candidate will have a strong understanding of LLM processes, including Pre-training, RLHF,...
-
Data Analyst
1 month ago
Sunnyvale, United States GForce Life Sciences Full timeBusiness Intelligence Data Analyst Must be able to work on a W2 Primary Function of Position:The Data Analyst will be a member of the RA/QA Quality Data Analytics team and use technical expertise with the understanding of business needs to turn data into useful and actionable information. In this role, the analyst will support the RA/QA business needs for...
-
LLM Prompt Writer
4 weeks ago
Sunnyvale, United States US Tech Solutions Full timeDuration: 6 months contract Job Description:The primary focus is on enhancing customer prompt rewrites and contributing to tasks that drive improvements in generative AI models.These roles are essential for advancing the quality and functionality of content outputs for client's cross-functional projects, including those linked to AI-driven platforms....
-
Data Analyst Developer
1 week ago
Sunnyvale, California, United States Futran Tech Solutions Pvt. Ltd. Full time**About Futran Tech Solutions Pvt. Ltd.**">We are seeking a skilled Data Analyst Developer to join our team in Sunnyvale, CA.The estimated salary for this role is $160,000 - $200,000 per year, depending on experience.As a Data Analyst Developer, you will perform computational analysis using Snowflake, mySQL, Teradata, Python, Tableau and develop and maintain...
-
Data Analyst III
4 weeks ago
Sunnyvale, United States Walmart Full timePosition Summary It's an exciting time to join our Walmart Technology journey. We are seeking for a Data Analyst. This position is part of the Walmart Global Tech SEM Engineering team. SEM team is in charge of optimizing paid and free search for Walmart. We are tasked with optimizing ad spend on RoAS while making sure as many ads for the items from our...