Real Jobs. Real Change. See What's Next.
A living directory of real jobs that didn't exist 5 years ago. Curated for leaders, builders, and the curious.
Senior Manager of Data Quality LLM Operations
Oversees large-scale human annotation programs and training data quality systems to ensure optimal performance of large language models across diverse applications
Key Responsibilities:
- Lead global workforce of over 1,000 annotators including data science and quality assurance teams for LLM training data creation
- Collaborate with linguists, annotators, academics, and researchers to define and maintain cutting-edge data quality standards
- Own communication and thought leadership around data quality with key customers and strategic collaborators
- Supervise training materials and curriculum development for entire annotator workforce to ensure consistent quality
- Oversee creation of "golden datasets" to provide scalable quality signals and performance metrics for annotator evaluation
- Work with project teams to understand training data challenges and edge cases, providing strategic perspectives to customers
- Plan, design, implement, and test annotation tooling in collaboration with engineering teams
Skills & Tools:
- Large language model (LLM) experience and deep understanding of training data requirements
- Applied machine learning expertise with focus on data quality and human-in-the-loop systems
- Global team leadership and management experience (1000+ person teams)
- Dataset creation, annotation, and re-purposing methodologies
- Annotation tooling and platform management
- Cross-functional collaboration with engineering, research, and product teams
- Strong communication and thought leadership capabilities
- Data science and quality assurance program development
- Ambiguous problem-solving with iterative testing approaches
Where This Role Has Appeared:
- NVIDIA (AI Hardware/Software, Remote/US, $216k-$345k, July 2025)
Variants & Related Titles:
- Head of AI Training Data Operations
- LLM Data Quality Director
- AI Annotation Program Manager
- Human-in-the-Loop Operations Lead
- Training Data Excellence Manager
Why This Role Is New:
Senior Manager of Data Quality LLM Operations emerged in 2023-2024 as AI companies discovered that LLM performance is fundamentally limited by training data quality, not just compute power. The role addresses the unprecedented scale challenge of coordinating thousands of human annotators globally while maintaining consistent quality standards that directly impact billion-dollar AI model performance.
Trend Insight:
As competition intensifies among AI companies, the quality of human training data has become a critical differentiator, driving massive investments in human annotation operations that require specialized leadership to manage at global scale.
Seen this role elsewhere? Submit an example or share your story.