Ajay M

AI monitoring & Evaluation Specialist - Legal SME

Mercor

Jun 2026 - Present • 3 mos

Financial-Services & Regulatory AI Auditing: Serve as a high-specialist evaluator conducting comprehensive compliance and regulatory audits on enterprise financial-services AI models, verifying strict adherence to statutory frameworks, regulatory disclosure norms, and legal accuracy. Complex Prompt Engineering & Stress-Testing: Design adversarial, multi-turn, domain-specific prompts across law, environmental science, journalism, and education; systematically stress-test frontier LLMs to uncover latent hallucinations, logic breakdowns, and safety vulnerabilities. Rubric Architecture & Alignment: Engineer multi-dimensional, ground-truth evaluation rubrics, scoring criteria, and cut-score benchmarks to quantify model reasoning depth, factual precision, coherence, and safety policy compliance. Frontier RLHF & SFT Pipelines: Execute expert Reinforcement Learning from Human Feedback (RLHF) and Supervised Fine-Tuning (SFT) workflows in collaboration with premier AI labs to enhance domain reasoning in next-generation LLM releases. High-Stakes Technical Deliverables: Deliver executive-level synthesis reports, compliance memos, structured analytical datasets, comparative spreadsheets, and strategic slide decks to guide engineering and alignment teams.