Annie W

AI evaluator

Upwork

May 2025 - Present • 1 yr 3 mos

• Evaluate AI-generated healthcare responses for factual accuracy, clinical appropriateness, logical consistency, relevance, clarity, and safety. • Create diverse and challenging medical prompts involving psychiatric diagnosis, treatment planning, pharmacology, ethics, and clinical decision-making. • Compare model responses against evidence-based medical principles and identify unsupported claims, reasoning errors, omissions, and contradictions. • Review whether AI systems appropriately distinguish between common symptoms, urgent presentations, medical emergencies, and situations requiring specialist care. • Rank alternative AI responses according to correctness, completeness, usefulness, and adherence to task instructions. • Write clear explanations supporting evaluation decisions and provide actionable feedback for improving model performance. • Complete assignments independently while meeting accuracy, confidentiality, and quality-control requirements.