Job VC
AI Python Engineer
Technologies
Description
Role Overview
We are looking for a Senior Python GenAI Engineer to join internal teams working on AI-driven decision automation solutions.
The role focuses on evaluating, testing, and improving Generative AI agents used in risk assessment, workflow automation, and document processing.
Key Responsibilities
Evaluate GenAI agent outputs for accuracy, consistency, and business relevance
Design and apply AI evaluation frameworks (quality, hallucination, reliability, bias, latency)
Test underwriting / risk-assessment AI workflows using real business scenarios
Improve prompts, agent logic, orchestration flows, and output quality
Collaborate with engineers, analysts, and stakeholders on production AI systems
Support continuous optimization of AI agents in regulated environments
Define KPIs and benchmarks for GenAI performance
Required Skills and Experience
6+ years of commercial software development experience
Hands-on experience with GenAI / LLM products in production
Understanding of AI agents / agentic workflowsKnowledge of banking, fintech, underwriting, insurance, or risk domains is a strong plus
Familiarity with Microsoft Copilot Studio is beneficial
Experience with LangChain / LangGraph or similar orchestration tools
Experience with evaluation frameworks (RAGAS, DeepEval, LangSmith, prompt testing, etc.)
Strong Python skills
We are looking for a Senior Python GenAI Engineer to join internal teams working on AI-driven decision automation solutions.
The role focuses on evaluating, testing, and improving Generative AI agents used in risk assessment, workflow automation, and document processing.
Key Responsibilities
Evaluate GenAI agent outputs for accuracy, consistency, and business relevance
Design and apply AI evaluation frameworks (quality, hallucination, reliability, bias, latency)
Test underwriting / risk-assessment AI workflows using real business scenarios
Improve prompts, agent logic, orchestration flows, and output quality
Collaborate with engineers, analysts, and stakeholders on production AI systems
Support continuous optimization of AI agents in regulated environments
Define KPIs and benchmarks for GenAI performance
Required Skills and Experience
6+ years of commercial software development experience
Hands-on experience with GenAI / LLM products in production
Understanding of AI agents / agentic workflowsKnowledge of banking, fintech, underwriting, insurance, or risk domains is a strong plus
Familiarity with Microsoft Copilot Studio is beneficial
Experience with LangChain / LangGraph or similar orchestration tools
Experience with evaluation frameworks (RAGAS, DeepEval, LangSmith, prompt testing, etc.)
Strong Python skills