Predictive Science - AI Engineering & Prompt Architecture Lead - Vice President

Take a lead role in acquiring, managing and retaining meaningful relationships that deliver outstanding experience to our customers. In this role, you will balance your focus on business results by offering options and finding solutions to help our customers with issues.

As a Vice President – AI Engineering & Prompt Architecture Lead in Predictive Science, you will own the technical vision and delivery for a team of prompt engineers and AI/ML model developers building intelligent document processing and data extraction solutions in the KYC/AML domain.

Job Responsibilities:

• Architect agentic, multi-step AI workflows chaining classification → extraction → cross-validation → exception routing with human-in-the-loop checkpoints.

• Debug and remediate complex prompt failures (context-window overflow, instruction drift in long chains, RAG retrieval poisoning, output/format instability).

• Design prompt/model evaluation frameworks measuring accuracy plus consistency, robustness, latency, cost-per-call, and hallucination rate.

• Operationalize prompt lifecycle management as production code: versioning, CI/CD prompt tests, A/B experiments, rollback, and audited change history.

• Guide model selection and optimization (prompting vs fine-tuning vs custom training) balancing accuracy, latency, cost, and data sensitivity.

• Design RAG architectures for financial documents: chunking, embeddings, vector store design, re-ranking, and context injection.

• Oversee fine-tuning/training workflows: dataset curation, annotation quality, training configurations, and generalization across document variants.

• Build and maintain evaluation infrastructure: benchmark/golden datasets, regression suites, and automated scoring to catch regressions pre-production.

• Define confidence calibration and escalation logic so systems estimate uncertainty and route low-confidence outputs to human reviewers with the right context.

• Partner with governance/model risk and data engineering: produce validator-ready documentation (explainability/auditability) and ensure robust, refreshed data/annotation/eval pipelines; drive AI-native development practices to improve velocity.

Required qualifications, capabilities and skills:

• 10+ years in NLP/AI/ML or computational linguistics, including 3+ years leading technical teams with direct reports.

• Hands-on LLM internals expertise: tokenization impacts, attention limits, context window management, and temperature/sampling trade-offs.

• Proven prompt architecture design/debugging: multi-turn chains, few-/many-shot, chain-of-thought, self-consistency, and constitutional AI.

• Strong RAG system design experience: embeddings trade-offs (e.g., ada/BGE/Cohere), chunking for semi-structured docs, hybrid retrieval (dense+sparse), and re-ranking.