Skip to the page
Tecyfy

Agents interview question · medium

Your company built a question-answering agent over its policy documents with Databricks Knowledge Assistant, which is now legacy. How would you build the same agent today, and how would you move users across?

The short answerPick the replacement by who maintains the agent.

The full answer

Pick the replacement by who maintains the agent. If analysts configure it in screens and it only answers from documents and tables, rebuild it as a Genie Agent, the low-code successor. If engineers need custom logic, several tools or routing between agents, build it in code with the Agent Bricks CLI: scaffold a LangGraph or OpenAI Agents SDK project, run it locally, deploy it to Databricks' agent runtime, and add any Genie Agent as a tool. Then run old and new side by side on the same questions before switching users.

1. The map

LegacyReplacement
Knowledge AssistantGenie Agents
Supervisor AgentGenie Agents, or the Agent Bricks CLI for coordination in code
Legacy agent memoryAgent memory and agent sessions
Custom agent on Model ServingAgent Bricks CLI

2. The code path

pip install databricks-agentbricks
agentbricks init --framework langgraph policy-agent
cd policy-agent
agentbricks tools add genie-agent        # reuse an analyst-built Genie Agent as a tool
agentbricks sessions bind policy-sessions
agentbricks memory bind policy-memory
agentbricks dev                          # local test on port 8000
agentbricks deploy policy-agent

The CLI is in Beta, so pin its version.

3. Prove it before switching

  • Collect 100 or more real questions from the old agent's logs, with the answers users accepted.
  • Run both agents on them; compare correct answers, citations to the right document, refusals and latency.
  • Keep retrieval measured separately: if the right policy passage is not retrieved, change chunking or search before changing prompts. See RAG.

4. Move users

Point a pilot group at the new agent, watch feedback for a normal week, then switch everyone and retire the old one. Legacy means no new features, not switched off, and Databricks has not published an end date, so there is time to do this properly, but new work should not start on the legacy product.

What interviewers want: the low-code versus code choice tied to who maintains it, the replacement map, the CLI's scaffold, local run and deploy, memory and Genie Agents reused, a side-by-side evaluation on real questions, and a staged switch-over.

Last updated 22 October 2026