Learning Objectives

1Architect a production AI system combining all 13 days of skills
2Build RAG over a real GitHub repository's codebase
3Add evals, guardrails, and observability from Week 2
4Implement streaming UI with multi-model cost optimization
5Deploy and present your capstone — your AI engineering portfolio centerpiece

Ship It: AI codebase Q&A (portfolio centerpiece)

By the end of this day, you'll build and deploy a ai codebase q&a (portfolio centerpiece). This isn't a toy — it's a real project for your portfolio.

Before You Start — Rate Your Confidence

I can build a full AI pipeline with code-aware chunking, RAG, agents, evals, security, and observability — portfolio-ready.

1 = no idea · 5 = ship it blindfolded

Predict First — Then Learn

What makes an AI project 'hire-worthy' vs a tutorial exercise?

Final Capstone: AI Codebase Q&A — Your Portfolio Centerpiece

This is the project that gets you hired. An AI Codebase Q&A system that: ingests GitHub repos, understands code structure, answers questions about any codebase with source citations, and uses 10+ skills from the past 13 days. Recruiters and engineering managers will see this and immediately understand you can build production AI features.

💡This capstone uses 10+ skills from 13 days. It's not a tutorial — it's the project that gets you hired.

Quick Pulse Check

Why is an AI Codebase Q&A system an ideal portfolio project?

🚀 Full Pipeline Architecture

Watch all 14 days come together into a complete AI engineering pipeline.

0/11 stages · Click any stage or animate

Architecture: Full AI Pipeline

The system combines everything: (1) Code ingestion — parse files, extract functions/classes, build a code graph. (2) Chunking — smart code-aware chunking that respects function boundaries (not mid-function splits). (3) Embeddings — embed code chunks with metadata (file, function, language). (4) RAG — semantic search + re-ranking for code-specific retrieval. (5) Tools — the agent can search code, read files, and find definitions. (6) Streaming — responses stream with inline code snippets and file references. (7) Security — input validation, PII filtering, rate limiting. (8) Observability — trace every call, track costs, log metrics.

💡8 pipeline stages: ingest → chunk → embed → RAG → tools → stream → secure → observe. Each is a day's skill.

Quick Pulse Check

What's the role of tools/agents in the codebase Q&A system?

🎯 Skills Summary Radar

Your 14-day AI engineering skill map. Hover over axes for details.

Avg proficiency: 77% across 14 skills

Predict First — Then Learn

Why does generic text chunking fail for code?

Code-Aware Chunking: The Secret Ingredient

Generic text chunking (split every 500 chars) destroys code understanding. Code-aware chunking respects boundaries: each function is one chunk, each class is one chunk (or split by method), imports/types are separate chunks. Include metadata: filename, function name, line numbers, language. This is the difference between a toy demo and a tool engineers actually use.

💡Code-aware chunking respects function/class boundaries + includes metadata (file, function, lines). This is the secret sauce.

Quick Pulse Check

What metadata should each code chunk include?

Predict First — Then Learn

What should you show FIRST in a portfolio AI project demo?

The Demo: What to Show in Your Portfolio

Structure your demo for maximum impact: (1) Show it answering 'How does authentication work in this codebase?' with accurate, cited answers. (2) Show it finding bugs: 'Are there any unhandled promise rejections?' (3) Show streaming responses with inline code blocks. (4) Show source citations linking to actual files and line numbers. (5) Show it handling 'I don't know' gracefully. (6) Mention: built with Vercel AI SDK, RAG pipeline, tool-use agents, eval suite, cost optimization.

💡Demo structure: working Q&A → bug detection → streaming → citations → graceful 'I don't know' → tech stack.

✅ Portfolio Demo Checklist

Grade your capstone project. Check off each item as you complete it.

0/10 items complete

Check off items as you build

Architecture0/2

Core Features0/2

Quality0/2

Production0/2

Presentation0/2

What You've Built in 14 Days

You now have production-grade skills in: LLM APIs + structured output (Day 1), Embeddings (Day 2), RAG pipelines (Day 3), Tool-use (Day 4), Agent orchestration (Day 5), Multimodal + streaming (Day 6), Full-stack AI projects (Day 7), Evals (Day 8), Security (Day 9), Cost optimization (Day 10), LLMOps (Day 11), Fine-tuning (Day 12), AI integration (Day 13), and this capstone (Day 14). These aren't tutorial skills — they're the skills AI engineering teams hire for.

💡14 days, 14 production skills. These aren't tutorial skills — they're what AI engineering teams hire for.

Quick Pulse Check

Which Day 1-14 skill is MOST unique to AI engineering (not found in traditional SWE)?

The Full Evolution

Watch one function evolve through every concept you just learned.

🔄 Code Evolution — One Function, Five Stages

Step 1: Raw fetch()

The SWE starting point

Raw fetch, manual headers, raw text output

1async function reviewCode(code: string) {
2  const response = await fetch(
3    "https://api.openai.com/v1/chat/completions",
4    {
5      method: "POST",
6      headers: {
7        "Authorization": `Bearer ${API_KEY}`,
8        "Content-Type": "application/json",
9      },
10      body: JSON.stringify({
11        model: "gpt-4o-mini",
12        messages: [
13          { role: "user", content: `Review: ${code}` }
14        ],
15      }),
16    }
17  );
18  const data = await response.json();
19  return data.choices[0].message.content;
20  // Returns raw text — unparseable!
21}

1 / 5

Production Gotchas

Code chunking: don't embed minified or generated files (node_modules, dist, .min.js). Filter by .gitignore rules. Large repos need incremental indexing — re-embed only changed files. Respect rate limits during bulk embedding (batch with embedMany). Code search and text search need different approaches — function name matching is as important as semantic similarity. Test with repos of different sizes: a 10-file project and a 500-file monorepo have very different challenges.

Code Comparison

Simple Chatbot vs AI Codebase Q&A System

A toy demo vs a portfolio-worthy AI engineering project

Basic Chatbot (toy demo)Traditional

// Basic: just sends code to LLM and hopes
export async function POST(req: Request) {
  const { question, code } = await req.json();

  const result = await generateText({
    model: openai("gpt-4o"),
    system: "You are a code expert.",
    prompt: `Code: ${code}
Question: ${question}`,
  });

  return Response.json({ answer: result.text });
}

// Problems:
// - No RAG: can't handle large codebases
// - No chunking: context window overflow
// - No citations: can't verify answers
// - No streaming: slow UX
// - No tools: can't search or navigate
// - No security: prompt injection risk
// - No observability: invisible failures
// - Not impressive in a portfolio

Codebase Q&A (production-grade)AI Engineering

// Production: full AI pipeline
import { streamText, tool } from "ai";

export async function POST(req: Request) {
  // Security (Day 9)
  if (!rateLimitCheck(req)) return err(429);
  const { question } = validate(req);

  // Trace (Day 11)
  const trace = langfuse.trace({ name: "codeQA" });

  // RAG: embed + search (Day 2-3)
  const chunks = await codeSearch(question);

  // Agent with tools (Day 4-5)
  const result = streamText({
    model: routeModel(question), // Day 10
    system: HARDENED_PROMPT, // Day 9
    tools: {
      searchCode: tool({ /* Day 4 */ }),
      readFile: tool({ /* Day 4 */ }),
      findDefinition: tool({ /* Day 4 */ }),
    },
    messages: buildContext(chunks, question),
    onFinish: (r) => {
      trace.generation({ // Day 11
        output: r.text,
        usage: r.usage,
      });
    },
  });

  // Streaming (Day 6)
  return result.toDataStreamResponse();
}
// Every day's skill is used. THIS gets hired.

KEY DIFFERENCES

Combines ALL 13 days of skills into one production system
RAG for code understanding, Tools for navigation, Streaming for UX
Security, observability, and cost optimization built in
This is what AI engineering portfolios should look like

Bridge Map: Production deployment → Production-grade AI system

Click any bridge to see the translation

Hands-On Challenges

Build, experiment, and get AI-powered feedback on your code.

starter

Capstone: AI Codebase Q&A System

Build a simplified Codebase Q&A system. Given a set of pre-parsed code files, implement: (1) code-aware search using embeddings, (2) an AI agent that answers questions with source citations, (3) structured output with file references. This is the starter version — the full project adds streaming, tools, and multi-file analysis.

PLAYGROUND

import { useState } from "react";
import { z } from "zod";
// TODO: Import searchCode, generateObject, openai, codeFiles from the mock
// import { searchCode, generateObject, openai, codeFiles } from "./ai-sdk-mock";

// TODO: Define an answer schema with:
// - answer: string
// - sources: array of { file: string, functions: string[], relevance: "high" | "medium" | "low" }
// - confidence: "high" | "medium" | "low"

const sampleQuestions = [
  "How does authentication work?",
  "How are products created and validated?",
  "What's the rate limiting strategy?",
  "How does error handling work?",
  "Are there any security concerns?",
];

export default function App() {
  const [question, setQuestion] = useState("");
  const [result, setResult] = useState<any>(null);
  const [codeResults, setCodeResults] = useState<any[]>([]);
  const [loading, setLoading] = useState(false);
  const [showFiles, setShowFiles] = useState(false);

  async function handleAsk() {
    if (!question.trim()) return;
    setLoading(true);
    setResult(null);
    setCodeResults([]);

    try {
      // TODO: Step 1 - Search codebase for relevant files
      // const relevant = searchCode(question);
      // setCodeResults(relevant);

      // TODO: Step 2 - Build code context from search results
      // const context = relevant.map(f =>
      //   `--- ${f.path} ---\nFunctions: ${f.functions.join(', ')}\n${f.content}`
      // ).join("\n\n");

      // TODO: Step 3 - Generate answer with citations
      // Use generateObject with system prompt: "You are a codebase expert. Answer based on the code provided. Cite specific files and functions."
      // prompt: "Codebase context:\n{context}\n\nQuestion: {question}"

      setResult({ answer: "Implement the codebase Q&A pipeline!", sources: [], confidence: "low" });
    } finally {
      setLoading(false);
    }
  }

  return (
    <div style={{ padding: 20, fontFamily: "sans-serif", maxWidth: 750 }}>
      <h2>🔍 AI Codebase Q&A</h2>
      <p style={{ color: "#666", fontSize: 13 }}>
        Ask questions about any codebase — get answers with source citations
      </p>

      <div style={{ display: "flex", gap: 4, flexWrap: "wrap", margin: "8px 0" }}>
        {sampleQuestions.map((q, i) => (
          <button key={i} onClick={() => setQuestion(q)}
            style={{ padding: "4px 10px", fontSize: 11, borderRadius: 12, border: "1px solid #e2e8f0", background: "#f8fafc", cursor: "pointer" }}>
            {q}
          </button>
        ))}
      </div>

      <div style={{ display: "flex", gap: 8 }}>
        <input value={question} onChange={e => setQuestion(e.target.value)}
          onKeyDown={e => e.key === "Enter" && handleAsk()}
          placeholder="Ask about the codebase..."
          style={{ flex: 1, padding: 10, borderRadius: 8, border: "1px solid #e2e8f0", fontSize: 14 }} />
        <button onClick={handleAsk} disabled={loading}
          style={{ padding: "10px 20px", background: loading ? "#94a3b8" : "#8b5cf6", color: "white", border: "none", borderRadius: 8, cursor: "pointer" }}>
          {loading ? "Thinking..." : "Ask"}
        </button>
      </div>

      {codeResults.length > 0 && (
        <div style={{ marginTop: 12, padding: 10, background: "#f1f5f9", borderRadius: 8, fontSize: 12 }}>
          <strong>Searched {codeResults.length} relevant file(s):</strong>
          {codeResults.map((f: any, i: number) => (
            <div key={i} style={{ margin: "4px 0", padding: 4 }}>
              📄 <strong>{f.path}</strong>
              <span style={{ color: "#64748b" }}> — {f.functions.join(", ")}</span>
            </div>
          ))}
        </div>
      )}

      {result && (
        <div style={{ marginTop: 12, padding: 16, background: result.confidence === "low" ? "#fef2f2" : "#f0fdf4", borderRadius: 8, border: `1px solid ${result.confidence === "low" ? "#fecaca" : "#bbf7d0"}` }}>
          <div style={{ lineHeight: 1.7, fontSize: 14 }}>{result.answer}</div>
          {result.sources?.length > 0 && (
            <div style={{ marginTop: 12, paddingTop: 8, borderTop: "1px solid #e2e8f0" }}>
              <strong style={{ fontSize: 12 }}>Sources:</strong>
              {result.sources.map((s: any, i: number) => (
                <div key={i} style={{ fontSize: 12, color: "#666", margin: "4px 0", padding: "4px 8px", background: "#f1f5f9", borderRadius: 4 }}>
                  📄 <strong>{s.file}</strong>
                  {s.functions?.length > 0 && <span> — {s.functions.join(", ")}</span>}
                  <span style={{ float: "right", color: s.relevance === "high" ? "#16a34a" : "#ca8a04" }}>
                    {s.relevance}
                  </span>
                </div>
              ))}
            </div>
          )}
          <div style={{ marginTop: 8, fontSize: 11, color: "#94a3b8" }}>
            Confidence: {result.confidence}
          </div>
        </div>
      )}

      <button onClick={() => setShowFiles(!showFiles)}
        style={{ marginTop: 16, padding: "6px 12px", background: "white", border: "1px solid #e2e8f0", borderRadius: 6, cursor: "pointer", fontSize: 12 }}>
        {showFiles ? "Hide" : "Show"} Codebase Files
      </button>

      {showFiles && (
        <div style={{ marginTop: 8, fontSize: 11 }}>
          <div style={{ padding: 4, color: "#666" }}>📁 src/auth/login.ts — handleLogin, validateCredentials, createSession</div>
          <div style={{ padding: 4, color: "#666" }}>📁 src/api/products.ts — getProducts, createProduct, updateProduct, deleteProduct</div>
          <div style={{ padding: 4, color: "#666" }}>📁 src/middleware/rateLimiter.ts — rateLimiter, checkLimit, resetBucket</div>
          <div style={{ padding: 4, color: "#666" }}>📁 src/utils/errors.ts — NotFoundError, ValidationError, handleError</div>
        </div>
      )}
    </div>
  );
}

Open Sandbox

Real-World Challenge

AI Codebase Q&A System

Build and deploy your portfolio centerpiece: an AI-powered codebase Q&A tool that indexes a GitHub repo, answers questions about the code using RAG with code-aware chunking, includes an agent for code navigation, and combines every skill from the past 13 days. This is the project that gets you hired.

~6h estimated

Next.js 14+Vercel AI SDKOpenAI GPT-4o + text-embedding-3-smallLangfuse (observability)Tailwind CSSVercel (deploy)

Acceptance Criteria

Ingest code files from a GitHub repo or local directory with code-aware parsing
Chunk code intelligently: by function/class, not fixed-size, with file and line metadata
Embed code chunks and implement semantic code search with keyword re-ranking
Build an agent with tools: searchCode, readFile, findDefinition, findUsages
Stream responses with syntax-highlighted code blocks and clickable source citations
Add evals (Day 8), security (Day 9), and observability (Day 11)
Deploy to a public URL with a polished, portfolio-ready UI

Build Roadmap

0/7

Create a new Next.js app and plan the architecture: code ingestion → embedding → code search → agent → streaming UI. This is a complex project — start with architecture.

npx create-next-app@latest codebase-qa --typescript --tailwind --app

Create /lib/ingestion, /lib/search, /lib/agent, /app/api folders

Plan the data flow: repo → parse files → chunk → embed → search → agent → stream

Deploy Tip

Push to GitHub and import into Vercel. Pre-index a popular open-source repo (your own, or a small well-known one) so the demo works immediately. Write a killer README — this project is what gets you interviews.

Sign in to submit your deployed project.

After Learning — Rate Your Confidence Again

I can build a full AI pipeline with code-aware chunking, RAG, agents, evals, security, and observability — portfolio-ready.

1 = no idea · 5 = ship it blindfolded

Day 13: Adding AI to Existing Apps

Discussion

Sign in to join the discussion.

Loading...