Gemini's 1M Token Context Window: Processing Entire Codebases and Books

The Long Context Advantage

Most large language models operate with context windows between 8K and 128K tokens. Gemini 2.0 Pro supports up to 1 million tokens of input — roughly 700,000 words or an entire codebase of 30,000 lines. This eliminates the need for complex chunking, retrieval augmented generation, or summarization pipelines in many use cases.

For agent developers, long context means your agent can reason over an entire repository, a complete legal contract, or hours of meeting transcripts without losing information to truncation or retrieval errors. The model sees everything at once.

Calculating Token Usage

Before loading large documents, understand how tokens map to your content:

flowchart LR
    INPUT(["User intent"])
    PARSE["Parse plus<br/>classify"]
    PLAN["Plan and tool<br/>selection"]
    AGENT["Agent loop<br/>LLM plus tools"]
    GUARD{"Guardrails<br/>and policy"}
    EXEC["Execute and<br/>verify result"]
    OBS[("Trace and metrics")]
    OUT(["Outcome plus<br/>next action"])
    INPUT --> PARSE --> PLAN --> AGENT --> GUARD
    GUARD -->|Pass| EXEC --> OUT
    GUARD -->|Fail| AGENT
    AGENT --> OBS
    style AGENT fill:#4f46e5,stroke:#4338ca,color:#fff
    style GUARD fill:#f59e0b,stroke:#d97706,color:#1f2937
    style OBS fill:#ede9fe,stroke:#7c3aed,color:#1e1b4b
    style OUT fill:#059669,stroke:#047857,color:#fff

import google.generativeai as genai
import os

genai.configure(api_key=os.environ["GOOGLE_API_KEY"])

model = genai.GenerativeModel("gemini-2.0-pro")

# Count tokens before sending
sample_text = open("large_document.txt").read()
token_count = model.count_tokens(sample_text)
print(f"Document tokens: {token_count.total_tokens}")
print(f"Percentage of context used: {token_count.total_tokens / 1_000_000 * 100:.1f}%")

A rough rule of thumb: 1 token is approximately 4 characters of English text, or about 0.75 words. A 500-page book is typically 200K-400K tokens.

Loading an Entire Codebase

Here is a practical pattern for loading a full project into Gemini's context:

import os
import google.generativeai as genai

genai.configure(api_key=os.environ["GOOGLE_API_KEY"])

def load_codebase(root_dir: str, extensions: set = None) -> str:
    """Load all source files from a directory into a single string."""
    if extensions is None:
        extensions = {".py", ".ts", ".js", ".sql", ".yaml", ".json"}

    files_content = []
    for dirpath, dirnames, filenames in os.walk(root_dir):
        # Skip common non-source directories
        dirnames[:] = [d for d in dirnames if d not in {
            "node_modules", ".git", "__pycache__", ".venv", "dist"
        }]

        for filename in sorted(filenames):
            if any(filename.endswith(ext) for ext in extensions):
                filepath = os.path.join(dirpath, filename)
                relative = os.path.relpath(filepath, root_dir)
                try:
                    content = open(filepath).read()
                    files_content.append(
                        f"=== FILE: {relative} ===\n{content}\n"
                    )
                except (UnicodeDecodeError, PermissionError):
                    continue

    return "\n".join(files_content)

codebase = load_codebase("./my-project")
model = genai.GenerativeModel("gemini-2.0-pro")

token_count = model.count_tokens(codebase)
print(f"Codebase size: {token_count.total_tokens} tokens")

Analyzing Documents with Long Context

Once the content is loaded, you can ask complex analytical questions:

model = genai.GenerativeModel(
    "gemini-2.0-pro",
    system_instruction="""You are a senior software architect reviewing a codebase.
    Identify architectural patterns, potential issues, and improvement opportunities.
    Always reference specific file paths and line numbers in your analysis.""",
)

response = model.generate_content([
    f"Here is the complete codebase:\n\n{codebase}",
    "\nAnalyze the error handling patterns across this codebase. "
    "Which files have inconsistent error handling? "
    "What patterns should be standardized?"
])

print(response.text)

Context Caching for Repeated Queries

When you need to ask multiple questions about the same large document, context caching avoids re-sending the full content each time:

# Create a cached context
cache = genai.caching.CachedContent.create(
    model="models/gemini-2.0-flash",
    display_name="project-codebase",
    system_instruction="You are a code review assistant.",
    contents=[codebase],
)

# Use the cache for multiple queries — much faster and cheaper
model = genai.GenerativeModel.from_cached_content(cache)

response1 = model.generate_content("List all API endpoints in this codebase.")
response2 = model.generate_content("Find any SQL injection vulnerabilities.")
response3 = model.generate_content("What database migrations are pending?")

# Clean up when done
cache.delete()

Context caching reduces costs by up to 75% for repeated queries against the same base content. The cache has a minimum size of 32K tokens and a default TTL of 1 hour.

Strategies for Near-Limit Context

When your content approaches the 1M token limit, apply these strategies:

1. Prioritize relevant files — load core business logic first, test files last
2. Strip comments and docstrings for code analysis tasks where documentation is not relevant
3. Use file-level summaries for less critical files while including full content for the files under review
4. Split into focused sessions — analyze backend and frontend separately rather than together

def prioritized_load(root_dir: str, priority_dirs: list, max_tokens: int = 900_000):
    """Load high-priority directories first, stopping near the token limit."""
    model = genai.GenerativeModel("gemini-2.0-pro")
    accumulated = []
    total_tokens = 0

    for priority_dir in priority_dirs:
        dir_content = load_codebase(os.path.join(root_dir, priority_dir))
        count = model.count_tokens(dir_content).total_tokens
        if total_tokens + count > max_tokens:
            break
        accumulated.append(dir_content)
        total_tokens += count

    return "\n".join(accumulated), total_tokens

FAQ

Does using more context make responses slower?

Yes. Latency increases with input size, roughly linearly. A 1M token request takes significantly longer than a 10K token request. For interactive agents, consider whether the full context is necessary or if a targeted subset would suffice.

Is context caching available for all Gemini models?

Context caching is available for Gemini 2.0 Flash and Gemini 2.0 Pro through both AI Studio and Vertex AI. The cached content must be at least 32,768 tokens and has a configurable TTL with a minimum of 1 minute.

How does long context compare to RAG for document analysis?

Long context is simpler and avoids retrieval errors — the model sees everything. RAG is necessary when your data exceeds the context window or changes frequently. For documents under 500K tokens, long context typically produces more accurate answers because the model can cross-reference information across the entire document without depending on retrieval quality.

---

#GoogleGemini #LongContext #DocumentAnalysis #Python #AIAgents #AgenticAI #LearnAI #AIEngineering