Any plans to introduce code indexing?

[RastislavKish]

Hello,

first of all, a cool project!

Larger codebases often significantly exceed even the largest context windows available these days, while offline LLMs are even more troublesome in this regard.

It could be useful to implement an indexing feature, that would not generate a single prompt from the codebase, but instead output multiple smaller prompts containing max. N tokens, with the purpose of creating some kind of code abstraction. This abstract could be afterwards used together with just a single file of code where modifications should be made.

I don't use LLMs for coding very frequently, but this seems like the only plausible approach for fitting large codebases into LLLMs. Have you made any considerations/experiments with this approach and possible implementation into code2prompt?

[mufeedvh]

Hi @RastislavKish, this is an interesting feature request. Dividing prompts into multiple chunks would lose important context when working with the entire codebase and the context-window applies to the entire conversation with an LLM just that it acts as a sliding window where it loses context as we consume more tokens.

Could you please describe how you'd be using such a feature? I'll see if I could think about a feature that could tailor to your needs.

[swiftugandan]

Aider gives some clues on how you could compress the context by just looking at the symbols in the code. https://aider.chat/2023/10/22/repomap.html ... Perhaps you could consider something similar.

[dbenn8]

I once built a really simple python script that traversed the code files in a project and pulled out each function (and maybe symbol?) and created a single markdown file with the parameters, return type, and comments all grouped by file within it.

It was probably a hacky solution to this problem (context Windows were smaller then), but it does help the LLM get broad overall context if you also feed it the full details of sections of the code more relevant to the specific problem you want help with.

[ODAncona]

Hi @RastislavKish @swiftugandan,

Thank you for your interest in code2prompt and for your thoughtful feature suggestion regarding code abstraction for large codebases. This is a hot topic as Gemini 2.5 Coder was released with a context window of 1M tokens 🎉

Code2prompt is designed as a context-feeder/code ingestion tool that works for both humans and AI agents. Its primary purpose is to curate code by selecting relevant files and folders in order to forge a meaningful prompt reducing token usage, hallucinations and thus improving LLMs quality output. To sum up, it helps achieve the rule of thumb :

Provide as little context as possible, but as much as necessary

There are diverse way to provide meaningful context:

• RAG
• Agent summary
• Code compression
• Raw files

Code2prompt is a simple tool that helps agents gather context and will therefore not use RAG or Agent Summary techniques that require LLM inference under the hood. Instead, the vision would to develop a smart-context feature that is able to get relevant context of a particular file/function using AST (like tree sitter). The way I envision this, is to build an import graph and a call stack and feed the predecessor imports to the context. You could call it like this code2prompt . --smart-context <n> where n should be the number of predecessor of the import//call graph you include in the context. By default it could be n=2 What is your opinion on this ?

This is very important to keep in mind that code2prompt aim to integrate with agents like aider, goose, cline or open-hand. It could be called with an MCP or be integrated from source directly with rust or the python SDK.

Here's an example how I use it with Cline:

Generate a unit test for the hidden flag.
Get context of code2prompt-core/{configuration.rs and path.rs} in the core crate  + main.rs file in the cli crate

and then

As LLMs will improve and agents as well, it's important to see where code2prompt fits. As a context-feeder tool that integrate seamlessly with agents.