retriever support

[lmeyerov]

Beyond upcoming df & binary artifact upload support, there is interest in a generic retriever: #27

This makes sense natively, though unclear what it means for the remote client

[onestardao]

Interesting point — when you say “generic retriever,” are you thinking about a single interface that can work across both native and remote clients without extra config?
I’ve seen some setups where the client-side assumptions break when switching between local and remote backends, so clarifying your main use case might help spot potential pitfalls early.

[lmeyerov]

I can imagine a few extensible remote modes:

1. Louie's remote runtime is passing around a virtual retriever to a third-party remote runtime, like instead of in-process python dfr, just shimming to a remote vector db with its own out-of-process handling. So lui("do the thing", dfr) is being used as a shorthand for lui("load into vector db x, and access through its interface", df). So the remote runtime doesn't really care, just proxies.

2. The remote runtime allows static code loading, like a docker base image (ex: the popular path in extensible secure systems), or even pip install (ex: yolo web repls), and that gives executable semantics to the dfr being passed in. Then we support lllamaindex or some other notion of DFR, and folks can sideload a global engine pref here somehow.

3. We do full-on mobile code of some sort, similar to how dask (I think) lets you pass in methods.

4. Orthogonal to all this, maybe the answer is third-party DFRs encapsulated as MCP/Tools. So we still pass in the DF as flat data, and the DFR can be almost like a monad: DF -> DFR, str x DFR -> DFR, ... . Agents are free to use it.

The MCP path seems kind of clean for the remote code world...

[onestardao]

Interesting ideas — your mention of multiple remote modes makes me think about the failure patterns we often see when switching between local/remote retrievers (e.g., client-side assumptions breaking without clear interface guarantees).
Do you see this generic retriever also handling semantic mismatches (retrieval returns “correct” chunks but wrong meaning), or would that be left to downstream logic?

[lmeyerov]

I think it's "both". Imagine A2A protocol or some other.

An agent can help inform on its step based on its specialized view, but the orchestrator around it is doing an agentic reasoning (& learning) loop. When the agent doesn't sufficiently inform the loop, the loop has to guess based on output.

We have experience with our local protocols here, and those are based on popular learning paradigms, but not enough on ones like A2A that focuses on interop, so I don't have a good intuition on what'd help here. Thoughts / pointers to code ?

[onestardao]

Good points — the A2A protocol angle makes sense. On the semantic mismatch side, there’s actually a failure map I’ve been maintaining that covers these patterns (retriever mismatches, drift, etc.). It might be easier to scan it directly:
👉 https://github.com/onestardao/WFGY/tree/main/ProblemMap

It’s basically a “semantic firewall” catalog — no infra changes required, just a way to see how these mismatches have been debugged in practice.