[FR] Dynamic LoRA / Modular Knowledge Adapters API for websites, datasets and github repositories

What variant of Codex are you using?

APP, CLI, Extension

What feature would you like to see?

Feature description

In the model adaption process, one of the most painful issues is the context loss. To resolve the context loss, current solution is often the memorization mechanisms and the MCP servers to get the knowledge. For most of knowledge-bases solution, it can still be seen as providing a markdown file that describes the links of knowledge. And the model searches from website when it performing inference.

For developing a new programming language that few people use, we may generate some projects using codex, this generating process is based on our context, so in our case the model is already tuned by the context and proficient to generate such code.

However if we want to develop some code generating tools (or some product with code generation) that calls OpenAI api to generate this language, while only providing the github links, the model reads the repo and documents, which is much less efficient than the case that this model has already learned the grammar of this programming language.

I have checked the fine-tuning service: https://developers.openai.com/api/docs/guides/supervised-fine-tuning , but the request is similar but not the same as fine-tuning.

The thought is, can we use a pretrain model and inject an additional context like, directly to the model weights without searching for filling the context every time? (The skills are often only small md files, but have a knowledge of code repos would be more helpful)

My thought for solution

We know that a model can fit into various tasks using different task heads. For LLMs, most of weights are already trained and no need to be modified (Take human as an example, the weights that defines how he learn, how he react should be a fixed pattern). But the weights that trained by what he learned can be modified by another knowledge base. So we needn't fine-tune most of weights.

Consider that those weights are all stored in a head, such head can be < 100MB or smaller, making it portable and easy to transfer to another user.

Since most of us are also not able to provide the AI service, we still need OPENAI to host this service, but we will provide a head that helps the model to make better performance.

To be clearer, this includes following aspects :

1. We can train a "knowledge head" with configurable size (like 16MB, 32MB, 64MB or 256MB) and specify the model to use this head for better knowledge-based inference.
2. For users, we can specify a github repos, and wait for 1 or 2 hours (or more) that model being trained, then get a head weight. We can choose to store the head locally and just specify the path of it.
3. For providers/developers that wants to provide services:

• If this head can be downloaded as files, we can bundle it with our code generation tool, so that it knows the basic grammar. Such "bundle" is effective when the head is small.
• We can also upload heads into OPENAI accounts platform and specify an additional head id to provide our service. So the user can directly use this head by calling OPENAI api without downloading, this is effective especially when the head is large.

Is there already any mature solution for this? or any solution that I missed for this service? I'm really glad to know this service if it's already provided.

Additional information

Appreciate for your time and consideration😄!