Use case frustration

[peachkeel]

The problem I want to talk about lies somewhere in the vicinity of issues #246, #191, and #176 (comment). It emerges when one has to index a large amount of documents (i.e., a corpus) so that people can try out a given source storage with various assistants.

Right now, to handle this use case I have to:

1. Create a new account for the corpus (e.g., MyCorpus).
2. Use the API to upload all the files in the corpus to the new account.
3. Use the API to create an initial chat with the uploaded files and the required source storage (e.g., Vectara — see Vectara Ragna Plugin #203)
4. Let the long-running index process commence.
5. Make note of the chat ID (e.g., 55a097cf-c0ae-4082-8de3-b924e900079e) once indexing completes.
6. Subclass the class of the source storage used to create the initial chat. This neutered subclass does not store anything so as to prevent the re-indexing of the corpus. Rather, it only retrieves sources stored under the auspices of the initial chat. An example follows:

class Vectara(VectorDatabaseSourceStorage):
    # Class of the source storage used to create the initial chat.
class MyCorpusVectara(Vectara):

    def store(
        self,
        documents: list[Document],
        *,
        chat_id: uuid.UUID,
        chunk_size: int = 500,
        chunk_overlap: int = None
    ) -> None:
        pass

    def retrieve(
        self,
        documents: list[Document],
        prompt: str,
        *,
        chat_id: uuid.UUID,
        chunk_size: int = 500,
        num_tokens: int = 4000
    ) -> list[Source]:
        return super().retrieve(
            documents,
            prompt,
            chat_id=uuid.UUID("55a097cf-c0ae-4082-8de3-b924e900079e"),
            chunk_size=chunk_size,
            num_tokens=num_tokens
        )

7. Use the API to create subsequent chats with the uploaded files, the neutered subclass, and the desired assistants.
8. Hand the chats off for testing, to be done either via the UI or the API.

Undesirable Side Effects

The above procedure yields several undesirable side effects, including:

• A proliferation of neutered subclasses across the code base.
• Accounts bound to corpora, not organizational end users.
• Complete abandonment of the UI when it comes to chat creation.
• Cross-contamination of accounts with specialized source storages (e.g., MyCorpusVectara source storage in MyOtherCorpus account).

I'm wondering if anyone else is struggling with this use case, and if there are ways we could refactor Ragna to address what seems to me to be some sort of conceptual gap.

[pmeier]

Thanks for starting the discussion. Coincidentally, this happened while I finally had some time to write up #256, which I had in my backlog for two weeks now.

I'm wondering if anyone else is struggling with this use case, and if there are ways we could refactor Ragna to address what seems to me to be some sort of conceptual gap.

TL;DR Yes, you are not the only one struggling with this. Please have a look at #256 which should acknowledge all the issues you are facing here. If not, feel free to chime in there to avoid scattering this across multiple issues or discussions.

[peachkeel]

Discussion moved to #256.