feat: integrate ragflow

[anunay999]

Description:

Integrate the RAGFlow Python SDK into our app to enhance and streamline our Retrieval-Augmented Generation (RAG) workflow. This integration will leverage RAGFlow’s efficient document retrieval and deep document understanding capabilities to improve our backend processing and overall performance.

Tasks:

SDK Setup:

• Install the latest stable ragflow-sdk (e.g., pip install ragflow-sdk==0.16.0).
• Configure API keys and base URLs to match our RAGFlow backend setup.

Module Development:

• Create a dedicated module (e.g., ragflow_integration.py) to encapsulate all interactions with the SDK.
• Implement functions to handle document retrieval using the SDK’s retrieve method.

Pipeline Integration:

• Update our RAG pipeline to invoke the new module for processing user queries.
• Ensure the retrieved data is seamlessly integrated into the generation process.

Acceptance Criteria:

• The app successfully retrieves and processes data using the RAGFlow Python SDK.
• The new RAG workflow shows measurable improvements in performance and reliability.
• All new code is covered by automated tests and adheres to our security and configuration standards.

[nitinsurya]

@anunay999 , should we mark it is won't implement, and going with mem0 instead?

[anunay999]

Yes