Project Shard is a pet project of mine. It is designed to be a simple and user-friendly GUI interface for users to interact with their favorite LLMs localy. Currently , it supports both Ollama models and Open Router .
NEW - added support for Open Router! Some models are free! Requires API key!
The core idea is to provide an all-in-one solution that runs privately on your localhost.
The main focus is on being FREE, OPEN SOURCE, and PRIVATE.
✅ Multiple Profiles – Create unique profiles for different household members.
✅ Stream Chat Interface with Markdown Support – Enjoy a modern, real-time chat experience like you would expect!
✅ Document-Based RAG – Upload documents to enhance retrieval-augmented generation (RAG) - feed your knowledge to the model.
✅ Model Management & Customization – Swap models,pull models, create you own by adjusting parameters like temperature or context size
✅ Chat Storage – Securely save conversations in a database and resume them anytime.
✅ Prompt Library – Store frequently used prompts for quick access.
✅ Personalized Settings – Define your name and default system prompts.
✅ Voice support – Tired of typing? Full voice support included!
✅ Light/Dark themes – Supports both light and dark theme to suit your style!
✅ Responsive Design – Use the app on all devices no matter the screen size!
Project Shard consists of several parts:
- App - A modern GUI interface built with Next.js.
- Database - PostgreSQL database with vector support to store all your data
- Ollama API - Ollama API to interact with local models
| Component | Technology |
|---|---|
| App | Next.js + TypeScript |
| Database | Drizzle ORM + PostgreSQL + PgVector |
| Styling | Tailwind CSS |
To get started with development, follow these steps:
-
Prerequisites:
- Ensure the Ollama API is installed.
- Install Docker to set up the PostgreSQL database.
- Dont forget to run Drizzle migrations via
npx drizzle-kit migratecommand too setup your DB tables
-
Configuration:
- Create a
.envfile based on the providedenv.exampletemplate and configure it with your settings.
- Create a
-
Setup and Run:
- Run
npm installto install the required dependencies. - Start the application by running
npm run dev.
- Run
This project is licensed under the MIT License.
If you’d like to contribute to the development of Project Shard, feel free to fork the repository and submit pull requests. We welcome suggestions and improvements!
To ensure a smooth collaboration process, please create a separate branch for your changes rather than merging directly into master. This helps us review and integrate contributions without disrupting the main branch.
🚧 Work in Progress