154 lines
5.4 KiB
Markdown
154 lines
5.4 KiB
Markdown
# Configuration Guide
|
|
|
|
This document explains how to set up and configure the application using the `config.toml` file. The configuration file controls various aspects of the application including API endpoints, roles, RAG settings, TTS/STT features, and more.
|
|
|
|
## Getting Started
|
|
|
|
1. Copy the example configuration file:
|
|
```bash
|
|
cp config.example.toml config.toml
|
|
```
|
|
|
|
2. Edit the `config.toml` file to match your requirements.
|
|
|
|
## Configuration Options
|
|
|
|
### API Settings
|
|
|
|
#### llama.cpp
|
|
- **ChatAPI**: The endpoint for chat completions API. This is the primary API used for chat interactions.
|
|
- **CompletionAPI**: The endpoint for completion API. Used as an alternative to the chat API.
|
|
|
|
#### FetchModelNameAPI (`"http://localhost:8080/v1/models"`)
|
|
- The endpoint to fetch available models from the API provider.
|
|
|
|
#### DeepSeek Settings
|
|
- **DeepSeekChatAPI**: The endpoint for DeepSeek chat API. Default: `"https://api.deepseek.com/chat/completions"`
|
|
- **DeepSeekCompletionAPI**: The endpoint for DeepSeek completion API. Default: `"https://api.deepseek.com/beta/completions"`
|
|
- **DeepSeekModel**: The model to use with DeepSeek API. Default: `"deepseek-reasoner"`
|
|
- **DeepSeekToken**: Your DeepSeek API token. Uncomment and set this value to enable DeepSeek features.
|
|
|
|
#### OpenRouter Settings
|
|
- **OpenRouterChatAPI**: The endpoint for OpenRouter chat API. Default: `"https://openrouter.ai/api/v1/chat/completions"`
|
|
- **OpenRouterCompletionAPI**: The endpoint for OpenRouter completion API. Default: `"https://openrouter.ai/api/v1/completions"`
|
|
- **OpenRouterToken**: Your OpenRouter API token. Uncomment and set this value to enable OpenRouter features.
|
|
|
|
### Role Settings
|
|
|
|
#### UserRole (`"user"`)
|
|
- The role identifier for user messages in the conversation.
|
|
|
|
#### ToolRole (`"tool"`)
|
|
- The role identifier for tool responses in the conversation.
|
|
|
|
#### AssistantRole (`"assistant"`)
|
|
- The role identifier for assistant responses in the conversation.
|
|
|
|
### Display and Logging Settings
|
|
|
|
#### ShowSys (`true`)
|
|
- Whether to show system and tool messages in the chat interface.
|
|
|
|
#### LogFile (`"log.txt"`)
|
|
- The file path where application logs will be stored.
|
|
|
|
#### SysDir (`"sysprompts"`)
|
|
- Directory containing system prompt templates (character cards).
|
|
|
|
### Content and Performance Settings
|
|
|
|
#### ChunkLimit (`100000`)
|
|
- Maximum size of text chunks to recieve per request from llm provider. Mainly exists to prevent infinite spam of random or repeated tokens when model starts hallucinating.
|
|
|
|
#### AutoScrollEnabled (`true`)
|
|
- Whether to automatically scroll chat window while llm streams its repsonse.
|
|
|
|
#### AutoCleanToolCallsFromCtx (`false`)
|
|
- Whether to automatically clean tool calls from the conversation context to manage token usage.
|
|
|
|
### RAG (Retrieval Augmented Generation) Settings
|
|
|
|
#### EmbedURL (`"http://localhost:8082/v1/embeddings"`)
|
|
- The endpoint for embedding API, used for RAG (Retrieval Augmented Generation) functionality.
|
|
|
|
#### RAGEnabled (`false`)
|
|
- Enable or disable RAG functionality for enhanced context retrieval.
|
|
|
|
#### RAGBatchSize (`1`)
|
|
- Number of documents to process in each RAG batch.
|
|
|
|
#### RAGWordLimit (`80`)
|
|
- Maximum number of words in a batch to tokenize and store.
|
|
|
|
#### RAGWorkers (`2`)
|
|
- Number of concurrent workers for RAG processing.
|
|
|
|
#### RAGDir (`"ragimport"`)
|
|
- Directory containing documents for RAG processing.
|
|
|
|
#### HFToken (`""`)
|
|
- Hugging Face token for accessing models and embeddings. In case your embedding model is hosted on hf.
|
|
|
|
|
|
### Text-to-Speech (TTS) Settings
|
|
|
|
#### TTS_ENABLED (`false`)
|
|
- Enable or disable text-to-speech functionality.
|
|
|
|
#### TTS_URL (`"http://localhost:8880/v1/audio/speech"`)
|
|
- The endpoint for TTS API.
|
|
|
|
#### TTS_SPEED (`1.2`)
|
|
- Playback speed for speech output (1.0 is normal speed).
|
|
|
|
### Speech-to-Text (STT) Settings
|
|
|
|
#### STT_ENABLED (`false`)
|
|
- Enable or disable speech-to-text functionality.
|
|
|
|
#### STT_TYPE (`"WHISPER_SERVER"`)
|
|
- Type of STT engine to use. Options are `"WHISPER_SERVER"` or `"WHISPER_BINARY"`. Whisper server is used inside of docker continer, while binary can be local.
|
|
|
|
#### STT_URL (`"http://localhost:8081/inference"`)
|
|
- The endpoint for STT API (used with WHISPER_SERVER).
|
|
|
|
#### WhisperBinaryPath (`"./batteries/whisper.cpp/build/bin/whisper-cli"`)
|
|
- Path to the whisper binary (used with WHISPER_BINARY mode).
|
|
|
|
#### WhisperModelPath (`"./batteries/whisper.cpp/ggml-large-v3-turbo-q5_0.bin"`)
|
|
- Path to the whisper model file (used with WHISPER_BINARY mode).
|
|
|
|
#### STT_LANG (`"en"`)
|
|
- Language for speech recognition (used with WHISPER_BINARY mode).
|
|
|
|
#### STT_SR (`16000`)
|
|
- Sample rate for mic recording.
|
|
|
|
### Database and File Settings
|
|
|
|
#### DBPATH (`"gflt.db"`)
|
|
- Path to the SQLite database file used for storing conversation history and other data.
|
|
|
|
#### FilePickerDir (`"."`)
|
|
- Directory where the file (image) picker should start when selecting files.
|
|
|
|
#### FilePickerExts (`"png,jpg,jpeg,gif,webp"`)
|
|
- Comma-separated list of allowed file extensions for the file picker.
|
|
|
|
### Additional Features
|
|
|
|
Those could be switched in program, but also bould be setup in config.
|
|
|
|
#### ToolUse
|
|
- Enable or disable explanation of tools to llm, so it could use them.
|
|
|
|
#### ThinkUse
|
|
- Enable or disable insertion of <think> token at the beggining of llm resp.
|
|
|
|
## Environment Variables
|
|
|
|
The application supports using environment variables for API keys as fallbacks:
|
|
|
|
- `OPENROUTER_API_KEY`: Used if `OpenRouterToken` is not set in the config
|
|
- `DEEPSEEK_API_KEY`: Used if `DeepSeekToken` is not set in the config
|