gf-lt/docs/config.md

Configuration Guide

This document explains how to set up and configure the application using the config.toml file. The configuration file controls various aspects of the application including API endpoints, roles, RAG settings, TTS/STT features, and more.

Getting Started

1. Copy the example configuration file:

   cp config.example.toml config.toml
   

2. Edit the config.toml file to match your requirements.

Configuration Options

API Settings

llama.cpp

• ChatAPI: The endpoint for chat completions API. This is the primary API used for chat interactions.
• CompletionAPI: The endpoint for completion API. Used as an alternative to the chat API.

FetchModelNameAPI ("http://localhost:8080/v1/models")

• The endpoint to fetch available models from the API provider.

DeepSeek Settings

• DeepSeekChatAPI: The endpoint for DeepSeek chat API. Default: "https://api.deepseek.com/chat/completions"
• DeepSeekCompletionAPI: The endpoint for DeepSeek completion API. Default: "https://api.deepseek.com/beta/completions"
• DeepSeekModel: The model to use with DeepSeek API. Default: "deepseek-reasoner"
• DeepSeekToken: Your DeepSeek API token. Uncomment and set this value to enable DeepSeek features.

OpenRouter Settings

• OpenRouterChatAPI: The endpoint for OpenRouter chat API. Default: "https://openrouter.ai/api/v1/chat/completions"
• OpenRouterCompletionAPI: The endpoint for OpenRouter completion API. Default: "https://openrouter.ai/api/v1/completions"
• OpenRouterToken: Your OpenRouter API token. Uncomment and set this value to enable OpenRouter features.

Role Settings

UserRole ("user")

• The role identifier for user messages in the conversation.

ToolRole ("tool")

• The role identifier for tool responses in the conversation.

AssistantRole ("assistant")

• The role identifier for assistant responses in the conversation.

Display and Logging Settings

ShowSys (true)

• Whether to show system and tool messages in the chat interface.

LogFile ("log.txt")

• The file path where application logs will be stored.

SysDir ("sysprompts")

• Directory containing system prompt templates (character cards).

Content and Performance Settings

ChunkLimit (100000)

• Maximum size of text chunks to recieve per request from llm provider. Mainly exists to prevent infinite spam of random or repeated tokens when model starts hallucinating.

AutoScrollEnabled (true)

• Whether to automatically scroll chat window while llm streams its repsonse.

AutoCleanToolCallsFromCtx (false)

• Whether to automatically clean tool calls from the conversation context to manage token usage.

RAG (Retrieval Augmented Generation) Settings

EmbedURL ("http://localhost:8082/v1/embeddings")

• The endpoint for embedding API, used for RAG (Retrieval Augmented Generation) functionality.

RAGEnabled (false)

• Enable or disable RAG functionality for enhanced context retrieval.

RAGBatchSize (1)

• Number of documents to process in each RAG batch.

RAGWordLimit (80)

• Maximum number of words in a batch to tokenize and store.

RAGWorkers (2)

• Number of concurrent workers for RAG processing.

RAGDir ("ragimport")

• Directory containing documents for RAG processing.

HFToken ("")

• Hugging Face token for accessing models and embeddings. In case your embedding model is hosted on hf.

Text-to-Speech (TTS) Settings

TTS_ENABLED (false)

• Enable or disable text-to-speech functionality.

TTS_URL ("http://localhost:8880/v1/audio/speech")

• The endpoint for TTS API (used with kokoro provider).

TTS_SPEED (1.2)

• Playback speed for speech output (1.0 is normal speed).

TTS_PROVIDER ("kokoro")

• TTS provider to use. Options: "kokoro" or "google".
  • "kokoro": Uses Kokoro FastAPI TTS server (requires TTS_URL to be set). Provides high-quality voice synthesis but requires a running Kokoro server.
  • "google": Uses Google Translate TTS with gopxl/beep for local playback. Works offline using Google's public TTS API with local audio playback via gopxl/beep. Supports multiple languages via TTS_LANGUAGE setting.

TTS_LANGUAGE ("en")

• Language code for TTS (used with google provider).
  • Examples: "en" (English), "es" (Spanish), "fr" (French)
  • See Google Translate TTS documentation for supported languages.

Speech-to-Text (STT) Settings

STT_ENABLED (false)

• Enable or disable speech-to-text functionality.

STT_TYPE ("WHISPER_SERVER")

• Type of STT engine to use. Options are "WHISPER_SERVER" or "WHISPER_BINARY". Whisper server is used inside of docker continer, while binary can be local.

STT_URL ("http://localhost:8081/inference")

• The endpoint for STT API (used with WHISPER_SERVER).

WhisperBinaryPath ("./batteries/whisper.cpp/build/bin/whisper-cli")

• Path to the whisper binary (used with WHISPER_BINARY mode).

WhisperModelPath ("./batteries/whisper.cpp/ggml-large-v3-turbo-q5_0.bin")

• Path to the whisper model file (used with WHISPER_BINARY mode).

STT_LANG ("en")

• Language for speech recognition (used with WHISPER_BINARY mode).

STT_SR (16000)

• Sample rate for mic recording.

Database and File Settings

DBPATH ("gflt.db")

• Path to the SQLite database file used for storing conversation history and other data.

FilePickerDir (".")

• Directory where the file (image) picker should start when selecting files.

FilePickerExts ("png,jpg,jpeg,gif,webp")

• Comma-separated list of allowed file extensions for the file picker.

Additional Features

Those could be switched in program, but also bould be setup in config.

ToolUse

• Enable or disable explanation of tools to llm, so it could use them.

ThinkUse

• Enable or disable insertion of token at the beggining of llm resp.

Environment Variables

The application supports using environment variables for API keys as fallbacks:

• OPENROUTER_API_KEY: Used if OpenRouterToken is not set in the config
• DEEPSEEK_API_KEY: Used if DeepSeekToken is not set in the config