grail/gf-lt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Doc: update

Browse Source
This commit is contained in:
Grail Finder
2026-01-06 09:43:39 +03:00
parent 0ed6b240d9
commit 27b732f3ce
2 changed files with 63 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

188
docs/config.md
View File

@@ -19,10 +19,8 @@ This document explains how to set up and configure the application using the `co
- **ChatAPI**: The endpoint for chat completions API. This is the primary API used for chat interactions. - **ChatAPI**: The endpoint for chat completions API. This is the primary API used for chat interactions.
- **CompletionAPI**: The endpoint for completion API. Used as an alternative to the chat API. - **CompletionAPI**: The endpoint for completion API. Used as an alternative to the chat API.
#### FetchModelNameAPI #### FetchModelNameAPI (`"http://localhost:8080/v1/models"`)
- **Type**: String - The endpoint to fetch available models from the API provider.
- **Default**: `"http://localhost:8080/v1/models"`
- **Description**: The endpoint to fetch available models from the API provider.
#### DeepSeek Settings #### DeepSeek Settings
- **DeepSeekChatAPI**: The endpoint for DeepSeek chat API. Default: `"https://api.deepseek.com/chat/completions"` - **DeepSeekChatAPI**: The endpoint for DeepSeek chat API. Default: `"https://api.deepseek.com/chat/completions"`
@@ -37,177 +35,115 @@ This document explains how to set up and configure the application using the `co
### Role Settings ### Role Settings
#### UserRole #### UserRole (`"user"`)
- **Type**: String - The role identifier for user messages in the conversation.
- **Default**: `"user"`
- **Description**: The role identifier for user messages in the conversation.
#### ToolRole #### ToolRole (`"tool"`)
- **Type**: String - The role identifier for tool responses in the conversation.
- **Default**: `"tool"`
- **Description**: The role identifier for tool responses in the conversation.
#### AssistantRole #### AssistantRole (`"assistant"`)
- **Type**: String - The role identifier for assistant responses in the conversation.
- **Default**: `"assistant"`
- **Description**: The role identifier for assistant responses in the conversation.
### Display and Logging Settings ### Display and Logging Settings
#### ShowSys #### ShowSys (`true`)
- **Type**: Boolean - Whether to show system and tool messages in the chat interface.
- **Default**: `true`
- **Description**: Whether to show system and tool messages in the chat interface.
#### LogFile #### LogFile (`"log.txt"`)
- **Type**: String - The file path where application logs will be stored.
- **Default**: `"log.txt"`
- **Description**: The file path where application logs will be stored.
#### SysDir #### SysDir (`"sysprompts"`)
- **Type**: String - Directory containing system prompt templates (character cards).
- **Default**: `"sysprompts"`
- **Description**: Directory containing system prompt templates (character cards).
### Content and Performance Settings ### Content and Performance Settings
#### ChunkLimit #### ChunkLimit (`100000`)
- **Type**: Integer - Maximum size of text chunks to recieve per request from llm provider. Mainly exists to prevent infinite spam of random or repeated tokens when model starts hallucinating.
- **Default**: `100000`
- **Description**: Maximum size of text chunks to recieve per request from llm provider. Mainly exists to prevent infinite spam of random or repeated tokens when model starts hallucinating.
#### AutoScrollEnabled #### AutoScrollEnabled (`true`)
- **Type**: Boolean - Whether to automatically scroll chat window while llm streams its repsonse.
- **Default**: `true`
- **Description**: Whether to automatically scroll chat window while llm streams its repsonse.
#### AutoCleanToolCallsFromCtx #### AutoCleanToolCallsFromCtx (`false`)
- **Type**: Boolean - Whether to automatically clean tool calls from the conversation context to manage token usage.
- **Default**: `false` (commented out)
- **Description**: Whether to automatically clean tool calls from the conversation context to manage token usage.
### RAG (Retrieval Augmented Generation) Settings ### RAG (Retrieval Augmented Generation) Settings
#### EmbedURL #### EmbedURL (`"http://localhost:8082/v1/embeddings"`)
- **Type**: String - The endpoint for embedding API, used for RAG (Retrieval Augmented Generation) functionality.
- **Default**: `"http://localhost:8082/v1/embeddings"`
- **Description**: The endpoint for embedding API, used for RAG (Retrieval Augmented Generation) functionality.
#### RAGEnabled #### RAGEnabled (`false`)
- **Type**: Boolean - Enable or disable RAG functionality for enhanced context retrieval.
- **Default**: Not set in example (false by default)
- **Description**: Enable or disable RAG functionality for enhanced context retrieval.
#### RAGBatchSize #### RAGBatchSize (`1`)
- **Type**: Integer - Number of documents to process in each RAG batch.
- **Default**: `1`
- **Description**: Number of documents to process in each RAG batch.
#### RAGWordLimit #### RAGWordLimit (`80`)
- **Type**: Integer - Maximum number of words in a batch to tokenize and store.
- **Default**: `80`
- **Description**: Maximum number of words to include in RAG context.
#### RAGWorkers #### RAGWorkers (`2`)
- **Type**: Integer - Number of concurrent workers for RAG processing.
- **Default**: `2`
- **Description**: Number of concurrent workers for RAG processing.
#### RAGDir #### RAGDir (`"ragimport"`)
- **Type**: String - Directory containing documents for RAG processing.
- **Default**: `"ragimport"`
- **Description**: Directory containing documents for RAG processing.
#### HFToken #### HFToken (`""`)
- **Type**: String - Hugging Face token for accessing models and embeddings. In case your embedding model is hosted on hf.
- **Default**: Not set in example
- **Description**: Hugging Face token for accessing models and embeddings. In case your embedding model is hosted on hf.
### Text-to-Speech (TTS) Settings ### Text-to-Speech (TTS) Settings
#### TTS_ENABLED #### TTS_ENABLED (`false`)
- **Type**: Boolean - Enable or disable text-to-speech functionality.
- **Default**: `false`
- **Description**: Enable or disable text-to-speech functionality.
#### TTS_URL #### TTS_URL (`"http://localhost:8880/v1/audio/speech"`)
- **Type**: String - The endpoint for TTS API.
- **Default**: `"http://localhost:8880/v1/audio/speech"`
- **Description**: The endpoint for TTS API.
#### TTS_SPEED #### TTS_SPEED (`1.2`)
- **Type**: Float - Playback speed for speech output (1.0 is normal speed).
- **Default**: `1.2`
- **Description**: Playback speed for speech output (1.0 is normal speed).
### Speech-to-Text (STT) Settings ### Speech-to-Text (STT) Settings
#### STT_ENABLED #### STT_ENABLED (`false`)
- **Type**: Boolean - Enable or disable speech-to-text functionality.
- **Default**: `false`
- **Description**: Enable or disable speech-to-text functionality.
#### STT_TYPE #### STT_TYPE (`"WHISPER_SERVER"`)
- **Type**: String - Type of STT engine to use. Options are `"WHISPER_SERVER"` or `"WHISPER_BINARY"`. Whisper server is used inside of docker continer, while binary can be local.
- **Default**: `"WHISPER_SERVER"`
- **Description**: Type of STT engine to use. Options are `"WHISPER_SERVER"` or `"WHISPER_BINARY"`. Whisper server is used inside of docker continer, while binary can be local.
#### STT_URL #### STT_URL (`"http://localhost:8081/inference"`)
- **Type**: String - The endpoint for STT API (used with WHISPER_SERVER).
- **Default**: `"http://localhost:8081/inference"`
- **Description**: The endpoint for STT API (used with WHISPER_SERVER).
#### WhisperBinaryPath #### WhisperBinaryPath (`"./batteries/whisper.cpp/build/bin/whisper-cli"`)
- **Type**: String - Path to the whisper binary (used with WHISPER_BINARY mode).
- **Default**: `"./batteries/whisper.cpp/build/bin/whisper-cli"`
- **Description**: Path to the whisper binary (used with WHISPER_BINARY mode).
#### WhisperModelPath #### WhisperModelPath (`"./batteries/whisper.cpp/ggml-large-v3-turbo-q5_0.bin"`)
- **Type**: String - Path to the whisper model file (used with WHISPER_BINARY mode).
- **Default**: `"./batteries/whisper.cpp/ggml-large-v3-turbo-q5_0.bin"`
- **Description**: Path to the whisper model file (used with WHISPER_BINARY mode).
#### STT_LANG #### STT_LANG (`"en"`)
- **Type**: String - Language for speech recognition (used with WHISPER_BINARY mode).
- **Default**: `"en"`
- **Description**: Language for speech recognition (used with WHISPER_BINARY mode).
#### STT_SR #### STT_SR (`16000`)
- **Type**: Integer - Sample rate for mic recording.
- **Default**: `16000`
- **Description**: Sample rate for mic recording.
### Database and File Settings ### Database and File Settings
#### DBPATH #### DBPATH (`"gflt.db"`)
- **Type**: String - Path to the SQLite database file used for storing conversation history and other data.
- **Default**: `"gflt.db"`
- **Description**: Path to the SQLite database file used for storing conversation history and other data.
#### FilePickerDir #### FilePickerDir (`"."`)
- **Type**: String - Directory where the file (image) picker should start when selecting files.
- **Default**: `"."`
- **Description**: Directory where the file (image) picker should start when selecting files.
#### FilePickerExts #### FilePickerExts (`"png,jpg,jpeg,gif,webp"`)
- **Type**: String - Comma-separated list of allowed file extensions for the file picker.
- **Default**: `"png,jpg,jpeg,gif,webp"`
- **Description**: Comma-separated list of allowed file extensions for the file picker.
### Additional Features ### Additional Features
Those could be switched in program, but also bould be setup in config. Those could be switched in program, but also bould be setup in config.
#### ToolUse #### ToolUse
- **Type**: Boolean - Enable or disable explanation of tools to llm, so it could use them.
- **Default**: Not set in example (false by default)
- **Description**: Enable or disable explanation of tools to llm, so it could use them.
#### ThinkUse #### ThinkUse
- **Type**: Boolean - Enable or disable insertion of <think> token at the beggining of llm resp.
- **Default**: Not set in example (false by default)
- **Description**: Enable or disable insertion of <think> token at the beggining of llm resp.
## Environment Variables ## Environment Variables

2
docs/tutorial_rp.md
View File

@@ -1,5 +1,5 @@
after ![installing](linktoinstruction) after ![installing](linktoinstruction)
![set up your config](link) ![set up your config](config.md)
To roleplay, we would need to create a character card or get one from the web. To roleplay, we would need to create a character card or get one from the web.
For this tutorial, we are going to use the default character Seraphina from [SillyTavern (ST)](https://github.com/SillyTavern/SillyTavern/blob/release/default/content/default_Seraphina.png). For this tutorial, we are going to use the default character Seraphina from [SillyTavern (ST)](https://github.com/SillyTavern/SillyTavern/blob/release/default/content/default_Seraphina.png).