Features · Modes · Models · Install · Configuration · Text · Images · Audio · RAG · Search · Prompts · Data · Requirements

Loca is a python application for running local GGUF-based LLMs for text generation, document-grounded retrieval, semantic search, prompt engineering, multimodal workflow staging, function-calling experimentation, guarded web context ingestion, and SQLite-backed data management. It is designed to run local language models through llama-cpp-python while giving analysts direct control over model selection, inference parameters, prompt templates, document context, retrieval behavior, semantic chunking, local data assets, and model-specific capability gates.

The application is organized around named local assistants such as Bro, Gipity, Buddy, Boo, Jimi, Leeroy, and Nisty. Each assistant is configured through config.py, where its GGUF path, logo, base model metadata, description, and allowed application modes are declared.

LocaLlama is built around a practical local-AI workflow:

• Select a local assistant model and load its configured GGUF path.
• Run instruction-following, reasoning, coding, extraction, translation, and summarization tasks.
• Upload PDF, TXT, or DOCX documents for local document question answering.
• Build semantic indexes from uploaded documents using sentence-transformer embeddings.
• Route retrieved document or semantic chunks into Text Generation.
• Stage image and audio workflows for models that advertise multimodal capability.
• Generate and validate app-mediated function calls.
• Fetch public web pages into bounded local context for Gipity-driven analysis.
• Store chat history, prompts, embeddings, document metadata, chunks, image metadata, and imported data in SQLite.
• Use Prompt Engineering mode to create, edit, clone, search, and apply prompt templates.
• Use Data Management mode to import Excel data, browse tables, run guarded SQL, profile data, and administer local AI-asset tables.

The current application mode contract is:

Model availability is controlled by local environment-variable paths in config.py. A model appears in the application when its configured path is present and the selected registry entry advertises one or more supported modes.

Streamlit UI
    │
    ├── Sidebar Model Registry
    │       ├── config.py MODEL_REGISTRY
    │       ├── environment-variable GGUF paths
    │       ├── selected model state
    │       ├── selected mode state
    │       └── model capability gates
    │
    ├── Text Generation
    │       ├── llama-cpp-python
    │       ├── selected local GGUF model
    │       ├── chat_history table
    │       ├── Prompts table
    │       ├── system instructions
    │       ├── task-specific instruction block
    │       ├── thinking / coding / function-calling prompt extensions
    │       └── Gipity tool and web-context controls
    │
    ├── Images API
    │       ├── image upload
    │       ├── image preview
    │       ├── model capability gate
    │       ├── runtime adapter gate
    │       ├── optional analyze_image_with_model adapter
    │       └── image context routing into Text Generation
    │
    ├── Audio API
    │       ├── audio upload
    │       ├── audio preview
    │       ├── model capability gate
    │       ├── runtime adapter gate
    │       ├── optional analyze_audio_with_model adapter
    │       └── audio context routing into Text Generation
    │
    ├── Document Q&A
    │       ├── uploaded PDFs / TXT / DOCX
    │       ├── PyMuPDF / text extraction
    │       ├── sentence-transformers embeddings
    │       ├── sqlite-vec vector table when available
    │       ├── cosine fallback search
    │       └── compact Buddy retrieval profile when Buddy is selected
    │
    ├── Semantic Search
    │       ├── chunked uploaded documents
    │       ├── embeddings table
    │       ├── vector diagnostics
    │       ├── selected semantic context
    │       └── context routing into Text Generation / Document Q&A
    │
    ├── Prompt Engineering
    │       ├── Prompts table
    │       ├── template metadata
    │       ├── starter prompt generation
    │       └── system-instruction cascade
    │
    ├── Gipity Tools
    │       ├── strict JSON function-call generation
    │       ├── JSON extraction and normalization
    │       ├── allowlisted function execution
    │       ├── guarded public HTTP/HTTPS web fetch
    │       └── tool-grounded final answer generation
    │
    └── Data Management
            ├── Excel import
            ├── CRUD / profiling / visualization
            ├── guarded SQL console
            └── AI asset governance tables

LocaLlama/
├─ app.py                         # Main Streamlit application
├─ config.py                      # App constants, modes, model registry, paths, logos, help text
├─ requirements.txt               # Python dependencies
├─ resources/
│  └─ images/
│     ├─ loca_project.png
│     ├─ LocaLlama-streamlit.gif
│     ├─ loca-llama_logo.png
│     ├─ bro_logo.png
│     ├─ gipity_logo.png
│     ├─ buddy_logo.png
│     ├─ boo_logo.png
│     ├─ jimi_logo.png
│     ├─ leeroy_logo.png
│     └─ nisty_logo.png
├─ stores/
│  └─ sqlite/
│     └─ loca.db                  # Chat history, prompts, embeddings, documents, chunks, images
├─ models/
│  └─ *.gguf                      # Optional local model directory depending on environment variables
└─ README.md

git clone https://github.com/is-leeroy-jenkins/LocaLlama.git
cd LocaLlama

python -m venv .venv
.venv\Scripts\Activate.ps1

python -m venv .venv
.venv\Scripts\activate.bat

python3 -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

LocaLlama expects local GGUF model paths to be supplied through environment variables and consumed by config.py.

Example on Windows PowerShell:

$env:BRO_LLM_PATH="C:\Users\you\models\bro-3-4b-it-Q4_K_M.gguf"
$env:GIPITY_LLM_PATH="C:\Users\you\models\gpt-oss-20b-Q4_K_M.gguf"
$env:BUDDY_LLM_PATH="C:\Users\you\models\buddy-gemma-3-270m-it-Q4_K_M.gguf"
$env:BOO_LLM_PATH="C:\Users\you\models\boo-phi-4-mini-instruct-Q4_K_M.gguf"
$env:JIMI_LLM_PATH="C:\Users\you\models\jimi-gemma-4-e4b-it-Q4_K_M.gguf"
$env:LEEROY_LLM_PATH="C:\Users\you\models\leeroy-llama-3.2-1b-instruct-Q4_K_M.gguf"
$env:NISTY_LLM_PATH="C:\Users\you\models\nisty-gemma-4-e4b-it-Q4_K_M.gguf"

Restart the terminal or IDE after changing environment variables.

Run Streamlit through Python so the correct virtual environment is used.

python -m streamlit run app.py

The application opens in wide layout, renders the LocaLlama subtitle, and exposes the model and mode selectors in the sidebar.

LocaLlama reads runtime configuration from config.py and Streamlit session state.

Image and audio modes are model-gated and runtime-gated. The mode may be visible for Jimi or Nisty, but actual multimodal inference requires an image/audio-capable local adapter to be wired into app.py.

Optional config flags may be added later:

IMAGE_RUNTIME_AVAILABLE = True
AUDIO_RUNTIME_AVAILABLE = True
WEB_RUNTIME_AVAILABLE = True
MULTIMODAL_RUNTIME_NAME = 'custom-local-adapter'

Without those runtime flags and adapter functions, Image Mode and Audio Mode fail closed with a clear status message instead of raising a runtime error.

Text Generation is the primary local chat interface. It uses chat history, system instructions, task-specific prompt blocks, optional semantic context, optional document context, and model-specific capability gates to construct a local model prompt.

Images API stages image analysis workflows for models configured with image capability, currently Jimi and Nisty. The mode is designed to be safe even when the local runtime is still text-only.

Optional adapter signatures:

def analyze_image_with_model( model_path: str, model_name: str, image_bytes: bytes,
	image_name: str, prompt: str ) -> str:

or:

def analyze_image_with_model( image_bytes: bytes, prompt: str ) -> str:

Audio API stages transcription, translation, summarization, and audio-analysis workflows for models configured with audio capability, currently Jimi and Nisty. The mode is safe to expose before a true audio runtime adapter is wired.

Optional adapter signatures:

def analyze_audio_with_model( model_path: str, model_name: str, audio_bytes: bytes,
	audio_name: str, prompt: str ) -> str:

or:

def analyze_audio_with_model( audio_bytes: bytes, prompt: str ) -> str:

Document Q&A provides local retrieval-augmented answering over uploaded files. The app accepts PDF, TXT, and DOCX uploads, extracts text, chunks documents, indexes chunks, retrieves relevant excerpts, and routes document-grounded prompts through the same local generation pipeline.

Buddy is configured for Document Q&A and Semantic Search, but it uses a smaller retrieval profile because it is a compact 270M model.

Semantic Search builds a reusable semantic index from uploaded PDF, TXT, or DOCX files. It stores chunk vectors in the local SQLite embeddings table and allows ranked semantic retrieval.

Semantic Search is app-level retrieval infrastructure. The selected LLM does not perform embedding generation directly. Instead, embeddings are created by the configured embedding layer, and the selected assistant model reads or reasons over retrieved chunks.

Function Calling and Web Browsing are app-mediated workflows. The model can request a function call, but LocaLlama validates, normalizes, and executes only allowlisted functions.

Allowlisted functions:

Example function-call JSON:

{
	"name": "web_browse_url",
	"arguments": {
		"url": "https://example.com",
		"prompt": "Summarize the page in five bullets.",
		"allowed_domain": "example.com",
		"max_chars": 12000
	}
}

Prompt Engineering manages reusable prompt templates in SQLite. It can search, sort, page, edit, clone, generate, and cascade prompts into other modes.

Data Management operates on the local SQLite database and exposes both general table operations and AI asset governance workflows.

The application initializes core local tables when the database is created.

LocaLlama renders a fixed footer showing the current operating state.

The table below reflects the active imports and runtime features used by the current application. Use requirements.txt as the installation source of truth when version pins are present.

This application is provided for personal, research, and open-source use. Refer to the project and model repositories for application and model-specific licensing terms.