Transforming My Homelab with a Local LLM
Managing a homelab can be both rewarding and challenging, especially when you have various services like Docker
containers, Home Assistant, network storage, and automated backups running simultaneously. Traditionally, each
service demands its own dashboard or direct machine access for log checks and management. But, what if a local
Language Learning Model (LLM) could streamline this process? Inspired by this possibility, I integrated Ollama
to run the model and Open WebUI as the chat interface, effectively connecting the model to Home Assistant
through its MCP server. Self-hosted n8n workflows took charge of actions related to Docker, storage, and
networked machines.
Local LLM Capabilities
The local model’s capabilities are defined by the tools and workflows I have made available. Its tasks include
checking system health, reading container logs, restarting approved services, monitoring storage, and verifying
backups. Importantly, all operations are conducted locally, ensuring data privacy and security.
I Gave the Model Access Through MCP
Open WebUI Acts as the Main Interface
In my setup, the Open WebUI serves as the primary interface. It connects with Ollama for local models and
supports MCP servers, which facilitate access to external tools within a conversation. The model gets a list of
these tools, complete with descriptions and accepted parameters. It autonomously selects and deploys the
appropriate tool based on the task, using the returned information for responses.
Home Assistant’s built-in MCP server plays a crucial role, providing the model with the current state of exposed
entities and enabling control through Home Assistant’s Assist API. This includes devices, sensors, media
players, and automations, with configurable access to specific entities. This flexibility allows for read-only
access if desired.
The n8n workflows manage Docker, storage, backups, and other networked machine tasks. Each workflow is
dedicated to a specific task, like listing unhealthy containers, retrieving logs, or confirming backup
completion. The model passes necessary information to the workflow, which then executes the command.
data-img-url=”https://static0.xdaimages.com/wordpress/wp-content/uploads/wm/2026/05/1778940304206.jpg?q=49&fit=crop&w=220&h=124&dpr=2″
src=”https://static0.xdaimages.com/wordpress/wp-content/uploads/wm/2026/05/1778940304206.jpg?q=49&fit=crop&w=220&h=124&dpr=2″ />
Related
My local LLM can call Claude when it’s stuck, and it changed everything about my local-first setup
Local LLMs aren’t very good on their own
Qwen 3.5 4B Handles Most of the Routine Work
It Supports Tool Calling and Is Small Enough to Run Alongside Ollama
After evaluating several local LLMs, I opted for Qwen 3.5 4B, which supports tool calling and is compact enough
to operate alongside Ollama and Open WebUI on an 8GB RAM laptop. It’s highly suited for admin tasks that
typically require dashboard access or machine connections. It handles requests such as “show me which services
are unhealthy,” “check how much storage remains on the NAS,” or “confirm whether an automated backup has been
completed.”
Qwen is adept at combining multiple actions within a single conversation. For instance, if Jellyfin is
unavailable, it can first verify the container status and then fetch recent logs. If the container is stopped,
it invokes the restart workflow and re-checks its status. Each step is managed by individual n8n workflows.
Home Assistant further enhances control capabilities. Qwen can examine the current states of exposed devices and
sensors, trigger automations, and manage supported devices—all from the same chat. This means a request can
engage multiple parts of the homelab, like checking a media service’s status and managing the device without
switching between interfaces.
The main limitation is ensuring clear tool descriptions and a manageable selection of tools. If too many
workflows have similar names, errors are more likely. To mitigate this, I focus each workflow on one task and
use descriptive names.
I Make Sure Nothing Leaves My Local Network
I Disabled Cloud Features and Self-hosted Everything
alt=”Home Assistant update helped stop complicating automations”
data-img-url=”https://static0.xdaimages.com/wordpress/wp-content/uploads/wm/2026/08/home-assistant-update-helped-stop-complicating-automations.JPG?q=49&fit=crop&w=825&dpr=2″
src=”https://static0.xdaimages.com/wordpress/wp-content/uploads/wm/2026/08/home-assistant-update-helped-stop-complicating-automations.JPG?q=49&fit=crop&w=825&dpr=2″ />
Running Qwen through Ollama ensures model inference occurs locally on the laptop. However, achieving a fully
local setup required disabling cloud features within Open WebUI, which traditionally supports cloud models,
external search providers, and online embedding services. I configured it to utilize only the local Ollama
endpoint. Home Assistant and n8n are also self-hosted, with Open WebUI accessing them via local network
addresses.
After downloading Qwen 3.5 4B and the necessary container images, I disabled Ollama’s cloud features and blocked
external network access for all involved services. While the laptop can still communicate with Home Assistant,
the NAS, and other networked machines, it cannot send requests outside the network, safeguarding prompts,
container logs, filenames, device states, and backup data from external exposure.
There’s no cloud fallback if Qwen fails to understand a request, nor does it search the web for additional
information. This limitation is negligible because the model operates with tool descriptions and real-time data
from my services. Its role is to select the correct tool, pass parameters, and interpret results, all within the
local network.
Local AI Is a Lot More Capable Than You Think
While it’s unrealistic to expect locally hosted AI to match the performance of cloud models like Fable 5, local
AI remains highly useful for many tasks, whether at home or for personal workloads with modest computational
needs. I’ve even used a local LLM for coding alongside a cloud supervisor, achieving satisfactory results. For
further details, visit the original article
