Run a Hugging Face model locally with AI
Choose an available GGUF variant for your machine, check the actual filename on the Hub and try it with llama.cpp through a local command or server.
To run a Hugging Face model locally with AI, choose the actual GGUF file before building the launch command. The huggingface-local-models skill searches for llama.cpp-compatible repositories and checks the available filenames and quantisation labels.
A small workstation may need a lighter version than a machine with ample GPU memory. Tell the agent which hardware you have and whether you want a one-off command or a local service for an application. A brief local prompt is a practical way to test the selection.
What it does
- Find a model repository that supports llama.cpp.
- Compare the GGUF variants that are actually present.
- Check an exact filename when the naming is unusual.
- Choose between a local CLI and a local server.
- Send a small request to check the server response.
Install and use
- Install the skill and the official llama.cpp tools through a documented package manager. Sign in to Hugging Face only if the chosen model requires authorised access.
- Ask “Find a GGUF variant for this laptop and show me the exact file before setting up a local test.”
- Check disk and memory requirements, download the selected variant and submit a short prompt locally before integrating your application.
FAQ
Source
Made by Hugging Face. Original skill: huggingface-local-models, published on GitHub under the Apache-2.0 licence.
View on GitHub