All products

Run a Hugging Face model locally with AI

Choose an available GGUF variant for your machine, check the actual filename on the Hub and try it with llama.cpp through a local command or server.

To run a Hugging Face model locally with AI, choose the actual GGUF file before building the launch command. The huggingface-local-models skill searches for llama.cpp-compatible repositories and checks the available filenames and quantisation labels.

A small workstation may need a lighter version than a machine with ample GPU memory. Tell the agent which hardware you have and whether you want a one-off command or a local service for an application. A brief local prompt is a practical way to test the selection.

What it does

  • Find a model repository that supports llama.cpp.
  • Compare the GGUF variants that are actually present.
  • Check an exact filename when the naming is unusual.
  • Choose between a local CLI and a local server.
  • Send a small request to check the server response.

Install and use

  1. Install the skill and the official llama.cpp tools through a documented package manager. Sign in to Hugging Face only if the chosen model requires authorised access.
  2. Ask “Find a GGUF variant for this laptop and show me the exact file before setting up a local test.”
  3. Check disk and memory requirements, download the selected variant and submit a short prompt locally before integrating your application.

FAQ

Source

Made by Hugging Face. Original skill: huggingface-local-models, published on GitHub under the Apache-2.0 licence.

View on GitHub