Running a local model

Ask a model something

woodentable ai -m ~/models/model.gguf -p "Explain what a Tcl safe interpreter is"

Without -p, it starts a conversation instead. Give it files the same way as anywhere else, before ai:

woodentable -readonly ~/project ai -m ~/models/model.gguf -p "Summarise /project/README.md"

To use files and tools, choose a model that supports tool calling; one that doesn't can only answer from what it already knows.

The model runs inside the woodentable process, using llama.cpp. It listens only on this machine, behind a key made for the run, and stops when the run does.

Use your graphics card

Woodentable runs models on the CPU out of the box. For an NVIDIA, AMD or Intel graphics card add the Vulkan backend, and on Apple silicon the Metal one:

woodentable pkg install vulkan
woodentable pkg install metal

Then choose it for a run with -vulkan or -metal after ai. To see what's installed:

woodentable pkg list

Save a model setup as a profile

A runtime profile saves the model and how it runs, such as the backend and the size of its context window, under a name:

woodentable runtime add fast -m ~/models/small.gguf -c 8192
woodentable runtime list

woodentable -runtime fast ai -p "Explain what a Tcl safe interpreter is"

Go deeper with the built-in guides:

woodentable help ai
woodentable help runtime
woodentable help pkg