llama-cpp-alternative

Here are 3 public repositories matching this topic...

brontoguana / krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

transformer inference-engine inference-optimization mixture-of-experts cpu-inference large-language-models gpu-inference llm-inference high-performance-inference hybrid-inference gguf-model-support llama-cpp-alternative

Updated Aug 6, 2026
C++

konjoai / squish

Star

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

Updated Aug 3, 2026
Python

mithraeums / hako

Star

C inference engine; own loader, kernels, and tokenizer. Runs hako-sho (3B) and hako-koi (7B) on CPU. No llama.cpp, no ollama.

machine-learning tokenizer c99 simd transformer avx2 quantization no-dependencies inference-engine hako cpu-inference llm local-llm qwen llama-cpp-alternative mithraeum q4k ggml-alternative

Updated Jul 26, 2026
C

Improve this page

Add a description, image, and links to the llama-cpp-alternative topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the llama-cpp-alternative topic, visit your repo's landing page and select "manage topics."

Learn more