Skip to content

Colibrì: Running GLM-5.2 (744B) Locally in RAM With No GPU

By Fahd Mirza

10 min video·en··11809 views

This is an AI-generated summary of “Colibrì: Running GLM-5.2 (744B) Locally in RAM With No GPU” — a 10 min YouTube video by Fahd Mirza, published July 14, 2026. It condenses the full transcript into 9 key takeaways with clickable timestamps.

Summary

This video demonstrates how to run the massive 744 billion parameter GLM 5.2 Mixture of Experts model on a single machine with limited RAM by streaming model experts from disk using a custom C engine called Colibri.

Key Points

  • The video showcases running GLM 5.2, a 744 billion parameter Mixture of Experts (MoE) model, on a single machine without a cluster or multiple GPUs. 
  • This feat is accomplished using Colibri, a tiny C engine with zero dependencies that streams model components from disk on demand. 
  • The core principle leverages the fact that the model needs about 370 GB on disk but only a fraction of that (around 10 GB for dense layers plus cached experts) in RAM at any given point. 
  • As an MoE model, GLM 5.2 only activates a small subset of its 21,050 experts (e.g., 8 out of 256 per layer) for each token, rather than using all parameters simultaneously. 
  • The system's speed improves with continued use because Colibri intelligently caches frequently hit experts in RAM, leading to a higher 'expert hit rate' and faster token generation. 
  • The primary performance bottleneck for this system is the disk read speed, not the CPU or GPU, as the GPU remains largely idle during the generation process. 
  • Colibri manages memory by keeping the dense model parts and frequently accessed 'hot' experts in RAM, while streaming other necessary experts directly from disk. 
  • This streaming technique is specifically effective for Mixture of Experts models because only a small portion of the model is required per token, making it impractical for dense models. 
  • While the current Colibri implementation is handwritten and optimized for GLM 5.2's exact architecture, the underlying technique is general and holds promise for other MoE models. 
Colibrì: Running GLM-5.2 (744B) Locally in RAM With No GPU

Colibrì: Running GLM-5.2 (744B) Locally in RAM With No GPU

This video demonstrates how to run the massive 744 billion parameter GLM 5.2 Mixture of Experts model on a single machine with limited RAM by streaming model experts from disk using a custom C engine called Colibri.

Key Points

—The video showcases running GLM 5.2, a 744 billion parameter Mixture of Experts (MoE) model, on a single machine without a cluster or multiple GPUs.
—This feat is accomplished using Colibri, a tiny C engine with zero dependencies that streams model components from disk on demand.
—The core principle leverages the fact that the model needs about 370 GB on disk but only a fraction of that (around 10 GB for dense layers plus cached experts) in RAM at any given point.
—As an MoE model, GLM 5.2 only activates a small subset of its 21,050 experts (e.g., 8 out of 256 per layer) for each token, rather than using all parameters simultaneously.
—The system's speed improves with continued use because Colibri intelligently caches frequently hit experts in RAM, leading to a higher 'expert hit rate' and faster token generation.
—The primary performance bottleneck for this system is the disk read speed, not the CPU or GPU, as the GPU remains largely idle during the generation process.
—Colibri manages memory by keeping the dense model parts and frequently accessed 'hot' experts in RAM, while streaming other necessary experts directly from disk.
—This streaming technique is specifically effective for Mixture of Experts models because only a small portion of the model is required per token, making it impractical for dense models.
—While the current Colibri implementation is handwritten and optimized for GLM 5.2's exact architecture, the underlying technique is general and holds promise for other MoE models.
Summarize any video — free
Summarizer.tube
Copy All
Share Link
Bookmark

Summarize any YouTube video, free

You just read an AI summary of this video. Paste any other YouTube link and get the key points with clickable timestamps in seconds — no signup, 5 free a day.

YouTube Transcript Tool

More Resources

YouTube Video Summarizervs Summarize.techAll AlternativesFor StudentsFor ProfessionalsFor Content CreatorsAll Use CasesHow to Summarize YouTube
Or summarize right on YouTube with our free Chrome extension →

More Summaries

22 min

Italian Journalist Takes ONE-WAY Trip To Gaza!

Beyond Belief with Nuriyah Khan en

The video warns Western activists against blindly supporting radical Islamic groups like Hamas, using the tragic story of Italian journalist Vittorio Arrigoni as a cautionary tale, and drawing paralle

9 min

Italy Just Triggered a Chain Reaction Against Islamism

Allied Nationsen

Italy is confronting political Islam not with new laws or bans, but by rigorously enforcing existing administrative rules, particularly a missing formal agreement for religious recognition, which allo

4 min

How can a simple pagination destroy your app?

Software Developer Diariesen

This video highlights a critical performance issue with traditional limit and offset pagination for large datasets and introduces the more efficient key set (cursor) pagination technique, explaining i