N.KOJIRA.IO
Timeline
I've been running a weird quant of Gemma 4 26B-A4B it Heretic ARA: Q8 attention, Q4 experts. The one by AnubysIII

Intel Arc B580 (12GB) GPU, 16GB DDR5 RAM, regular desktop (so only 4/6 threads dedicated to it on CPU)

This was the benchmark with 16k context and 100 token gen from koboldcpp:
ProcessingTime: 19.392 s
ProcessingSpeed: 839.72 T/s
GenerationTime: 3.479 s
GenerationSpeed: 28.74 T/s
TotalTime: 22.871 s

It's absolutely nuts that this is possible on what a low-mid range PC can do. If I eventually stack another arc card in, it can go zoooom.

#aubreykunyap