InfoQ - Local Inference

Presentation: Running AI at the Edge: Running Real Workloads Directly in the Browser

James Hall — Mon, 31 Aug 2026 11:00:00 GMT

James Hall discusses the strategic and technical imperative of moving AI workloads from cloud providers to local edge devices. He shares practical approaches using WebGPU, Transformers.js, and DuckDB to achieve near-native performance in JavaScript. Through real-world case studies, he explains how to minimize data privacy risks, optimize browser inference, and build rigorous evaluation suites.

By James Hall

FreeToken Unlocks Frontier MoE Inference on Consumer Hardware via Dynamic Co-Execution

Olimpiu Pop — Sat, 29 Aug 2026 05:05:00 GMT

Researchers from UC Berkeley and MIT have developed FreeToken, an open-source inference engine that enhances the utility of Mixture-of-Experts models on consumer hardware. By implementing a dynamic scheduling policy and optimising weight management, FreeToken improves decoding speeds and execution efficiency in edge AI applications, fostering self-hosted reasoning systems.

By Olimpiu Pop