Pinned
Modal
1,533 posts
Modal
@modal
AI infrastructure that developers love 💚
Run inference, sandboxes, batch processing, training, and many other things on Modal
- Modal reposted
Join Applied Compute and @modal for an evening in Seoul! We’re hosting an ICML rooftop happy hour for top researchers, PhDs, AI lab scientists, and the people building the future of ML infrastructure and custom models. Come unwind and enjoy one of Korea’s hidden gems with us - - Modal reposted
Claude Science has a @modal integration built-in. Great to see the advantages of Modal as a compute substrate shine through here — fan out, shared storage, reproducible environments, GPU flexibility. Try it out!
The most demanding problems in life sciences need more than a capable model, they need infrastructure that scales. Today we're announcing our integration with Claude Science, bringing Modal's elastic compute to researchers when they need it. We're committing up to $100K in
00:00- Our new Auto Endpoints feature is powered by a new Modal primitive: Modal Servers. In this blogpost, we walk through design principles and detailed architecture: @EnvoyProxy, @googlecloud Spanner config store, and a @Cloudflare Pingora-based custom proxy.
00:00 - Modal reposted
Still don't think people fully appreciate how big dflash can be for inference latency/throughput. Genuine game changer for latency-sensitive workloads.Modal Auto Endpoints provide state-of-the-art open source inference perf with a click. Learn how we developed our low latency inference playbook with @DecagonAI, delivering responses 60ms faster than the best proprietary provider. modal.com/blog/achieve-s…
- Modal reposted
The no-longer-secret ingredient is DFlash by @zhijianliu_ and @jianchen1799. If you train a custom DFlash speculator on your data, you can get to lower latencies than any generic inference API can achieve. That's the benefit of owning your inference!Modal Auto Endpoints provide state-of-the-art open source inference perf with a click. Learn how we developed our low latency inference playbook with @DecagonAI, delivering responses 60ms faster than the best proprietary provider. modal.com/blog/achieve-s… - Modal Auto Endpoints provide state-of-the-art open source inference perf with a click. Learn how we developed our low latency inference playbook with @DecagonAI, delivering responses 60ms faster than the best proprietary provider. modal.com/blog/achieve-s…
- Modal repostedYou no longer have to pick between the performance of a black box API and the flexibility and control of @modal. Auto Endpoints give you both. We're unlocking frontier performance for everyone without having to talk to sales or an FDE. More cooking here, stay tuned.
- Modal reposted
Managed private LLM endpoints, now available for everyone in @modal. Deploy in a few clicks with the UI or a few keystrokes with our CLI. The coolest thing is that these are not black boxes – customers have full access to the code underneath.
