Fei Wang

Updates from #llamacon2025 Last week, we open-sourced two new tools to help address major pain points developers face when building with Llama. llama-prompt-ops A tool to optimize or migrate prompts from other LLMs to Llama — systematically and at scale. https://lnkd.in/g-D9vwaB synthetic-data-kit A modular toolkit to generate high-quality post-training data (Q&A pairs, reasoning samples, DPO datasets, and more) to fine-tune Llama models and help close the gap with closed-source models. https://lnkd.in/g2CdKh4i Avialble on #unsloth now -- https://lnkd.in/gGZ-RDUM We’re just getting started. Let’s build together — contribute, fork, test, and let us know what features you’d love to see next. #Llama #OpenSourceAI #LLM #PromptEngineering #SyntheticData #LlamaEcosystem

109

3 Comments

When Steve Huynh was a principal engineer at Amazon, meetings began with a "study hall." Amazon had a "reading culture" even among engineers, Huynh recently told the Pragmatic Engineer podcast, speaking of his time at the tech giant. Employees frequently drafted six-page memos, which they shared with the company to update progress and demonstrate new projects. Huynh, who said the company's embrace of writing and reading the memos was part of its "secret sauce," said employees' writing was often constrained to the format, whether it was a business strategy or press release. Huynh started at Amazon in 2006, only a few years after the company turned its first profit and while Jeff Bezos was at the helm. Read more about how Bezos instilled this culture of memo-writing from the top down on Business Insider: https://lnkd.in/gBqbEf_j (Credit: Daniel Berman) #amazon #reading #jeffbezos

216

2 Comments

🚀 Big news from Anyshift! We are launching Anyshift Root Cause Analysis (RCA). Anyshift RCA is the AI on-call engineer that understands every dependency in your cloud infrastructure. It continuously builds a deep knowledge graph from your GCP, AWS, k8s and other platforms, then fuses that context with live observability data. The result: precise root cause insight delivered in minutes, not hours. • No more guessing games with scattered logs; RCA tells you exactly what broke and why • True system-wide context means recurring incidents get fixed at the source • Shorter outages, happier customers and more sleep for your on call team Ready to give your team back their nights and weekends? Let’s talk.

73

15 Comments

Update: Google is going back to in-person onsites for SWE roles in the next 2-3 months. I published a piece a few weeks ago where I summarized AI's effect on eng interview processes after surveying ~50 FAANG interviewers on interviewing.io, and in that piece I said there was no indication that any of the FAANGs were going back to in-person. (Piece linked in the comments.) Then I got this update from one of our interviewers last week. Things change fast. I'll share them as I learn more!

451

45 Comments

The biggest difference between Google and Waymo, almost one year in: Waymo has largely adopted the Amazonian practice of presenting information in documents, not slides. Meetings start with a few minutes of pre-read / frantic commenting on a Google Doc, followed by a live burn-down of the comments that require discussion. It's efficient and information-dense. I love it.

2,183

63 Comments

My fellow Googlers recently have published a white paper on AI Agents. Many believe that AI Agent is the application layer of LLMs, and it seems like we are stepping closer to having useful and reliable AI Agents. This is a recommended must-read.

92

6 Comments

🔥 Exciting News in Semantic Caching Technology! 🔥 Redis's AI Research team has just released langcache-embed-v1 - a powerful new embedding model specifically designed for semantic caching of LLMs! Check it out on Hugging Face: https://lnkd.in/gGukYc3k Fine-tuned from gte-modernbert-base this model, maps sentences and paragraphs to 768-dimensional dense vector spaces with impressive performance metrics: 90% cosine accuracy 87% F1 score 92% average precision 💡 Why this matters: Semantic caching dramatically reduces redundant LLM calls, cutting both latency and costs in production AI systems. This purpose-built model optimizes specifically for caching scenarios rather than general embeddings. 📑 Technical highlights: 8192 token sequence length Built on ModernBERT architecture Optimized with Online Contrastive Loss Simple integration with sentence-transformers library Learn more about "Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data" in their recent paper (arxiv:2504.02268). #Redis #AI #MachineLearning #SemanticCaching #LLM #Embeddings #AIPerformance #RedisEnterprise

106

1 Comment

this is super important because TPUs have definitely advantages for cost and performance over other accelators but were much more difficult to use. no longer the case. "vLLM TPU is now powered by tpu-inference, an expressive and powerful new hardware plugin unifying JAX and PyTorch under a single lowering path. It is not only faster than the previous generation of vLLM TPU, but also offers broader model coverage and feature support." https://lnkd.in/diDKzYbS

34

1 Comment

Excited to announce Ray on Anyscale is now available on CoreWeave to further deliver on our multi-cloud support. This gives users access to CoreWeave’s purpose-built AI infrastructure, optimized for distributed AI at scale with low latency, fast startup times, and high performance GPUs. It's a powerful combination for teams building and scaling production AI workloads. Read more from CoreWeave: https://lnkd.in/dN9AjA3s

83

1 Comment