Matei Zaharia

Great new launch from the Databricks AI team! We trained a document parsing system that delivers leading quality at 3-5x lower cost and outperforms frontier models like GPT-5 and Claude. https://lnkd.in/gJA8cTTi This is critical to connect AI to so many kinds of data that is locked in PDFs and other documents. We parse each document into a nice JSON format, including tables, text, and even figure descriptions, which you can then feed into AI agents or downstream apps. We also made it super easy to use from SQL or Apache Spark. Just call ai_parse_document and we automatically handle batch execution at scale. Docs here: https://lnkd.in/gCQnjF7a

2,743

69 Comments

K2-V2 is a 70B parameter 360-open #LLM built from scratch as a superior base for reasoning adaptation, in addition to functions such as conversation and knowledge retrieval from general LLMs. It stands as the strongest fully open model, rivals open-weight leaders in its size class, outperforms Qwen2.5-72B and approaches the performance of Qwen3-235B. We actively infuse domain knowledge, reasoning, long-context, and tool use throughout the training process. This explicitly prepares the model for complex reasoning tasks. We demonstrate this potential using simple supervised fine-tuning, establishing a strong baseline that indicates significant headroom for advanced alignment. By releasing the full training history and data composition, we maximize the effectiveness of continuous training, a key open source production scenario, not mentioning the much desired transparency, reproducibility, and safety not offered in all existing widely used open-weight models. The newer version of K2-Think built on K2 also surpasses our earlier version (using Qwen as a base model) as the most powerful open reasoning model. https://lnkd.in/dXm9angn

246

9 Comments

Have you ever wondered how large language models (LLMs) could improve their understanding of long contexts and enhance document retrieval? The newly released paper, "𝗤𝘂𝗲𝗿𝘆-𝗳𝗼𝗰𝘂𝘀𝗲𝗱 𝗮𝗻𝗱 𝗠𝗲𝗺𝗼𝗿𝘆-𝗮𝘄𝗮𝗿𝗲 𝗥𝗲𝗿𝗮𝗻𝗸𝗲𝗿 𝗳𝗼𝗿 𝗟𝗼𝗻𝗴 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴," by Yuqing Li, Jiangnan Li, Mo Yu, et al., addresses this very challenge. The authors propose a groundbreaking 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴 𝗳𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 that utilizes the attention scores of selected heads in LLMs to estimate passage-query relevance. This 𝗹𝗶𝘀𝘁𝘄𝗶𝘀𝗲 𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻 leverages holistic information within the entire candidate shortlist, producing continuous relevance scores that facilitate training on arbitrary datasets without the need for Likert-scale supervision. The key contribution of this research is the demonstration that 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗵𝗲𝗮𝗱𝘀 𝗱𝗶𝗿𝗲𝗰𝘁𝗹𝘆 𝗮𝘀 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿𝘀 can yield faster and more stable results compared to generation-based approaches. This method not only overcomes the geometric bottleneck of embedding models but also avoids the instability of listwise generation methods. However, it's important to note that 𝗽𝗼𝗶𝗻𝘁𝘄𝗶𝘀𝗲 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿𝘀 may still miss global context about other documents in the candidate set, potentially limiting ranking accuracy. For ML engineers, this research offers a 𝗹𝗶𝗴𝗵𝘁𝘄𝗲𝗶𝗴𝗵𝘁 𝗮𝗻𝗱 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 solution that requires only small-scale models to achieve strong performance. Despite its advantages, the approach highlights the need for careful consideration of contextual information and the potential trade-offs in efficiency and performance. Follow us for more research insights, or dig deeper into this paper at PaperChime: https://lnkd.in/gVm_s5hD #LargeLanguageModels #AI #ContinuousRelevanceScores

2

NVIDIA TensorRT LLM provides a high-level Python LLM API, with its PyTorch-native architecture enabling developers to experiment with the runtime or extend functionality. Learn how these latest TensorRT-LLM optimizations boost reasoning inference performance in our recent blog post. 🖇️ https://lnkd.in/gx52zwTA #PyTorch #OpenSourceAI #AI #Inference #Innovation

94

2 Comments

At PyBay 2025 this weekend, Guido van Rossum presented a new Python package called "typeagent" that implements "structured RAG", a different approach to indexing and retrieval. This is how I understand it: 📑 Ingestion: It uses an LLM to turn the content into structured data like entities, topics, actions. It defines the schema in TypeScript, since LLMs are adept at generating TypeScript-adherent schemas. Then the structured data is stored in a standard database (no graph DB needed). 🔎 Retrieval: It again uses an LLM to turn the user query into similarly structured data, and then retrieves matching structured data from the database. If the token budget allows, it also adds the original content the data was extracted from. It is *not* the same as GraphRAG, as that approach builds up hierarchical knowledge graphs based off multiple pieces of related data. GraphRAG enables you to ask more zoomed out questions, but doesn't scale well for scenarios where new data is being added all the time. Guido demonstrated the approach by asking questions on his personal Gmail inbox, like about people and events mentioned in emails. My impression is that this approach could work particularly well for conversation retrieval, perhaps in combination with a hybrid search. Try out the new package and see what you think - it's on my TODO list to explore soon! https://lnkd.in/gdgt7xCr

260

11 Comments

Learn how you can minimize latency and LLM costs at scale: Partha Sarkar's new guide unpacks the nuances of validation-aware, multi-tier caching.

8