Stephen Bronstein

Interesting answer from AlphaSense CEO Jaakko (Jack) Kokko on why they acquired Tegus - all about access to proprietary data, in this case the transcript library on private companies. You recently acquired a company called Tegus. Was this about having more proprietary data to feed into your models? First and foremost, it was about providing our customers with the best possible intelligence on the market, especially on private companies. A couple of years ago, we acquired a company called Stream, the No. 2 player in what you might think of as a library of interview transcripts between investors and experts on a company — their former executives, competitors, customers, partners. It allowed us to produce information that you can’t access anywhere else, and our customers loved it. Tegus is the market leader in that sector and was the natural next step for us. They have the world’s best library of private company content, and we immediately wrapped our search capabilities, our generative AI, our monitoring capabilities around their content and made it much more valuable. We now have the broadest coverage of insights on both public and private companies, allowing our AI to provide even better answers to our customers.

161

8 Comments

How 𝘾𝙖𝙪𝙨𝙖𝙡 𝘼𝙄 Can Help Reduce Hallucinations: New Q&A with Jayeeta Putatunda I recently spoke with Jayeeta, Lead Data Scientist and Director at Fitch Group Inc, about using causal AI to ground language models in real relationships, not just correlations: 𝘛𝘩𝘪𝘯𝘬 𝘰𝘧 𝘧𝘪𝘯𝘢𝘯𝘤𝘦 𝘰𝘳 𝘢𝘯𝘺 𝘥𝘰𝘮𝘢𝘪𝘯 𝘸𝘩𝘦𝘳𝘦 𝘺𝘰𝘶 𝘤𝘢𝘳𝘦 𝘢𝘣𝘰𝘶𝘵 𝘤𝘢𝘶𝘴𝘦 𝘢𝘯𝘥 𝘦𝘧𝘧𝘦𝘤𝘵 𝘳𝘢𝘵𝘩𝘦𝘳 𝘵𝘩𝘢𝘯 𝘫𝘶𝘴𝘵 𝘤𝘰𝘳𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯. 𝘌𝘤𝘰𝘯𝘰𝘮𝘦𝘵𝘳𝘪𝘤𝘴 𝘩𝘢𝘴 𝘢𝘭𝘸𝘢𝘺𝘴 𝘥𝘰𝘯𝘦 𝘵𝘩𝘢𝘵, 𝘵𝘳𝘺𝘪𝘯𝘨 𝘵𝘰 𝘴𝘦𝘦 𝘪𝘧 𝘢 5% 𝘤𝘩𝘢𝘯𝘨𝘦 𝘪𝘯 𝘱𝘢𝘳𝘢𝘮𝘦𝘵𝘦𝘳 𝘈 𝘭𝘦𝘢𝘥𝘴 𝘵𝘰 𝘴𝘰𝘮𝘦 𝘴𝘩𝘪𝘧𝘵 𝘪𝘯 𝘱𝘢𝘳𝘢𝘮𝘦𝘵𝘦𝘳𝘴 𝘉 𝘰𝘳 𝘊. 𝘠𝘰𝘶 𝘤𝘢𝘯 𝘳𝘶𝘯 𝘴𝘪𝘮𝘶𝘭𝘢𝘵𝘪𝘰𝘯𝘴 𝘵𝘰 𝘴𝘦𝘦 𝘪𝘧 𝘵𝘩𝘢𝘵 𝘭𝘪𝘯𝘬 𝘩𝘰𝘭𝘥𝘴, 𝘰𝘳 𝘪𝘧 𝘵𝘩𝘦𝘳𝘦’𝘴 𝘫𝘶𝘴𝘵 𝘢 𝘭𝘰𝘰𝘴𝘦 𝘤𝘰𝘳𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯. 𝘕𝘰𝘸 𝘤𝘰𝘮𝘣𝘪𝘯𝘦 𝘵𝘩𝘢𝘵 𝘸𝘪𝘵𝘩 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘈𝘐. 𝘐𝘧 𝘺𝘰𝘶 𝘧𝘦𝘦𝘥 𝘵𝘩𝘦 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘮𝘰𝘥𝘦𝘭 𝘷𝘢𝘭𝘪𝘥𝘢𝘵𝘦𝘥 𝘤𝘢𝘶𝘴𝘢𝘭 𝘮𝘢𝘱𝘴 𝘰𝘳 𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯𝘴𝘩𝘪𝘱𝘴, 𝘵𝘩𝘦𝘯 𝘸𝘩𝘦𝘯 𝘪𝘵 𝘱𝘳𝘰𝘥𝘶𝘤𝘦𝘴 𝘢 𝘳𝘦𝘱𝘰𝘳𝘵, 𝘪𝘵’𝘴 𝘭𝘦𝘴𝘴 𝘭𝘪𝘬𝘦𝘭𝘺 𝘵𝘰 𝘪𝘯𝘷𝘦𝘯𝘵 𝘧𝘢𝘭𝘴𝘦 𝘤𝘰𝘯𝘯𝘦𝘤𝘵𝘪𝘰𝘯𝘴. 𝘐𝘧 𝘺𝘰𝘶’𝘳𝘦 𝘴𝘶𝘮𝘮𝘢𝘳𝘪𝘻𝘪𝘯𝘨 𝘩𝘰𝘸 𝘈𝘱𝘱𝘭𝘦’𝘴 𝘨𝘳𝘦𝘦𝘯 𝘪𝘯𝘪𝘵𝘪𝘢𝘵𝘪𝘷𝘦𝘴 𝘭𝘦𝘥 𝘵𝘰 𝘢 𝘳𝘦𝘥𝘶𝘤𝘵𝘪𝘰𝘯 𝘪𝘯 𝘤𝘢𝘳𝘣𝘰𝘯 𝘦𝘮𝘪𝘴𝘴𝘪𝘰𝘯𝘴, 𝘺𝘰𝘶 𝘤𝘢𝘯 𝘷𝘢𝘭𝘪𝘥𝘢𝘵𝘦 𝘵𝘩𝘢𝘵 5% 𝘰𝘳 10% 𝘧𝘪𝘨𝘶𝘳𝘦 𝘵𝘩𝘳𝘰𝘶𝘨𝘩 𝘺𝘰𝘶𝘳 𝘤𝘢𝘶𝘴𝘢𝘭 𝘮𝘰𝘥𝘦𝘭. 𝘛𝘩𝘦 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘮𝘰𝘥𝘦𝘭 𝘤𝘢𝘯 𝘵𝘩𝘦𝘯 𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘦 𝘵𝘦𝘹𝘵 𝘵𝘩𝘢𝘵 𝘳𝘦𝘧𝘭𝘦𝘤𝘵𝘴 𝘵𝘩𝘰𝘴𝘦 𝘳𝘦𝘭𝘢𝘵𝘪𝘰𝘯𝘴𝘩𝘪𝘱𝘴, 𝘳𝘢𝘵𝘩𝘦𝘳 𝘵𝘩𝘢𝘯 𝘨𝘶𝘦𝘴𝘴𝘪𝘯𝘨. Jayeeta also shares her path from econometrics to NLP innovations, her perspective on the rapid evolution of AI technologies, and the importance of bridging research with real-world industry challenges. Other takeaways from our discussion: • Finance needs specialized frameworks beyond generic AI metrics • Regulators require clear evidence chains, not black boxes • Traditional statistical models + new AI techniques are key to adoption The full interview is out on Thursday. Subscribe to AI Street to get in your inbox. Link in the first comment. Follow me (Matt Robinson) and AI Street for more on AI + Finance.  

27

3 Comments

Today's article on MCMC - read to learn why there are better and more efficient techniques than chasing LLMs and other #AI buzz. https://lnkd.in/eaw6Paan

8

LLMs are trained on internet-scale data. But to make AI work for work, models need to be grounded in your enterprise data -- a much harder problem than meets the eye! How do you connect to all your data sources and ingest them frequently enough while managing compute/storage costs? How do you make sure that you only show users results from documents they have access to? How do you rank results without internet-scale click and link data? How do you process the content of documents that are often hidden in PDFs and presentations? These are tricky problems, and it's why enterprise search didn't work for nearly a decade, until Glean came along. Turns out it's the exact same set of challenges needed to give LLMs the right context to work their magic on your enterprise content. Glean spent 5 years building the ingestion, search, and ranking infra needed to solve this problem well. This has given them the right to run the "Work AI" race and delight users at work. CEO Arvind Jain shares more on the latest episode of Training Data. Listen here, or wherever you get your podcasts: YouTube: https://seq.vc/ytd Apple: https://seq.vc/pit Spotify: https://seq.vc/jnz 

203

10 Comments

The cost of training top models is increasing 2-3X per year, and at this rate will reach $1 billion per model by 2027. The findings are from Epoch AI and follow their recent analysis of increases in compute for foundation models running at 4-5X per year. They've also provided some interesting data on the relative breakdown of cost components like staff, hardware, and energy. A few thoughts after reading this: - I would assume staff costs won't increase as much as hardware and energy costs, so the relative contribution of human labor to future AI models will decrease, in yet another sign of capital's increasing dominance over labor in the industry (just look at NVIDIA's ever-rising stock price). - If compute is increasing 4-5X per year but cost is increasing 2-3X, is it safe to assume that training runs are getting more efficient? - Some people, like Anthropic CEO Dario Amodei, have said we'll see $1 billion training runs in the next year. Does this mean the 2-3X pace is about to increase? That Epoch is wrong in its estimate? Or that Amodei is overstating the cost, possibly to put fear into competitors who can't keep pace? - Only a few companies are going to be able to train future models. We're likely going to see further consolidation. Perhaps we get to a future with only two large general purpose models, like the Windows/MacOS and Android/iOS duopolies we've seen in the past. Article here: https://lnkd.in/g5SejXen

7

While you probably don't have to worry about aliens invading on the fourth of July, you should be aware that text embeddings may expose your private information. Fortunately it's an addressable risk. Our Senior AI Scientist, Joseph Ferrara, PhD, covers how to mitigate the problem on a pdf. Broadly, the steps are: -Extract the text -De-identify the sensitive info -Chunk the resulting text -Embed it in a Pinecone database -Perform some test queries If you want to see more detail and sample code, check out the how to guide the comments. #RAG #AI #embeddings

15

3 Comments

Excellent study from Damian Rollison and the SOCi, Inc. team which seems to confirm my longtime hypothesis around citations (https://lnkd.in/gxZQwdx3): the only citations that are valuable are the ones that rank in Google for the keywords you want to be known for. https://lnkd.in/g49DMSeN

38

11 Comments

The most common feedback I get from customers is "we have so much data, but don't know how to get value out of it." Current tools and product analytics models have failed to fill that void. Love how the team at Motif explains the difference between what leaders think a "causal driver" is, what it means in practice, and how you figure out what those are. Incredibly complex concepts explained extremely well! No, it won't give you a (probably BS, but conveniently packaged) narrative like "a Facebook user is hooked after 7 friends in 10 days". But it can give you the most points to build your intuition and experiments around. Tools like Motif (and we've been working on 😉 ) are reducing the domain, technical expertise, and time required to get value out of your data. We won't replace analysts in the near future, but I see it as an upskilling tool. If every junior analyst or data scientist is effectively more experienced, the strategic questions to answer and business context - that are nearly impossible for LLMs to capture today - become the differentiators between project success & failure. If this is a space you're interested in, would love to connect!

7

LLMs cannot reason and don't think Agents built on LLMs do not have agency In the attached blog, Melanie Mitchell deconstructs the so-called emerging properties like "reasoning" and "chain of thought". It's very well written and you will find some excellent examples and references. I harbour no doubts: LLMs are not intelligent, they don't reason and they don't have thoughts. There is no chain of thought: it's simply a chain of tokens. Why does GPT4-o1 says "Thinking" in the UI? Because the OpenAGI programmers wrote it in there. It doesn't come from the model. The delay is no doubt caused by inference batching across multiple models. That's not thinking, it's simply cost optimisation in the back end. Putting "Thinking" in the UI is wantonly trying to mislead the user. Which brings me to another point: watch out because agents do not have agency. You can think of LLM agents as a method to decompose a task into subtasks using statistical methods, with a layer of orchestration. There's no reasoning, and it is not reliable: you are still problem-solving on a wobbly statistical model that needs to be supervised by a highly skilled human. It is intelligent? No Can it be useful? YES !!! https://lnkd.in/d8isCjYz

126

29 Comments

Last fall Ibbaka research found that the most common pricing metric for AI applications wasper user. This was confirmed in a recent survey by Kyle Poyar and Palle Broe for Growth Unhinged. What are the reasons for this? https://lnkd.in/gNzyy3r6 #pricing #pricingmodel #pricingmetric #AI #generativeAI

6

4 Comments

Apple announces ToolSandbox A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities abs: Recent large language models (LLMs) advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges, which calls for comprehensive evaluation of tool-use capabilities. While previous works focused on either evaluating over stateless web services (RESTful API), based on a single turn user prompt, or an off-policy dialog trajectory, ToolSandbox includes stateful tool execution, implicit state dependencies between tools, a built-in user simulator supporting on-policy conversational evaluation and a dynamic evaluation strategy for intermediate and final milestones over an arbitrary trajectory. We show that open source and proprietary models have a significant performance gap, and complex tasks like State Dependency, Canonicalization and Insufficient Information defined in ToolSandbox are challenging even the most capable SOTA LLMs, providing brand-new insights into tool-use LLM capabilities. ToolSandbox evaluation framework is released at github.

11

I just read an insightful piece by Eric Broda (published in Towards Data Science) that outlines the Agentic Mesh—a new ecosystem designed for autonomous agents to interact, collaborate, and transact. His work introduces a structured approach for managing the coming surge in autonomous agents, showing how they can function independently yet collectively. Agentic Mesh serves as a networked environment where autonomous agents operate in a safe, transparent, and efficient manner. It allows agents to find each other and securely interact under a framework of policies. Each agent has a specific purpose and autonomy, making it a valuable contributor to the ecosystem without sacrificing security or control. Core Components: 1. Marketplace: A centralized space where users discover agents by tasks and capabilities, offering a trusted environment for engagement. 2. Registry: A metadata repository organizing each agent’s purpose, owner, and policies, allowing agents and users to securely find and connect with each other. 3. DNS Integration: Each agent has a recognizable digital identity, enabling seamless discovery and interaction, just like an internet service. 4. Agent Stack: This layered structure of communication protocols, control services, and interfaces helps agents operate effectively and responsibly. It ensures that agents act independently within controlled parameters. Agentic Mesh’s effectiveness depends on agents sharing consistent attributes, including: • Purpose: A clear objective drives each agent’s actions. • Ownership: Transparency around ownership fosters responsibility and trust. • Autonomy: Agents act independently within set parameters to complete tasks. • Discoverability: Easily locatable, ensuring efficient interaction within the network. • Intelligence: Agents utilize language models to plan and execute tasks, adapting to complexity as needed. Operational Flow: Registration, Discovery, and Execution Agent interactions in the Mesh follow three core flows: 1. Registration: Agents set up purpose, policies, and registration data to gain visibility in the mesh. 2. Discovery: Through the registry, agents and users locate potential collaborators based on defined needs. 3. Execution: Engaged agents use AI models to plan and perform tasks, enlisting collaborators when necessary to complete workflows. Read the full article: https://lnkd.in/eVmq5BFC

16

1 Comment

LLM Data Engineering Pattern 3: Aggregating Unstructured Insights at Scale Modern financial analysis often requires synthesizing information from a vast array of unstructured sources—regulatory filings (e.g., 10-K, 10-Q), research reports, market commentary, and even audio transcripts of earnings calls. In many cases, these sources must be combined to form a comprehensive view of an industry’s risk profile. Historically, analysts have relied on manual selection and review of documents, a labor-intensive process that does not scale well in today’s data-rich environment. Consider analyzing credit risk in the U.S. commercial real estate sector: 1) Regulatory Filings: An LLM can systematically parse through hundreds of 10-K and 10-Q filings to extract commentary on asset valuations, loan-to-value ratios, or refinancing challenges. 2) Research Reports: It can synthesize qualitative assessments from research houses covering property market trends, default probabilities, or vacancy rates. 3) Earnings Call Transcripts: It can highlight executives’ forward-looking statements regarding rental income stability, interest rate exposures, or occupancy forecasts. By integrating these unstructured data points, an analyst can address questions like: “Given the commentary on sector vulnerabilities in these filings, research reports, and macroeconomic indicators, what are the key risk factors influencing the commercial real estate market?” Financial people, how are you doing this today? #llm #dataengineering #llmdataengineering #financialdata #finance

20

1 Comment

#RAG DOES NOT REDUCE HALLUCINATIONS IN #LLMS: Think I've called this out a few times, but the validation is nice - https://lnkd.in/eAi6zxW2. The article makes two well established points: 1) "Hallucination is a problem of reasoning and not relevance! Any amount of relevant text fed through RAG to a machine will retain the original perplexity and entropy in the system to hallucinate independent of the text"; and 2) "The initial challenge arises from the retrieval phase’s reliance on semantic similarity, which is quantified by inner product spaces or cosine similarities. This metric, while capturing semantic closeness, does not differentiate between factually accurate and inaccurate information." This should be clear to everyone, but the industry still shills flimsy RAG scaffolding anyway (shovels to gold miners).

104

32 Comments

As an advisor for Silverside AI, I wrote this article about how AI has really effected our relationship to data and how society should take a moment to think about it. Please, take a look and share your ideas.

7

3 Comments

Faster, more accurate, less data needed = BaseModel.AI

5