Future Trends in AI and Graph Technologies

One year ago today, Dean Allemang Bryon Jacob and I released our paper "A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases" and WOW! Early 2023, everyone was experimenting with LLMs to do text to sql. Examples were "cute" questions on "cute" data. Our work provided the first piece of evidence (to the best of our knowledge) that investing in Knowledge Graph provides higher accuracy for LLM-powered question-answering systems on SQL databases. The result was that by using a knowledge graph representations of SQL databases achieves 3X the accuracy for question-answering tasks compared to using LLMs directly on SQL databases. The release of our work sparked industry-wide follow-up: - The folks at dbt, led by Jason Ganz, replicated our findings, generating excitement across the semantic layer space - Semantic layer companies began citing our research, using it to advocate for the role of semantics - We continuously get folks thanking us for the work because they have been using it as supporting evidence for why their organizations should invest in knowledge graphs - RAG got extended with knowledge graphs: GraphRAG - This research has also driven internal innovation at data.world forming the foundation of our AI Context Engine where you can build AI apps to chat with data and metadata. Over the past year, I've observed two trends: 1) Semantics is moving from "nice-to-have" towards foundational: Organizations are realizing that semantics are fundamental for effective enterprise AI. Major cloud data vendors are incorporating these principles, broadening the adoption of semantics. While approaches vary (not always strictly using ontologies and knowledge graphs), the message is clear: semantics provides your unique business context that LLMs don't necessarily have. Heck, Ontology isn't a frowned upon word anymore 😀   2) Knowledge Graphs as the ‘Enterprise Brain’: Our work pushed to combine Knowledge Graphs with RAG, GraphRAG, in order to have semantically structured data that represents the enterprise brain of your organization. Incredibly honored to see Neo4j Graph RAG Manifesto citing our research as critical evidence for why knowledge graphs drive improved LLM accuracy. It's really exciting that the one year anniversary of our work is while Dean and I are at the International Semantic Web Conference. We are sharing our work on how ontologies come to the rescue to further increase the accuracy to 4x (we released that paper in May). This image is an overview of how it's achieved. It's pretty simple, and that is a good thing! I've dedicated my entire career (close to 2 decades) to figure out how to manage data and knowledge at scale and this GenAI boom has been the catalyst we needed in order to incentivize organizations to invest in foundations in order to truly speed up an innovate. There are so many people to thank! Here’s to more innovation and impact!

Folks interested in AI / AI PM, I recommend watching this recent session by the awesome Aishwarya Naresh Reganti talking about Gen AI Trends. ANR is a "Top Voice" that I follow regularly, leverage her awesome GitHub repository, consume her Instagram shorts like candy and looking forward to her upcoming Maven Course on AI Engineering. https://lnkd.in/g4DiZXBU Aishwarya highlights the growing importance of prompt engineering, particularly goal engineering, where AI agents break down complex tasks into smaller steps and self-prompt to achieve higher-order goals. This trend reduces the need for users to have extensive prompt engineering skills. In the model layer, she discusses the rise of small language models (SLMs) that achieve impressive performance with less computational power, often through knowledge distillation from larger models. Multimodal foundation models are also gaining traction, with research focusing on integrating text, images, videos, and audio seamlessly. Aishwarya emphasizes Retrieval Augmented Generation (RAG) as a successful application of LLMs in the enterprise. She notes ongoing research to improve RAG's efficiency and accuracy, including better retrieval methods and noise handling. AI agents are discussed in detail, with a focus on their potential and current limitations in real-world deployments. Finally, Aishwarya provides advice for staying updated on AI research, recommending focusing on reliable sources like Hugging Face and prioritizing papers relevant to one's specific interests. She also touches upon the evolving concept of "trust scores" for AI models and the importance of actionable evaluation metrics. Key Takeaways: Goal Engineering: AI agents are learning to break down complex tasks into smaller steps, reducing the need for users to have extensive prompt engineering skills. Small Language Models (SLMs): SLMs are achieving impressive performance with less computational power, often by learning from larger models. Multimodal Foundation Models: These models are integrating text, images, videos, and audio seamlessly. Retrieval Augmented Generation (RAG): RAG is a key application of LLMs in the enterprise, with ongoing research to improve its efficiency and accuracy. AI Agents: AI agents have great potential but face limitations in real-world deployments due to challenges like novelty and evolution. Staying Updated: Focus on reliable sources like Hugging Face and prioritize papers relevant to your interests. 🤔 Trust Scores: The concept of "trust scores" for AI models is evolving, emphasizing the importance of actionable evaluation metrics. 📏 Context Length: Models can now handle much larger amounts of input text, enabling more complex tasks. 💰 Cost: The cost of using AI models is decreasing, making fine-tuning more accessible. 📚 Modularity: The trend is moving towards using multiple smaller AI models working together instead of one large model.

Happy New Year! If you are an Enterprise CTO, you are probably thinking about your GenAI strategy. Here's a decent write-up by Gartner: https://gtnr.it/3RQodsK. To augment that, here are some #genai trends to track and act on in 2024: --Open and Smaller Models: Open models like Llama, Mistral, BERT, and FLAN are becoming competitive with larger, closed-source models. They're suitable for many use cases and offer transparency for Responsible AI. In my opinion, open & closed models are not in a zero-sum game; BOTH should be used for the right use case. Action: Implement a clear plan for using different models. Amazon Web Services (AWS) users can leverage Bedrock & SageMaker (https://bit.ly/3vkX4qa). -- Domain-Adapted Models: Use your enterprise proprietary data to extend a large language model via continued pre-training (CPT) for domain-specific tasks. Action: Assess your use cases and data for CPT alongside Fine-tuning. Learn more: https://lnkd.in/eNjbQm-m -- Multi-modal Models (MMMs): MMMs will gain prominence in 2024. Both commercial (like GPT-4V) and open-source models (like Llava) will be popular. Action: Expand into business cases served by MMMs. More about Llava: https://lnkd.in/eTvn82iM. -- AI Agents (RAG+++): AI agents using LLMs can improve upon RAG by intelligently utilizing multiple data sources. Action: Prepare APIs and Data Sources for AI agents. More information: https://lnkd.in/eVf_J_bA. -- LLMs with Graphs: Graphs are one of the best representations of real-world knowledge which when combined with LLMs can be very effective in various domains. Action: Identify suitable business cases and explore Graphs+LLMs. Details: https://lnkd.in/eF68FbVA. -- AI Routers: Most enterprises will end up using a dozen or more models and it will become necessary to manage multiple models - Auth, Audit and Smart Model Selection. Action: Build an AI router. AWS Bedrock can assist, but more is needed. Info: https://go.aws/4aFLkyA. -- FinOps meets MLOps: Focus on cost optimization for GenAI projects. 2023 was all about GenAI POCs; 2024 will be about production & big bills! Action: Learn about GenAI business cases and FinOps for GenAI: https://lnkd.in/ezfV8NTa. --Make AI Invisible. Technology is at its best when its invisible and seamless to the end user. Action: Look at existing enterprise applications and look for ways to rethink the user experience using GenAI while keeping the tech invisible. (https://bit.ly/3vkXvAO) What are you tracking? Watch out for more on domain-focused AI trends in areas like AI for the Edge, robotics, and Drug discovery etc. in upcoming posts.