Large Language Models (LLMs) have demonstrated remarkable in-context learning (ICL) capabilities, where they can learn tasks from demonstrations without requiring additional training. A critical challenge in this field is understanding and predicting the relationship between the number of demonstrations provided and the model’s performance improvement, known as the ICL curve. This relationship needs to be… →

Large language models (LLMs) are getting better at scaling and handling long contexts. As they are being used on a large scale, there has been a growing demand for efficient support of high-throughput inference. However, efficiently serving these long-context LLMs presents challenges related to the key-value (KV) cache, which stores previous key-value activations to avoid… →

Recent advancements in generative language modeling have propelled natural language processing, making it possible to create contextually rich and coherent text across various applications. Autoregressive (AR) models generate text in a left-to-right sequence and are widely used for tasks like coding and complex reasoning. However, these models face limitations due to their sequential nature, which… →

Creating a common semantic space where queries and items can be represented as dense vectors is the main goal of embedding-based retrieval. Instead of depending on precise keyword matches, this method enables effective matching based on semantic similarities. Semantically related things are positioned closer to one another in this common area since searches and items… →

In the fast-paced digital age, AI assistants have become essential tools for enhancing productivity, managing workflows, and providing personalized support in our everyday lives. From voice-activated home devices to advanced chatbots, these AI assistants are designed to simplify tasks, answer questions, and help users stay organized, efficient, and informed. The rise of AI assistance comes… →

Evaluating NLP models has become increasingly complex due to issues like benchmark saturation, data contamination, and the variability in test quality. As interest in language generation grows, standard model benchmarking faces challenges from rapidly saturated evaluation datasets, where top models reach near-human performance levels. Creating new, high-quality datasets is resource-intensive, demanding human annotation, data cleaning,… →

Conversational AI is now a cornerstone of technology, but achieving fast, efficient, and real-time interaction remains challenging. Latency—the delay between input and response—limits applications like customer service bots and virtual assistants, making interactions feel sluggish. Existing models often require significant computational power, putting real-time AI out of reach for smaller setups and independent developers. An… →

Mathematical reasoning within artificial intelligence has emerged as a focal area in developing advanced problem-solving capabilities. AI can revolutionize scientific discovery and engineering fields by enabling machines to approach high-stakes logical challenges. However, complex tasks, especially Olympiad-level mathematical reasoning, continue to stretch AI’s limits, demanding advanced search methods to navigate solution spaces effectively. Recent strides… →

Recent advancements in Large Language Models (LLMs) have demonstrated exceptional natural language understanding and generation capabilities. Research has explored the unexpected abilities of LLMs beyond their primary training task of text prediction. These models have shown promise in function calling for software APIs, supported by the launch of GPT-4 plugin features. Integrated tools include web… →