Dynamical systems are mathematical models that explain how a system evolves due to physical interactions or forces. These systems are fundamental to understanding various phenomena across scientific fields like physics, biology, and engineering. For example, they model fluid dynamics, celestial mechanics, and robotic movements. The core challenge in modeling these systems lies in their complexity,… →
Long-context Large language models (LLMs) are designed to handle long input sequences, enabling them to process and understand large amounts of information. As the interference computation power is increased the large language models (LLMs) can perform diverse tasks. Particularly for knowledge-intensive tasks that rely mainly on Retrieval augmented generation (RAG), increasing the quantity or size… →
Large language models (LLMs) have demonstrated consistent scaling laws, revealing a power-law relationship between pretraining performance and computational resources. This relationship, expressed as C = 6ND (where C is compute, N is model size, and D is data quantity), has proven invaluable for optimizing resource allocation and maximizing computational efficiency. However, the field of diffusion… →
Code generation AI models (Code GenAI) are becoming pivotal in developing automated software demonstrating capabilities in writing, debugging, and reasoning about code. However, their ability to autonomously generate code raises concerns about security vulnerabilities. These models may inadvertently introduce insecure code, which could be exploited in cyberattacks. Furthermore, their potential use in aiding malicious actors… →
The key challenge in the image autoencoding process is to create high-quality reconstructions that can retain fine details, especially when the image data has undergone compression. Traditional autoencoders, which rely on pixel-level losses such as mean squared error (MSE), tend to produce blurry outputs without capturing high-frequency details, textual information, and edge information. While adversarial… →
Recent advancements in large language models (LLMs) have significantly enhanced their ability to handle long contexts, making them highly effective in various tasks, from answering questions to complex reasoning. However, a critical bottleneck has emerged: the memory requirements for storing key-value (KV) caches escalate significantly as the number of model layers and the length of… →
Large language models (LLMs) have demonstrated significant reasoning capabilities, yet they face issues like hallucinations and the inability to conduct faithful reasoning. These challenges stem from knowledge gaps, leading to factual errors during complex tasks. While knowledge graphs (KGs) are increasingly used to bolster LLM reasoning, current KG-enhanced approaches—retrieval-based and agent-based—struggle with either accurate knowledge… →
Training and deploying large-scale language models (LLMs) is complex, requiring significant computational resources, technical expertise, and access to high-performance infrastructure. These barriers limit reproducibility, increase development time, and make experimentation challenging, particularly for academia and smaller research institutions. Addressing these issues requires a lightweight, flexible, and efficient approach that reduces friction in LLM research. Meta… →
Deep neural networks are powerful tools that excel in learning complex patterns, but understanding how they efficiently compress input data into meaningful representations remains a challenging research problem. Researchers from the University of California, Los Angeles, and New York University propose a new metric, called local rank, to measure the intrinsic dimensionality of feature manifolds… →
Recent advancements in Large Language Models (LLMs) have reshaped the Artificial intelligence (AI)landscape, paving the way for the creation of Multimodal Large Language Models (MLLMs). These advanced models expand AI capabilities beyond text, allowing understanding and generation of content like images, audio, and video, signaling a significant leap in AI development. Despite the remarkable progress… →