A key question about LLMs is whether they solve reasoning tasks by learning transferable algorithms or simply memorizing training data. This distinction matters: while memorization might handle familiar tasks, true algorithmic understanding allows for broader generalization. Arithmetic reasoning tasks could reveal if LLMs apply learned algorithms, like vertical addition in human learning, or if they… →

In recent years, multimodal large language models (MLLMs) have revolutionized vision-language tasks, enhancing capabilities such as image captioning and object detection. However, when dealing with multiple text-rich images, even state-of-the-art models face significant challenges. The real-world need to understand and reason over text-rich images is crucial for applications like processing presentation slides, scanned documents, and… →

Quantization is an essential technique in machine learning for compressing model data, which enables the efficient operation of large language models (LLMs). As the size and complexity of these models expand, they increasingly demand vast storage and memory resources, making their deployment a challenge on limited hardware. Quantization directly addresses these challenges by reducing the… →

Large Language Models (LLMs) have emerged as powerful tools in natural language processing, yet understanding their internal representations remains a significant challenge. Recent breakthroughs using sparse autoencoders have revealed interpretable “features” or concepts within the models’ activation space. While these discovered feature point clouds are now publicly accessible, comprehending their complex structural organization across different… →

Escalation in AI implies an increased infrastructure expenditure. The massive and multidisciplinary research exerts economic pressure on institutions as high-performance computing (HPC)  costs an arm and a leg. HPC is financially draining and critically impacts energy consumption and the environment. By 2030, AI is projected to account for 2% of global electricity consumption. New approaches… →

In recent times, large language models (LLMs) built on the Transformer architecture have shown remarkable abilities across a wide range of tasks. However, these impressive capabilities usually come with a significant increase in model size, resulting in substantial GPU memory costs during inference. The KV cache is a popular method used in LLM inference. It… →

The Evidence Lower Bound (ELBO) is a key objective for training generative models like Variational Autoencoders (VAEs). It parallels neuroscience, aligning with the Free Energy Principle (FEP) for brain function. This shared objective hints at a potential unified machine learning and neuroscience theory. However, both ELBO and FEP lack prescriptive specificity, partly due to limitations… →

The ability to generate accurate conclusions based on data inputs is essential for strong reasoning and dependable performance in Artificial Intelligence (AI) systems. The softmax function is a crucial element that supports this functionality in modern AI models. A major component of differentiable query-key lookups is the softmax function, which enables the model to concentrate… →

Multimodal Retrieval Augmented Generation (RAG) technology has opened new possibilities for artificial intelligence (AI) applications in manufacturing, engineering, and maintenance industries. These fields rely heavily on documents that combine complex text and images, including manuals, technical diagrams, and schematics. AI systems capable of interpreting both text and visuals have the potential to support intricate, industry-specific… →