Integer Linear Programming (ILP) is the foundation of combinatorial optimization, which is extensively applied across numerous industries to resolve challenging decision-making issues. Under a set of linear equality constraints, an ILP aims to minimize or maximize a linear objective function, with the important condition that all variables must be integers. Even while ILP is an…
Input space mode connectivity in deep neural networks builds upon research on excessive input invariance, blind spots, and connectivity between inputs yielding similar outputs. The phenomenon exists generally, even in untrained networks, as evidenced by empirical and theoretical findings. This research expands the scope of input space connectivity beyond out-of-distribution samples, considering all possible inputs.…
Multi-agent reinforcement learning (MARL) is a field focused on developing systems where multiple agents cooperate to solve tasks that exceed the capabilities of individual agents. This area has garnered significant attention due to its relevance in autonomous vehicles, robotics, and complex gaming environments. The aim is to enable agents to work together efficiently, adapt to…
Artificial Intelligence (AI) safety has become an increasingly crucial area of research, particularly as large language models (LLMs) are employed in various applications. These models, designed to perform complex tasks such as solving symbolic mathematics problems, must be safeguarded against generating harmful or unethical content. With AI systems growing more sophisticated, it is essential to…
In artificial intelligence and natural language processing, long-context reasoning has emerged as a crucial area of research. As the volume of information that needs to be processed grows, machines must be able to synthesize and extract relevant data from massive datasets efficiently. This goes beyond simple retrieval tasks, requiring models to locate specific pieces of…
Computational reproducibility poses a significant challenge in scientific research across various fields, including psychology, economics, medicine, and computer science. Despite the fundamental importance of reproducing results using provided data and code, recent studies have exposed severe shortcomings in this area. Researchers face numerous obstacles when replicating studies, even when code and data are available. These…
Federated Learning (FL) is a technique that allows Machine Learning models to be trained on decentralized data sources while preserving privacy. This method is especially helpful in industries like healthcare and finance, where privacy issues prevent data from being centralized. However, there are big problems when trying to include Homomorphic Encryption (HE) to protect the…
Chain-of-thought (CoT) prompting has emerged as a popular technique to enhance large language models’ (LLMs) problem-solving abilities by generating intermediate steps. Despite its better performance in mathematical reasoning, CoT’s effectiveness in other domains remains questionable. Current research is focused more on mathematical problems, possibly overlooking how CoT could be applied more broadly. In some areas,…
Artificial intelligence (AI) has given rise to powerful models capable of performing diverse tasks. Two of the most impactful advancements in this space are Retrieval-Augmented Generation (RAG) and Agents, which play distinct roles in improving AI-driven applications. However, the emerging concept of Agentic RAG presents a hybrid model that utilizes the strengths of both systems.…
Transformer models have revolutionized sequence modeling tasks, but their standard attention mechanism faces significant challenges when dealing with long sequences. The quadratic complexity of softmax-based standard attention hinders the efficient processing of extensive data in fields like video understanding and biological sequence modeling. While this isn’t a major concern for language modeling during training, it…