Deep learning has revolutionized various domains, with Transformers emerging as a dominant architecture. However, Transformers must improve the processing of lengthy sequences due to their quadratic computational complexity. Recently, a novel architecture named Mamba has shown promise in building foundation models with comparable abilities to Transformers while maintaining near-linear scalability with sequence length. This survey… →

Knowledge Distillation (KD) has become a key technique in the field of Artificial Intelligence, especially in the context of Large Language Models (LLMs), for transferring the capabilities of proprietary models, like GPT-4, to open-source alternatives like LLaMA and Mistral. In addition to improving the performance of open-source models, this procedure is essential for compressing them… →

Data analysis has become increasingly accessible due to the development of large language models (LLMs). These models have lowered the barrier for individuals with limited programming skills, enabling them to engage in complex data analysis through conversational interfaces. LLMs have opened new avenues for extracting meaningful insights from data by simplifying the process of generating… →

Andrej Karpathy coined a new term, ‘Jagged Intelligence‘. ‘Jagged Intelligence‘ refers to modern AI systems’ peculiar and often counterintuitive nature, particularly large language models (LLMs). These models have demonstrated remarkable capabilities in performing complex tasks, from solving intricate mathematical problems to generating coherent and contextually relevant text. However, despite these impressive achievements, they often need… →

A key goal in the development of AI is the creation of general-purpose assistants utilizing Large Multimodal Models (LMMs). Building AI systems that can work in tandem with people in various settings and with a wide variety of jobs is central to the general-purpose assistant concept. These helpers aren’t confined to just one area of… →

RGB-D cameras have a difficult time accurately capturing the depth of transparent objects because of the optical effects of reflection and refraction. Because of this, the depth maps these cameras produce frequently contain inaccurate or missing information. To overcome this problem, recent research has developed sophisticated network designs and advanced visual features intended to recreate… →

Large Language Models (LLMs) have demonstrated exceptional performance on isolated code tasks, such as HumanEval and MBPP, but they struggle significantly when faced with the challenge of handling entire code repositories. The key difficulty lies in the inability of LLMs to manage long-context inputs and perform complex reasoning across intricate code structures within large projects.… →

Traditional biomedical AI models are often specialized and need more flexibility, making them less effective for real-world applications requiring integrating various data types. Generalist AI models, particularly those based on transformers, offer a versatile solution by handling textual and visual data. These models can streamline complex tasks like radiology interpretation and clinical summarization, overcoming the… →