Autoregressive pre-training has proved to be revolutionary in machine learning, especially concerning sequential data processing. Predictive modeling of the following sequence elements has been highly effective in natural language processing and, increasingly, has been explored within computer vision domains. Video modeling is one area that has hardly been explored, giving opportunities for extending into action… →

Large language models (LLMs) like GPT-4, Bard, and Copilot have made a huge impact in natural language processing (NLP). They can generate text, solve problems, and carry out conversations with remarkable accuracy. However, they also come with significant challenges. These models require vast computational resources, making them expensive to train and deploy. This excludes smaller… →

Multi-modal Large Language Models (MLLMs) have revolutionized various image and video-related tasks, including visual question answering, narrative generation, and interactive editing. A critical challenge in this field is achieving fine-grained video content understanding, which involves pixel-level segmentation, tracking with language descriptions, and performing visual question answering on specific video prompts. While state-of-the-art video perception models… →

Large Language Models (LLMs) have revolutionized generative AI, showing remarkable capabilities in producing human-like responses. However, these models face a critical challenge known as hallucination, the tendency to generate incorrect or irrelevant information. This issue poses significant risks in high-stakes applications such as medical evaluations, insurance claim processing, and autonomous decision-making systems where accuracy is… →

Understanding and processing human language has always been a difficult challenge in artificial intelligence. Early AI systems often struggled to handle tasks like translating languages, generating meaningful text, or answering questions accurately. These systems relied on rigid rules or basic statistical methods that couldn’t capture the nuances of context, grammar, or cultural meaning. As a… →

Large Language Models (LLMs) have shown remarkable capabilities across diverse natural language processing tasks, from generating text to contextual reasoning. However, their efficiency is often hampered by the quadratic complexity of the self-attention mechanism. This challenge becomes particularly pronounced with longer input sequences, where computational and memory demands grow significantly. Traditional methods that modify self-attention… →

Multi-hop queries have always given LLM agents a hard time with their solutions, necessitating multiple reasoning steps and information from different sources. They are crucial for analyzing a model’s comprehension, reasoning, and function-calling capabilities. At this time when new large models are booming every other day with claims of unparalleled capabilities, multi-hop tools realistically assess… →

The rise of multimodal applications has highlighted the importance of instruction data in training MLMs to handle complex image-based queries effectively. Current practices for generating such data rely on LLMs or MLMs, which, despite their effectiveness, face several challenges. These include high costs, licensing restrictions, and susceptibility to hallucinations—generating inaccurate or unreliable content. Additionally, the… →