Developing effective multi-modal AI systems for real-world applications requires handling diverse tasks such as fine-grained recognition, visual grounding, reasoning, and multi-step problem-solving. Existing open-source multi-modal language models are found to be wanting in these areas, especially for tasks that involve external tools such as OCR or mathematical calculations. The abovementioned limitations can largely be attributed…
The rapid growth of digital platforms has brought image safety into sharp focus. Harmful imagery—ranging from explicit content to depictions of violence—poses significant challenges for content moderation. The proliferation of AI-generated content (AIGC) has exacerbated these challenges, as advanced image-generation models can easily create unsafe visuals. Current safety systems rely heavily on human-labeled datasets, which…
Artificial Intelligence (AI) is revolutionizing how discoveries are made. AI is creating a new scientific paradigm with the acceleration of processes like data analysis, computation, and idea generation. Researchers want to create a system that eventually learns to bypass humans completely by completing the research cycle without human involvement. Such developments could raise productivity and…
GANs are often criticized for being difficult to train, with their architectures relying heavily on empirical tricks. Despite their ability to generate high-quality images in a single forward pass, the original minimax objective is challenging to optimize, leading to instability and risks of mode collapse. While alternative objectives have been introduced, issues with fragile losses…
Autoregressive pre-training has proved to be revolutionary in machine learning, especially concerning sequential data processing. Predictive modeling of the following sequence elements has been highly effective in natural language processing and, increasingly, has been explored within computer vision domains. Video modeling is one area that has hardly been explored, giving opportunities for extending into action…
Large language models (LLMs) like GPT-4, Bard, and Copilot have made a huge impact in natural language processing (NLP). They can generate text, solve problems, and carry out conversations with remarkable accuracy. However, they also come with significant challenges. These models require vast computational resources, making them expensive to train and deploy. This excludes smaller…
Multi-modal Large Language Models (MLLMs) have revolutionized various image and video-related tasks, including visual question answering, narrative generation, and interactive editing. A critical challenge in this field is achieving fine-grained video content understanding, which involves pixel-level segmentation, tracking with language descriptions, and performing visual question answering on specific video prompts. While state-of-the-art video perception models…
Large Language Models (LLMs) have revolutionized generative AI, showing remarkable capabilities in producing human-like responses. However, these models face a critical challenge known as hallucination, the tendency to generate incorrect or irrelevant information. This issue poses significant risks in high-stakes applications such as medical evaluations, insurance claim processing, and autonomous decision-making systems where accuracy is…
Understanding and processing human language has always been a difficult challenge in artificial intelligence. Early AI systems often struggled to handle tasks like translating languages, generating meaningful text, or answering questions accurately. These systems relied on rigid rules or basic statistical methods that couldn’t capture the nuances of context, grammar, or cultural meaning. As a…