Large language models (LLMs) are designed to understand and manage complex language tasks by capturing context and long-term dependencies. A critical factor for their performance is the ability to handle long-context inputs, which allows for a deeper understanding of content over extensive text sequences. However, this advantage comes with the drawback of increased memory usage,… →

Weight decay and ℓ2 regularization are crucial in machine learning, especially in limiting network capacity and reducing irrelevant weight components. These techniques align with Occam’s razor principles and are central to discussions on generalization bounds. However, recent studies have questioned the correlation between norm-based measures and generalization in deep networks. Although weight decay is widely… →

Reinforcement learning (RL) is a domain within artificial intelligence that trains agents to make sequential decisions through trial and error in an environment. This approach enables the agent to learn by interacting with its surroundings, receiving rewards or penalties based on its actions. However, training agents to perform optimally in complex tasks requires access to… →

Large Language Models (LLMs) are vulnerable to jailbreak attacks, which can generate offensive, immoral, or otherwise improper information. By taking advantage of LLM flaws, these attacks go beyond the safety precautions meant to prevent offensive or hazardous outputs from being generated. Jailbreak attack evaluation is a very difficult procedure, and existing benchmarks and evaluation methods… →

Reinforcement Learning from Human Feedback (RLHF) has emerged as a vital technique in aligning large language models (LLMs) with human values and expectations. It plays a critical role in ensuring that AI systems behave in understandable and trustworthy ways. RLHF enhances the capabilities of LLMs by training them based on feedback that allows models to… →

The adversarial attacks and defenses for LLMs encompass a wide range of techniques and strategies. Manually crafted and automated red teaming methods expose vulnerabilities, while white box access reveals potential for prefilling attacks. Defense approaches include RLHF, DPO, prompt optimization, and adversarial training. Inference-time defenses and representation engineering show promise but face limitations. The control… →

The advancement of large language models (LLMs) in natural language processing has significantly improved various domains. As more complex models are developed, evaluating their outputs accurately becomes essential. Traditionally, human evaluations have been the standard approach for assessing quality, but this process is time consuming and needs to be more scalable for the rapid pace… →

Text-to-image (T2I) models have seen rapid progress in recent years, allowing the generation of complex images based on natural language inputs. However, even state-of-the-art T2I models need help accurately capture and reflect all the semantics in given prompts, leading to images that may miss crucial details, such as multiple subjects or specific spatial relationships. For… →

Multi-View and Multi-Scale Alignment for Mammography Contrastive Learning:Contrastive Language-Image Pre-training (CLIP) has shown potential in medical imaging, but its application to mammography faces challenges due to limited labeled data, high-resolution images, and imbalanced datasets. This study introduces the first full adaptation of CLIP to mammography through a new framework called Multi-view and Multi-scale Alignment (MaMA).… →

AMD has recently introduced its new language model, AMD-135M or AMD-Llama-135M, which is a significant addition to the landscape of AI models. Based on the LLaMA2 model architecture, this language model boasts a robust structure with 135 million parameters and is optimized for performance on AMD’s latest GPUs, specifically the MI250. This release marks a… →