Contrastive learning has emerged as a transformative method for learning effective visual representations through the alignment of image and text embeddings. However, pairwise similarity computation in contrastive loss between image and text pairs poses computational challenges. This paper presents a novel weakly supervised pre-training of vision models on web-scale image-text data. The proposed method reframes…
Sleep staging is a clinically important task for diagnosing various sleep disorders but remains challenging to deploy at scale because it requires clinical expertise, among other reasons. Deep learning models can perform the task but at the expense of large labeled datasets, which are unfeasible to procure at scale. While self-supervised learning (SSL) can mitigate…
On-device machine learning (ML) moves computation from the cloud to personal devices, protecting user privacy and enabling intelligent user experiences. However, fitting models on devices with limited resources presents a major technical challenge: practitioners need to optimize models and balance hardware metrics such as model size, latency, and power. To help practitioners create efficient ML…
Neural knowledge-to-text generation models often struggle to faithfully generate descriptions for the input facts: they may produce hallucinations that contradict the given facts, or describe facts not present in the input. To reduce hallucinations, we propose a novel decoding method, TWEAK (Think While Effectively Articulating Knowledge). TWEAK treats the generated sequences at each decoding step…
The reproducibility and transparency of large language models are crucial for advancing open research, ensuring the trustworthiness of results, and enabling investigations into data and model biases, as well as potential risks. To this end, we release OpenELM, a state-of-the-art open language model. OpenELM uses a layer-wise scaling strategy to efficiently allocate parameters within each…
Extracting information quickly and efficiently from websites and digital documents is crucial for businesses, researchers, and developers. They require specific data from various online sources to analyze trends, monitor competitors, or gather insights for strategic decisions. Collecting this data can be time-consuming and prone to errors, presenting a significant challenge in data-driven industries. Traditionally, web…
Edge artificial intelligence (Edge AI) involves implementing AI algorithms and models on local devices like sensors or IoT devices at the network’s periphery. This setup allows for immediate data processing and analysis, reducing dependence on cloud infrastructure. Consequently, it empowers devices to make intelligent decisions quickly and autonomously without the need for data from distant…
Large Language Models (LLMs) are advancing at a very fast pace in recent times. However, the lack of adequate data to thoroughly verify particular features of these models is one of the main obstacles. An additional layer of complication arises when evaluating the precision and caliber of a model’s free-form text production on its own. …
Multimodal large language models (MLLMs) integrate text and visual data processing to enhance how artificial intelligence understands and interacts with the world. This area of research focuses on creating systems that can comprehend and respond to a combination of visual cues and linguistic information, mimicking human-like interactions more closely. The challenge often lies in the…
Initially designed for continuous control tasks, Proximal Policy Optimization (PPO) has become widely used in reinforcement learning (RL) applications, including fine-tuning generative models. However, PPO’s effectiveness relies on multiple heuristics for stable convergence, such as value networks and clipping, making its implementation sensitive and complex. Despite this, RL demonstrates remarkable versatility, transitioning from tasks like…