Graphical User Interfaces (GUIs) are ubiquitous, whether on desktop computers, mobile devices, or embedded systems, providing an intuitive bridge between users and digital functions. However, automated interaction with these GUIs presents a significant challenge. This gap becomes particularly evident in building intelligent agents that can comprehend and execute tasks based on visual information alone. Traditional…
The rapid growth of large language models (LLMs) has brought significant advancements across various sectors, but it has also presented considerable challenges. Models such as Llama 3 have made impressive strides in natural language understanding and generation, yet their size and computational requirements have often limited their practicality. High energy costs, lengthy training times, and…
Vision-language models (VLMs) are gaining prominence in artificial intelligence for their ability to integrate visual and textual data. These models play a crucial role in fields like video understanding, human-computer interaction, and multimedia applications, offering tools to answer questions, generate captions, and enhance decision-making based on video inputs. The demand for efficient video-processing systems is…
Machine learning, particularly the training of large foundation models, relies heavily on the diversity and quality of data. These models, pre-trained on vast datasets, are the foundation of many modern AI applications, including language processing, image recognition, and more. The effectiveness of foundation models depends on how well they are trained, which is influenced by…
Vision-Language Models (VLMs) are increasingly used for generating responses to queries about visual content. Despite their progress, they often suffer from a major issue: generating plausible but incorrect responses, also known as hallucinations. These hallucinations can lead to a lack of trust in these systems, especially in real-world, high-stakes applications. Evaluating the helpfulness and truthfulness…
In machine learning, embeddings are widely used to represent data in a compressed, low-dimensional vector space. They capture the semantic relationships well for performing tasks such as text classification, sentiment analysis, etc. However, they struggle to capture the intricate relationships in complex hierarchical structures within the data. This leads to suboptimal performances and increased computational…
Utilizing Large Language Models (LLMs) through different prompting strategies has become popular in recent years. However, many current methods frequently offer very general frameworks that neglect to handle the particular difficulties involved in creating compelling urges. Differentiating prompts in multi-turn interactions, which involve several exchanges between the user and model, is a crucial problem that…
Multi-modal entity alignment (MMEA) is a technique that leverages information from various data sources or modalities to identify corresponding entities across multiple knowledge graphs. By combining information from text, structure, attributes, and external knowledge bases, MMEA can address the limitations of single-modal approaches and achieve higher accuracy, robustness, and effectiveness in entity alignment tasks. However,…
Sparse autoencoders (SAEs) are an emerging method for breaking down language model activations into linear, interpretable features. However, they fail to fully explain model behavior, leaving “dark matter” or unexplained variance. The ultimate aim of mechanistic interpretability is to decode neural networks by mapping their internal features and circuits. SAEs learn sparse representations to reconstruct…