The rise of Transformer-based models has significantly advanced the field of natural language processing. However, the training of these models is often computationally intensive, requiring substantial resources and time. This research addresses the issue of improving the training efficiency of Transformer models without compromising their performance. Specifically, it seeks to explore whether the benefits of… →

Bayesian Optimization, widely used in experimental design and black-box optimization, traditionally relies on regression models for predicting the performance of solutions within fixed search spaces. However, many regression methods are task-specific due to modeling assumptions and input constraints. This issue is especially prevalent in learning-based regression, which depends on fixed-length tensor inputs. Recent advancements in… →

AI has significantly impacted healthcare, particularly in disease diagnosis and treatment planning. One area gaining attention is the development of Medical Large Vision-Language Models (Med-LVLMs), which combine visual and textual data for advanced diagnostic tools. These models have shown great potential for improving the analysis of complex medical images, offering interactive and intelligent responses that… →

Dynamical systems are mathematical models that explain how a system evolves due to physical interactions or forces. These systems are fundamental to understanding various phenomena across scientific fields like physics, biology, and engineering. For example, they model fluid dynamics, celestial mechanics, and robotic movements. The core challenge in modeling these systems lies in their complexity,… →

Long-context Large language models (LLMs) are designed to handle long input sequences, enabling them to process and understand large amounts of information. As the interference computation power is increased the large language models (LLMs) can perform diverse tasks. Particularly for knowledge-intensive tasks that rely mainly on Retrieval augmented generation (RAG), increasing the quantity or size… →

Large language models (LLMs) have demonstrated consistent scaling laws, revealing a power-law relationship between pretraining performance and computational resources. This relationship, expressed as C = 6ND (where C is compute, N is model size, and D is data quantity), has proven invaluable for optimizing resource allocation and maximizing computational efficiency. However, the field of diffusion… →

Code generation AI models (Code GenAI) are becoming pivotal in developing automated software demonstrating capabilities in writing, debugging, and reasoning about code. However, their ability to autonomously generate code raises concerns about security vulnerabilities. These models may inadvertently introduce insecure code, which could be exploited in cyberattacks. Furthermore, their potential use in aiding malicious actors… →

The key challenge in the image autoencoding process is to create high-quality reconstructions that can retain fine details, especially when the image data has undergone compression. Traditional autoencoders, which rely on pixel-level losses such as mean squared error (MSE), tend to produce blurry outputs without capturing high-frequency details, textual information, and edge information. While adversarial… →