A direct correlation exists between an LLM’s training corpus quality and its capabilities. Consequently, researchers have invested a great deal of effort into curating extensive, high-quality datasets, which, at present, are achievable with craftful human annotations. Man-made datasets, however, have one downside: their reliance becomes increasingly unsustainable as complexity grows.  Many methods have been worked… →

Researchers are focusing increasingly on creating systems that can handle multi-modal data exploration, which combines structured and unstructured data. This involves analyzing text, images, videos, and databases to answer complex queries. These capabilities are crucial in healthcare, where medical professionals interact with patient records, medical imaging, and textual reports. Similarly, multi-modal exploration helps interpret databases… →

LLMs have revolutionized software development by automating coding tasks and bridging the natural language and programming gap. While highly effective for general-purpose programming, they struggle with specialized domains like High-Performance Computing (HPC), particularly in generating parallel code. This limitation arises from the scarcity of high-quality parallel code data in pre-training datasets and the inherent complexity… →

Large Language Models (LLMs) have shown significant potential in reasoning tasks, using methods like Chain-of-Thought (CoT) to break down complex problems into manageable steps. However, this capability comes with challenges. CoT prompts often increase token usage, leading to higher computational costs and energy consumption. This inefficiency is a concern for applications that require both precision… →

The development of Transformer models has significantly advanced artificial intelligence, delivering remarkable performance across diverse tasks. However, these advancements often come with steep computational requirements, presenting challenges in scalability and efficiency. Sparsely activated Mixture-of-Experts (MoE) architectures provide a promising solution, enabling increased model capacity without proportional computational costs. Yet, traditional TopK+Softmax routing in MoE models… →

Operator learning is a transformative approach in scientific computing. It focuses on developing models that map functions to other functions, an essential aspect of solving partial differential equations (PDEs). Unlike traditional neural network tasks, these mappings operate in infinite-dimensional spaces, making them particularly suitable for scientific domains where real-world problems inherently exist in expansive mathematical… →

Agentic AI systems have revolutionized industries by enabling complex workflows through specialized agents working in collaboration. These systems streamline operations, automate decision-making, and enhance overall efficiency across various domains, including market research, healthcare, and enterprise management. However, their optimization remains a persistent challenge, as traditional methods rely heavily on manual adjustments, limiting scalability and adaptability.… →

Hypernetworks have gained attention for their ability to efficiently adapt large models or train generative models of neural representations. Despite their effectiveness, training hyper networks are often labor-intensive, requiring precomputed optimized weights for each data sample. This reliance on ground truth weights necessitates significant computational resources, as seen in methods like HyperDreamBooth, where preparing training… →