Large Language Models (LLMs) play a vital role in many AI applications, ranging from text summarization to conversational AI. However, evaluating these models effectively remains a significant challenge. Human evaluations, while reliable, often suffer from inconsistency, high costs, and long turnaround times. Automated evaluation tools, particularly those that are closed-source, frequently lack transparency and fail… →

Theory of Mind (ToM) is a foundational element of human social intelligence, enabling individuals to interpret and predict the mental states, intentions, and beliefs of others. This cognitive ability is essential for effective communication and collaboration, serving as a pillar for complex social interactions. Developing systems that emulate this reasoning in AI is crucial for… →

Reasoning systems such as o1 from OpenAI were recently introduced to solve complex tasks using slow-thinking processes. However, it is clear that large language models have limitations, as they cannot plan, break down problems, improve ideas, summarize, or rethink due to their training and methods. While these tools try to enhance reasoning, they depend on… →

The evaluation of LLMs in medical tasks has traditionally relied on multiple-choice question benchmarks. However, these benchmarks are limited in scope, often yielding saturated results with repeated high performance from LLMs, and do not accurately reflect real-world clinical scenarios. Clinical reasoning, the cognitive process physicians use to analyze and synthesize medical data for diagnosis and… →

The robotics and embodied AI field has long struggled with accessibility and efficiency issues. Creating realistic physical simulations requires extensive technical expertise, expensive hardware, and time-consuming manual processes. Existing tools often fail to deliver the speed, accuracy, and user-friendliness needed for widespread adoption, making robotics research an exclusive domain for well-funded institutions. The lack of… →

The rise of large language models (LLMs) has transformed natural language processing, but training these models comes with significant challenges. Training state-of-the-art models like GPT and Llama requires enormous computational resources and intricate engineering. For instance, Llama-3.1-405B needed approx. 39 million GPU hours, equivalent to 4,500 years on a single GPU. To meet these demands… →