Efficient long-context inference with LLMs requires managing substantial GPU memory due to the high storage demands of key-value (KV) caching. Traditional KV cache compression techniques reduce memory usage by selectively pruning less significant tokens, often based on attention scores. However, existing methods assess token importance independently, overlooking the crucial dependencies among tokens for preserving semantic… →

As deep learning models continue to grow, the quantization of machine learning models becomes essential, and the need for effective compression techniques has become increasingly relevant. Low-bit quantization is a method that reduces model size while attempting to retain accuracy. Researchers have been determining the best bit-width for maximizing efficiency without compromising performance. Various studies… →

Time series forecasting presents a fundamental challenge due to its intrinsic non-determinism, making it difficult to predict future values accurately. Traditional methods generally employ point forecasting, providing a single deterministic value that cannot describe the range of possible values. Although recent deep learning methods have improved forecasting precision, they require task-specific training and do not… →

In this tutorial, we demonstrate how to efficiently fine-tune the Llama-2 7B Chat model for Python code generation using advanced techniques such as QLoRA, gradient checkpointing, and supervised fine-tuning with the SFTTrainer. Leveraging the Alpaca-14k dataset, we walk through setting up the environment, configuring LoRA parameters, and applying memory optimization strategies to train a model… →

Logical reasoning remains a crucial area where AI systems struggle despite advances in processing language and knowledge. Understanding logical reasoning in AI is essential for improving automated systems in areas like planning, decision-making, and problem-solving. Unlike common-sense reasoning, logical reasoning requires precise rule-based deductions, making it more challenging for LLMs to master. A major obstacle… →

Code generation models have made remarkable progress through increased computational power and improved training data quality. State-of-the-art models like Code-Llama, Qwen2.5-Coder, and DeepSeek-Coder show exceptional capabilities across various programming tasks. These models undergo pre-training and supervised fine-tuning (SFT) using extensive coding data from web sources. However, the application of reinforcement learning (RL) in code generation… →