ByteDance Researchers Introduce DetailFlow: A 1D Coarse-to-Fine Autoregressive Framework for Faster, Token-Efficient Image Generation

Autoregressive image generation has evolved significantly due to advancements in sequential modeling, initially seen in natural language processing. This approach generates images one token at a time, akin to constructing sentences in language models. The primary advantage lies in maintaining structural coherence while allowing for high control during the generation process. Researchers applying these techniques to visual data have found that structured prediction not only preserves spatial integrity but also effectively supports tasks such as image manipulation and multimodal translation.

However, generating high-resolution images remains computationally expensive and slow. A significant challenge is the number of tokens required to represent complex visuals. Raster-scan methods that flatten 2D images into linear sequences necessitate thousands of tokens for detailed images, leading to prolonged inference times and high memory consumption. For instance, models like Infinity require over 10,000 tokens for a 1024×1024 image, making them unsustainable for real-time applications or when scaling to larger datasets. Thus, reducing the token burden while maintaining or enhancing output quality has become a pressing challenge.

To address token inflation, innovations such as next-scale prediction have emerged, as seen in VAR and FlexVAR. These models generate images by predicting progressively finer scales, mimicking the human tendency to sketch rough outlines before adding detail. Nevertheless, they still rely on hundreds of tokens—680 in the case of VAR and FlexVAR for 256×256 images. Approaches like TiTok and FlexTok utilize 1D tokenization to compress spatial redundancy but often struggle to scale efficiently. For example, FlexTok’s gFID increases from 1.9 at 32 tokens to 2.5 at 256 tokens, indicating a degradation in output quality as the token count rises.

In response to these challenges, researchers from ByteDance introduced DetailFlow, a 1D autoregressive image generation framework. This method organizes token sequences from global to fine detail through a process called next-detail prediction. Unlike traditional 2D raster-scan or scale-based techniques, DetailFlow employs a 1D tokenizer trained on progressively degraded images. This design allows the model to prioritize foundational image structures before refining visual details. By mapping tokens directly to resolution levels, DetailFlow significantly reduces token requirements, enabling images to be generated in a semantically ordered, coarse-to-fine manner.

The mechanism in DetailFlow focuses on a 1D latent space where each token contributes incrementally more detail. Earlier tokens encode global features, while later tokens refine specific visual aspects. To train this model, researchers developed a resolution mapping function linking token count to target resolution. During training, the model is exposed to images of varying quality levels and learns to predict progressively higher-resolution outputs as more tokens are introduced. It also implements parallel token prediction by grouping sequences and predicting entire sets simultaneously. To mitigate sampling errors introduced by parallel prediction, a self-correction mechanism was integrated. This system perturbs certain tokens during training and teaches subsequent tokens to compensate, ensuring that final images maintain structural and visual integrity.

Results from experiments on the ImageNet 256×256 benchmark were notable. DetailFlow achieved a gFID score of 2.96 using only 128 tokens, outperforming VAR at 3.3 and FlexVAR at 3.05, both of which utilized 680 tokens. Furthermore, DetailFlow-64 reached a gFID of 2.62 using 512 tokens. In terms of speed, it delivered nearly double the inference rate of VAR and FlexVAR. An ablation study confirmed that self-correction training and semantic ordering of tokens significantly improved output quality. For instance, enabling self-correction reduced the gFID from 4.11 to 3.68 in one setting. These metrics demonstrate both higher quality and faster generation compared to established models.

By focusing on semantic structure and reducing redundancy, DetailFlow offers a viable solution to long-standing issues in autoregressive image generation. The method’s coarse-to-fine approach, efficient parallel decoding, and self-correction capabilities illustrate how architectural innovations can address performance and scalability limitations. Through their structured use of 1D tokens, ByteDance researchers have developed a model that maintains high image fidelity while significantly reducing computational load, making it a valuable addition to image synthesis research.

Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 95k+ ML SubReddit and Subscribe to our Newsletter.