The Deepseek V4.1 flash 1017 update mentions a self-feedback-based reinforcement learning (RL) training technique. Self-feedback reinforcement learning refers to a training process where the model doesn't rely entirely on external "standard answers" or manually labeled rewards. Instead, it allows the model to evaluate, critique, and validate its own output, treating these self-evaluations as reward signals (or a crucial component of the reward) to optimize model behavior. The official description states that self-feedback-based reinforcement learning enhances the model's ability to build complete projects from scratch (in "wish-granting" scenarios such as game development and 3D modeling). These tasks often lack a clear "standard answer" (unlike math problems with a unique solution), making it difficult to fully cover them through external testing alone. The self-feedback mechanism teaches the model to: self-check and self-correct during the generation process; evaluate the quality of intermediate and final results; and deliver relatively complete projects directly through short rounds of interaction with minimal hints and reference materials. Traditional reinforcement learning (RL) relying solely on external rewards suffers from sparse signals in this scenario, making it difficult for the model to learn "how to complete a project from scratch." "Wish-granting programming" involves the user providing only a high-level description ("create a 3D display of the Forbidden City" or "create a web-based racing game"), requiring the model to plan, write multiple code files, adjust visual/interaction interfaces, and handle various implicit requirements. This type of task lacks a clear ground-truth answer, involves numerous intermediate steps (a typical scenario for long-range agents), and accumulates early errors, making it difficult to automatically score the final quality using simple rules. Reinforcement learning with self-feedback provides denser intermediate rewards, reducing reliance on external annotations and human preferences. There were quite a few papers in this field in the community between 2025 and 2026, and I've read some before. More recently, there have been a lot more in 2026: 1/ Post-Training Large Language Models via Reinforcement Learning from Self-Feedback: Uses the model's own confidence in the answer as an intrinsic reward, generating preference data and then performing preference optimization (RLSF). https://arxiv.org/abs/2507.21931 2/ RLSR: Reinforcement Learning from Self-Reward: The model acts as its own judge (self-judging), providing reward signals without a standard answer, achieving self-improvement. https://arxiv.org/abs/2505.08827 3/ Reflect, Retry, Reward: After the model answers incorrectly, it generates self-reflection, and then uses the success or failure of the second attempt to reward the reflection process (using GRPO). https://arxiv.org/abs/2505.24726 4/ Self-Rewarding Language Models: The model both generates answers and uses... LLM-as-a-Judge: Self-scoring and iterative training https://arxiv.org/abs/2401.10020 5/Self-Evolved Reward Learning for LLMs: Rewarding models to iteratively improve themselves using self-generated feedback https://arxiv.org/abs/2411.00418 6/ReflexiCoder: Teaching LLMs to Self-Reflect on Generated Code and Self-Correct It via RL Using RL to internalize the complete trajectory of "generation → reflection → self-correction" into the model weights, enabling self-correction during inference without an external execution environment https://arxiv.org/abs/2603.05863 7/ICRL: Learning to Internalize Self-Critique with Reinforcement Learning Jointly training the solver and critic, allowing the model to truly internalize the improvements brought by criticism into its own capabilities, in the agent... Significant gains in task performance: https://arxiv.org/abs/2605.15224 8/Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling Addresses the credit assignment problem of sparse rewards in long-horizon agent tasks by using model-generated citations for dense credit assignment. https://arxiv.org/abs/2610.28388 9/Agent-Reflex: Teaching Language Agents to Act Reflectively Systematically trains the agent's proactive reflective ability, combining environmental feedback and RL. https://arxiv.org/abs/2610.17953 10/ReVeal: Self-Evolving Code Agents via Iterative Generation-Verification A multi-round generation-self-verification closed-loop RL that allows generation and verification capabilities to evolve together. https://arxiv.org/abs/2506.11442