softmax

"生成式人工智能不仅仅是“单一模型”，而是一个“多层系统”。"

"文章指出，训练数据的质量和数量，尤其是既定模式和丰富上下文的存在，极大地影响了大型语言模型生成准确可靠结果的能力。"

"生成式人工智能的行为完全受两个数学定律支配：“Softmax Crowding”和“Semantic Drift”。"

"这篇文章解释说，微调只会改变 logits 的分布，并且只要温度 (T) 大于 0，选择不正确标记的概率在数学上永远不会为零。这意味着生成式人工智能的概率性质会导致非确定性赌博。"

"I propose a method called Teacher-Free Self-Distillation (TFSD) that relies on a "Geometric Turn": Metric Regime: Replace the dot product with negative squared Euclidean distance ($z = -|x - c|2$)."

"GRADE-STE achieves a test reward of 0.763 +- 0.344 compared to PPO's 0.510 +- 0.313 and REINFORCE's 0.617 +- 0.378, representing a 50% relative improvement over PPO."

"Softmax takes the raw, unbounded scores produced by a neural network and transforms them into a well-defined probability distribution..."

"MNIST（エムニスト）は、0から9までの手書き数字の画像データセットです。"

"The paper focuses on the relationship between softmax and linear attention in the large-prompt regime."