“Are you pruning your neural networks? "Delete parameters with small weights!" or "Gradients..."”

““Overall, our empirical observations strongly indicate that TTT-E2E should produce the same trend as full attention for scaling with training compute in large-budget production runs.””

“Based on conversations with Gemini, the article is constructed.”

“はじめに ディープラーニングの実装をしているとベクトル微分とかを頻繁に目にしますが、具体的な演算の定義を改めて確認したいなと思い、まとめてみました。”

“”

“DeepSeek mHC reimagines some of the established assumtions about AI scale.”

“I'm looking for resources to study the following: -statistics and probability -calculus (for applications like optimization, gradients, and understanding models) ... I don't want to study the entire math courses, just what is necessary for AI/ML.”

“DeepSeek solved the instability by constraining the learnable matrices to be "Double Stochastic" (all elements ≧ 0, rows/cols sum to 1). Mathematically, this forces the operation to act as a weighted average (convex combination). It guarantees that signals are never amplified beyond control, regardless of network depth.”

“DeepSeek solved the instability by constraining the learnable matrices to be "Double Stochastic" (all elements ≧ 0, rows/cols sum to 1).”

“Editor's note: This article is a part of our series on visualizing the foundations of machine learning.”

“The paper shows that the generalization error of DGFMs tends to zero as the number of neurons and the training time tend to infinity.”

“The basic inequality upper bounds f(θ_T)-f(z) for any reference point z in terms of the accumulated step sizes and the distances between θ_0, θ_T, and z.”

“The gradient expansion includes an unexpected zeroth order term depending on the differences between thermo-hydrodynamic fields at the decoupling and the initial hypersurface. This term encodes a memory of the initial state...”

“The paper proposes using the gradient cosine similarity of low-confidence examples to predict data efficiency based on a small number of labeled samples.”

“DMSAEs run an iterative distillation cycle: train a Matryoshka SAE with a shared core, use gradient X activation to measure each feature's contribution to next-token loss in the most nested reconstruction, and keep only the smallest subset that explains a fixed fraction of the attribution.”

“ADOPT explicitly models the dependency between each LLM step and the final task outcome, enabling precise text-gradient estimation analogous to computing analytical derivatives.”

“Even the top-performing OpenAI-GPT-5.1 achieves only 62.07% accuracy, and model performance displays a clear gradient distribution.”

“The number of crack spikes increases with the viscosity of the subphase.”

“For any objective with log-sum-exp structure over distances or energies, the gradient with respect to each distance is exactly the negative posterior responsibility of the corresponding component: $\partial L / \partial d_j = -r_j$.”

“Under mild assumptions, the sequence generated by the proposed algorithm is bounded and each of its cluster points is a stationary solution.”

“The paper presents the first resource-adaptive distributed bilevel optimization framework with a second-order free hypergradient estimator.”

“The paper focuses on gradient estimation in the context of functions with or without non-independent variables.”

“The newly proposed mCCAdL thermostat achieves a substantial improvement in the numerical stability over the original CCAdL thermostat, while significantly outperforming popular alternative stochastic gradient methods in terms of the numerical accuracy for large-scale machine learning applications.”

“HOLOGRAPH provides rigorous mathematical foundations while achieving competitive performance on causal discovery tasks.”

“The paper proposes a novel sparse-penalization framework for high-dimensional Pconf classification.”

“The study finds that the GPA does not generally hold for these systems under moderate experimental conditions.”

“The paper demonstrates that implicit score matching achieves the same rates of convergence as denoising score matching and allows for Hessian estimation without the curse of dimensionality.”

“The speed of information displacement is linearly related to the ratio of odd vs total kernel energy.”

“OptiVote integrates sign stochastic gradient descent (signSGD) with a majority-vote (MV) aggregation principle and pulse-position modulation (PPM), where each satellite conveys local gradient signs by activating orthogonal PPM time slots.”

“itePGDK outperformed these methods in these metrics. Particularly in short duration frames, itePGDK presents less bias and less artifacts in fast kinetics organs uptake compared with DeepKernel.”

“The paper proposes a gradient-based algorithm with lower per-iteration cost than existing methods and adapts it to exploit the piecewise-linear structure of ReLU networks.”

“The paper proposes a method that trains a neural network to predict the minimum distance between the robot and obstacles using latent vectors as inputs. The learned distance gradient is then used to calculate the direction of movement in the latent space to move the robot away from obstacles.”

“DATAMASK achieves significant improvements of 3.2% on a 1.5B dense model and 1.9% on a 7B MoE model.”

“The proposed loss introduces learnable class prototypes and equilibrates gradients contributed by different classes at each hierarchical level, ensuring that each hierarchical class contributes equally to the loss computation in every mini-batch.”

“MeLeMaD outperforms state-of-the-art approaches, achieving accuracies of 98.04% on CIC-AndMal2020 and 99.97% on BODMAS.”

“The paper claims an enhanced convergence rate of order $\mathcal{O}(h)$ in the $L^2$-Wasserstein distance, significantly improving the existing order-half convergence.”

“Hydrogen concentration gets localized in the colder region of the body (Soret effect).”

“Trellis replaces the standard KV cache with a fixed-size memory and train a two-pass recurrent compression mechanism to store new keys and values into memory.”

“The results show that attention-based adversarial examples lead to measurable drops in evaluation performance while remaining semantically similar to the original inputs.”

“Our results provide a natural explanation for long-standing experimental observations of spin injection in superconductors and predict novel effects arising from spin-charge coupling, including the electrical control of anomalous phase gradients in superconducting systems with spin-orbit coupling.”

“”

“DSC models the weight update as a residual trajectory within a Star-Shaped Domain, employing a Magnitude-Gated Simplex Interpolation to ensure continuity at the identity.”

“ISOPO normalizes the log-probability gradient of each sequence in the Fisher metric before contracting with the advantages.”

“By working through the backward pass manually, we gain a deeper intuition for how each operation influences the final output.”

“この論文で紹介されたある**「単純すぎるテクニック」**が、当時の研究者たちを驚かせました。”

“LogosQ leverages Rust static analysis to eliminate entire classes of runtime errors, particularly in parameter-shift rule gradient computations for variational algorithms.”

“”

“”

“The paper presents an alternating projected gradient descent and minimization algorithm for recovering a low-rank feature matrix in a diffusion-based decentralized and federated fashion.”

“The Dense Gradient admits a closed-form logit-level formula, enabling efficient GPU implementation.”