Deschide articolul original într-un tab nou Schimbarea acestei opțiuni se aplică imediat și reîncarcă pagina.
LECTURA
Înapoi la articole →

main

main-horse.github.io/index.xml
Vizitează site-ul

Cel mai recent articol

Ultima verificare

Urmăritori 0 urmăritori
Articole: 89

  1. 4 - Graph Building Process

    (main)

  2. 3 - Intra-machine Topology Analysis

    (main)

  3. 2 - Establishment of Bootstrap Network Connection

    (main)

  4. 1 - Initialization and Generation of ncclUniqueId

    (main)

  5. Why reduction precision matters

    (main)

  6. Understanding Adaptive Learning Rate Optimizers from the Perspective of Hessian Approximation

    (main)

  7. How Adam's Epsilon Affects the Learning Rate Scaling Law?

    (main)

  8. How Should the Learning Rate Change When Batch Size Increases?

    (main)

  9. Adding a Linear Transformation to the Codebook

    (main)

  10. SVD

    (main)

  11. 18. Principle for RoPE's Base Selection

    (main)

  12. 17. Simple Thoughts on Multimodal Positional Encoding

    (main)

  13. 16. A 'Review' of Length Extrapolation Techniques

    (main)

  14. 15. Key Normalization Boosts Length Extrapolation

    (main)

  15. VQing the Key Makes Transformer Complexity Linear

    (main)

  16. 'Rounding' Surpasses VQ-VAE

    (main)

  17. 14. When HWFA Meets ReRoPE

    (main)

  18. 13. Inverse Use of Leaky ReRoPE

    (main)

  19. 12. Infinitely Extrapolatable ReRoPE?

    (main)

  20. 11. Carrying the Base-β Position to the End

    (main)

  21. 10. RoPE is a β-Based Encoding

    (main)

  22. Exploring the Path to Minimums

    (main)

  23. 9. A New Approach to Global Length Extrapolation

    (main)

  24. 8. Length Extrapolation and Positional Robustness

    (main)

  25. 7. Length Extrapolation and Local Attention

    (main)

  26. 6. Completeness Analysis of Rotational Positional Encoding

    (main)

  27. What's so Difficult About Training a 1000-Layer Transformer?

    (main)

  28. Thoughts on Dimension Averaging Strategies for Non-Square Matrices in Initialization Methods

    (main)

  29. A Brief Discussion on Initialization, Parameterization, and Normalization of Transformers

    (main)

  30. 5. Linear Attention as Infinite Dimension

    (main)

  31. 4. Rotary Position Embedding for 2D Positions

    (main)

  32. 3. From Performer to Linear Attention

    (main)

  33. 2. Rotary Position Embedding with Diverse Strengths

    (main)

  34. 1. Tracing the Origins of Sinusoidal Position Encoding

    (main)

  35. Understanding Model Parameter Initialization Strategies from a Geometric Perspective

    (main)

  36. Distribution of the Angle Between Two Random Vectors in $n$-Dimensional Space

    (main)

  37. What Does BN Actually Do? An Analysis From First Principles

    (main)

  38. Vector Quantized AutoEncoder

    (main)

  39. Generalization and Generative Models

    (main)