← Back
YaRN
technique
Yet another RoPE extensioN - a technique for extending the context length capabilities of models using rotary position embeddings.
Topics
Also mentioned
(1)
Casual references without a clear endorsement
Sebastian Raschka
mentioned
"They had a YaRN extension and there was some custom scaling there, and I couldn't quite match the..."
▶ 2:02:29