Grant Sanderson
Connecting cross-entropy to language models
Part two of Compression is Intelligence connects data compression with the loss function used to train large language models, then explores distillation and KL divergence.
#machine learning#information theory
Compiled from a public source and linked above. Happenings link out to where they were published.
No notes yet