Title | Implicit dynamic regularization in deep networks |
Publication Type | CBMM Memos |
Year of Publication | 2020 |
Authors | Poggio, TA, Liao, Q |
Date Published | 08/2020 |
Abstract | Square loss has been observed to perform well in classification tasks, at least as well as crossentropy. However, a theoretical justification is lacking. Here we develop a theoretical analysis for the square loss that complements the existing asymptotic analysis for the exponential loss. |
DSpace@MIT |
Download:
TPR_ver2.pdf
Substantial edits
Edits that are extensive but minor in content
Extending theory, setting a post
Fine tuning
Corrections in Appendix about Neural Collapse
Small edits clarifying role of weight decay
Added: prove NC for multiclass+theorem on connected global minima
CBMM Memo No:
112 







Associated Module:
CBMM Relationship:
- CBMM Funded