Jan 28, 2022 · However, in this paper, we find that fine-tuning can achieve worse accuracy than linear probing out-of-distribution (OOD) when the pretrained features are good and the …

rameterized two-layer linear networks. We prove that the OOD error of fine-tuning is high when we initialize with a fixed or random head—this is because while fine-tuning learns the head, …

Can we refine features without distorting them too much? +10% over fine-tuning! What to do when linear probing not so good?

Fine-tuning will distort the pre-trained feature extractor, resulting in lower OOD accuracy, but initializing with a linear probe fixes it. Therefore, LP-FT is adopted to obtain better ID and OOD …

•Pretrained models give large improvements in accuracy, but how we fine-tune them is key •LP-FT is just a starting point, better methods? •What to do when linear probing not so good?

May 15, 2023 · しかし、本論文では、事前学習済みの特徴が良好で、分布のシフトが大きい場合、ファインチューニングが線形プロービングよりもOOD（分布外）で精度が低くなること …

rameterized two-layer linear networks. We prove that the OOD error of fine-tuning is high when we initialize with a fixed or random head—this is because while fine-tuning learns the head, …

However, in this paper, we find that fine-tuning can achieve worse accuracy than linear probing out-of-distribution (OOD) when the pretrained features are good and the distribution shift is large.

fine-tuning does worse than linear probing out-of-distribution (OOD). We consider a linear setting (feature extractor gB is linear) where the pretra ned features are “good” and the OOD shift is …

Feb 21, 2022 · The paper demonstrates that full fine-tuning achieves roughly 2% higher accuracy in-distribution but suffers from a 7% reduction in OOD accuracy compared to linear probing, …