This paper tries something like that https://arxiv.org/pdf/2202.13914.pdf The ga...

edwardjhu on March 24, 2023 | parent | context | favorite | on: LoRA: Low-Rank Adaptation of Large Language Models

This paper tries something like that

https://arxiv.org/pdf/2202.13914.pdf

The gain isn't that significant. We don't understand what these low-rank updates represent, and they might not correspond to "skills" that humans have.