Exciting New Methods for Efficient Fine-Tuning of LLMs using PEFT (BOFT, VeRA, and PiSSA)

Author(s): Mandar Karhade, MD. PhD.

Originally published on Towards AI.

The latest update to Huggingface’s PEFT v0.11.0 introduces several new Parameter-Efficient Fine-Tuning (PEFT) techniques (BOFT, VeRA, and PiSSA)

PEFT has been at the forefront of fine-tuning techniques due to its relative simplicity in application and less reliance on the high-end hardware. The new release of PEFT methods should increase the performance of the fine-tuning of large models.

BOFT: Parameter-Efficient Orthogonal Fine-Tuning via Butterfly FactorizationVeRA: Vector-based Random Matrix AdaptationPiSSA: Principal Singular Values and Singular Vectors Adaptation

Especially for tuning large models > 70B for niche domains (in my case, healthcare) could prove to be a boost. here is a quick review of what the current update looks like.

BOFT stands for Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization. Butterfly factorization as a data-sparse approximation for the matrices that satisfy a complementary low-rank property. The factorization can be constructed efficiently if either fast algorithms for applying the matrix and its adjoint are available or the entries of the matrix can be sampled individually, resulting in factorization is a product of O(log N) sparse matrices, each with O(N) nonzero entries. Hence, it can be applied rapidly in O(N log N) operations (https://web.stanford.edu/~lexing/BF.pdf)

It extends the concept of “Orthogonal Finetuning (OFT)” by reparameterizing pre-trained weight matrices into a block diagonal structure within an orthogonal matrix. This approach preserves the information in the pre-trained model while reducing the… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI