Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2212.04089

Model Merging is a very popular technique nowadays in LLM. Here is a chronological list of papers on the space that will help you get started with it!

Qualitatively characterizing neural network optimization problems

Paper • 1412.6544 • Published Dec 19, 2014 • 4
Convergent Learning: Do different neural networks learn the same representations?

Paper • 1511.07543 • Published Nov 24, 2015 • 2
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models

Paper • 1909.11299 • Published Sep 25, 2019 • 2
Model Fusion via Optimal Transport

Paper • 1910.05653 • Published Oct 12, 2019 • 1

Meta-Learning a Dynamical Language Model

Paper • 1803.10631 • Published Mar 28, 2018
TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation

Paper • 2003.11963 • Published Mar 26, 2020
BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model

Paper • 2212.04960 • Published Dec 9, 2022 • 1
Continuous Learning in a Hierarchical Multiscale Neural Network

Paper • 1805.05758 • Published May 15, 2018 • 1

Model Merging Papers

Collection of relevant papers about model merging

Qualitatively characterizing neural network optimization problems

Paper • 1412.6544 • Published Dec 19, 2014 • 4
Averaging Weights Leads to Wider Optima and Better Generalization

Paper • 1803.05407 • Published Mar 14, 2018 • 2
Merging Models with Fisher-Weighted Averaging

Paper • 2111.09832 • Published Nov 18, 2021 • 1
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Paper • 2203.05482 • Published Mar 10, 2022 • 6

Papers about model merging

referenced in the mergekit repo: https://github.com/cg123/mergekit

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Paper • 2203.05482 • Published Mar 10, 2022 • 6
Editing Models with Task Arithmetic

Paper • 2212.04089 • Published Dec 8, 2022 • 6
Resolving Interference When Merging Models

Paper • 2306.01708 • Published Jun 2, 2023 • 13
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

Paper • 2311.03099 • Published Nov 6, 2023 • 29

mistralai/Mixtral-8x7B-Instruct-v0.1

Text Generation • Updated Aug 19, 2024 • 488k • • 4.3k
mistralai/Mistral-7B-Instruct-v0.2

Text Generation • Updated Sep 27, 2024 • 3.24M • • 2.65k
Editing Models with Task Arithmetic

Paper • 2212.04089 • Published Dec 8, 2022 • 6
Orbina/Orbita-v0.1

Text Generation • Updated Apr 29, 2024 • 2.81k • 39

Company

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs