Architectures · intermediate · concept 92 of 311 · Updated Nov 4, 2026

🧬Model Merging

Combining two or more fine-tuned models into one by operating directly on their weights, with no training. It works when the models share a base checkpoint. Model soups average weights; task arithmetic adds the deltas each fine-tune made; TIES trims small deltas and resolves sign conflicts; DARE randomly drops deltas and rescales the rest. The mergekit toolkit made it a hobbyist pastime, and merges have ranked high on open leaderboards.