Loading
Life sciences · Preprint
arXiv · August 18, 2026
Posted before peer review. The findings may change or fail to hold.
CORAM is a novel algorithmic approach to merging fine-tuned language and vision-language models that uses orthogonal transformations and singular value decomposition to preserve the geometry of model updates. The method is compared against existing baselines (OrthoMerge and weight-space methods) on four evaluation suites across three model families (3B–9B parameters), showing improvements of 0.25–1.35 points over OrthoMerge. This is an early-stage technical contribution that has not been peer reviewed and does not include evidence of generalization beyond the tested benchmarks.
Preprint. Intervention: CORAM: partitioning each weight matrix into row slices, representing expert slices by singular value decomposition in base-model SVD frame, merging task-specific factors on corresponding manifolds, and applying amplification coefficient λ…. Compared with: OrthoMerge (single orthogonal transform per weight matrix) and weight-space baseline methods.
CORAM improves over OrthoMerge by 0.25 to 1.35 points across four evaluation suites Amplification coefficient λ = κĉ, where ĉ is approximately √N for N experts with comparable update magnitudes Restoration strength κ selected without evaluating candidate merged models remains within 0.72 points of best swept value across all evaluated suites
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
This is an unrefereed arXiv preprint describing a novel machine learning algorithm for model merging; it presents a technical method with experimental comparisons but has not undergone peer review.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
Merging finetuned models combines specialized capabilities without joint training or access to the original data. Most methods operate by linear arithmetic in Euclidean weight space, which cannot carry the geometry of the update. Orthogonal Model Merging (OrthoMerge) uses a single orthogonal transform for each weight matrix, but such a transform cannot change singular values. We propose CORAM, which partitions each target matrix into row slices, represents every expert slice by its singular value decomposition in the corresponding base-model SVD frame, and merges the task-specific factors on their corresponding manifolds. Because manifold averaging contracts the merged update, CORAM applies an amplification coefficient $λ=κ\hat{c}$. The scale c_hat is estimated from the expert and merged update norms and is approximately $\sqrt{N}$ for $N$ experts with comparable update magnitudes. The restoration strength kappa is selected from the dispersion of expert updates without evaluating candidate merged models. This rule remains within 0.72 points of the best swept value on all evaluated suites. CORAM also includes spread slicing to distribute highly updated rows across slices and a residual pathway for non-target layers. Across four suites covering three model families, 3B to 9B scales, and language and vision-language experts, CORAM improves over OrthoMerge by 0.25 to 1.35 points and matches or exceeds the strongest weight-space baselines.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.