EG-VAE: A Unified Framework for Electric Guitar Tone Transfer and Removal

Yen-Tung Yeh1, Yun-Ning (Amy) Hung2, Yi-Hsuan Yang1
1National Taiwan University, 2Moises
Paper

Abstract

Electric guitar tone transfer (EGTT) and tone removal (EGTR) are two fundamental, highly related tasks in guitar tone modeling: EGTT replaces a recording's tone with that of a reference, while EGTR recovers the dry direct-input (DI) signal from a wet processed recording. Despite their highly related nature, prior work has addressed them independently, and neither has yet achieved satisfactory results. In this paper, we propose EG-VAE, a unified framework that jointly models EGTT and EGTR by disentangling frame-level content and global tone representations from wet recordings with a variational autoencoder. Transfer recombines a source's content with a reference's tone, while removal is realized by a novel tone masking objective that serves a dual role: enforcing content–tone disentanglement during training and realizing the removal procedure at inference. To improve transfer to tones unseen in training, a second training stage shapes a smooth tone space through variational sampling and audio-effects augmentation. Experimental results from both objective and subjective evaluations demonstrate that EG-VAE outperforms task-specific baselines on transfer and remains competitive with models on removal.

Two Tasks: Tone Transfer and Tone Removal

In a typical electric guitar recording, the dry direct-input (DI) signal is processed by a chain of effects pedals and an amplifier, producing the wet signal heard by the listener. The character imparted by this chain is the recording's tone. We study two underexplored problems that act on the tone itself:

  • Tone Removal (EGTR): recover the dry DI signal from a wet recording, inverting the rendering process.
  • Tone Transfer (EGTT): replace the tone of a wet recording with that of another wet reference, while preserving the played content.

The two are complementary: they manipulate one shared object — the recording's tone — in opposite directions. Removal strips the tone away to recover the dry signal, while transfer replaces it with another. This shared structure motivates modeling both tasks jointly.

The two tone-modeling tasks. A dry DI signal is rendered into a wet recording by pedals and an amplifier (green). Tone removal (red) recovers the dry signal; tone transfer (blue) replaces the tone with that of a reference.

Proposed Model: EG-VAE

EG-VAE factorizes a wet recording into a frame-level content embedding \(\mathbf{z}_{1:T'}\) and a global tone embedding \(\mathbf{s}\) through a variational autoencoder built on the DAC-VAE backbone. A shared encoder maps the wet input to features \(\mathbf{h}_{1:T'}\), which feed a content extractor (a per-frame variational bottleneck) and a tone extractor (a Transformer with mean pooling and a variational head). A conditional layer normalization (CLN) module combines the two embeddings before decoding to the waveform.

Both tasks are realized by manipulating the tone embedding while preserving the content. For transfer, \(\mathbf{s}\) is replaced with the tone embedding of a reference recording; for removal, the tone is masked out so the content embedding reconstructs the dry DI signal alone.

Overview of EG-VAE. A shared encoder feeds a content extractor and a tone extractor; their embeddings are combined by CLN and decoded. Dashed boxes are attribute-supervision heads; pvpGD and tone masking support disentanglement and removal.

Tone Masking: One Operation for Removal

CLN modulates the content embedding through \(\boldsymbol{\gamma}(\mathbf{s})\) and \(\boldsymbol{\beta}(\mathbf{s})\): \(\tilde{\mathbf{z}}_t = \boldsymbol{\gamma}(\mathbf{s}) \odot \text{LN}(\mathbf{z}_t) + \boldsymbol{\beta}(\mathbf{s})\). Tone masking overrides this modulation by setting \(\boldsymbol{\gamma}=\mathbf{1}\) and \(\boldsymbol{\beta}=\mathbf{0}\), reducing CLN to plain layer normalization so the content embedding passes through unmodulated and the decoder produces the dry DI signal.

This single mechanism plays a dual role: during training it is a disentanglement objective that forces tone information out of the content embedding, and at inference it is the EGTR procedure itself — removal needs no extra module or fine-tuning, just the masked forward pass.

Tone conditioning versus tone masking, sharing the same CLN operation. Conditioning produces gamma and beta from the tone embedding (wet output); masking overrides them to gamma=1, beta=0, yielding the dry DI signal.

Audio Examples (Listening Test)

The following sections present the audio samples used in our subjective listening test across the two tasks: tone transfer (EGTT) and tone removal (EGTR). For each task we report two settings: seen tones, whose signal chains appear in training (Archetype: Cory Wong X), and unseen tones, which assess generalization to chains held out from training (Morgan Amps Suite). The Oracle column is the ground-truth target, included as an upper-bound anchor. The proposed EG-VAE is highlighted in the last column.

Electric Guitar Tone Transfer (EGTT)

The played content of a Source recording is combined with the tone of a Reference recording. The output should match the Oracle — the source content rendered with the reference tone. Baselines marked w/ EGTR are given a tone-removed (dry) input recovered by EG-VAE before applying the reference tone.

Seen Tones — Archetype: Cory Wong X

Source Tone Ref. Oracle DeepAFx DeepAFx w/ EGTR One-to-many One-to-many w/ EGTR EG-VAE (Proposed)
Example 1
Example 2
Example 3

Unseen Tones — Morgan Amps Suite

Source Tone Ref. Oracle DeepAFx DeepAFx w/ EGTR One-to-many One-to-many w/ EGTR EG-VAE (Proposed)
Example 1
Example 2
Example 3

Electric Guitar Tone Removal (EGTR)

Given a Wet Input, each method recovers the dry DI signal. The output should match the Dry (Oracle) ground truth, i.e., the recording with all signal-chain processing removed.

Seen Tones — Archetype: Cory Wong X

Wet Input Dry (Oracle) HDemucs DCUNet DPTNet Distortion Recovery EG-VAE (Proposed)
Example 1
Example 2
Example 3

Unseen Tones — Morgan Amps Suite

Wet Input Dry (Oracle) HDemucs DCUNet DPTNet Distortion Recovery EG-VAE (Proposed)
Example 1
Example 2
Example 3

Tone Removal Analysis

On a distortion-heavy example, the difficulty lies in the harmonic region (red boxes): nonlinear distortion introduces dense high-order harmonics absent from the dry DI. The baselines leave large residuals (right column), either over-suppressing or failing to remove the added harmonics, whereas EG-VAE's residual is visibly smaller — its content representation, trained to reconstruct the dry DI under tone masking, captures it more accurately than enhancement-based baselines.

Tone removal on a distortion-heavy tone. Top: wet input and target DI. For each method: predicted DI (left, log-magnitude dB) and the residual between prediction and target (right).

Tone Representation

We visualize the learned tone embeddings \(\mathbf{s}\) with t-SNE, colored by preset. For seen tones, the embeddings form clear, well-separated clusters: recordings sharing a signal chain map to nearby points while different presets occupy distinct regions, indicating that \(\mathbf{s}\) captures tone identity. For unseen tones, the clusters remain identifiable but less separated, consistent with the seen–unseen performance gap in transfer and removal.

t-SNE visualization of the tone embeddings, colored by preset, for seen (left) and unseen (right) tones.

BibTeX

BibTex Code Here