VERSA
VERSADocs

Installation

Set up VERSA and the metric backends you need.

On this page

Basic Installation

git clone https://github.com/wavlab-speech/versa.git
cd versa
pip install .

The base install keeps dependency resolution light and includes the shared scorer runtime. Install optional groups for metrics that need larger model stacks or external toolkits:

pip install ".[audio,text,ml]"
pip install ".[songeval]" # SongEval Python dependencies only
pip install ".[external]"  # Git/toolkit-backed metrics
pip install ".[dev]"       # tests, linting, and formatting

or alternatively, without cloning:

python -m pip install git+https://github.com/wavlab-speech/versa.git#egg=versa-speech-audio-toolkit --no-build-isolation

Metric-Specific Dependencies

VERSA aligns with original APIs provided by algorithm developers rather than redistributing models. The base package does not install every optional metric backend by default.

For metrics marked without "x" in the "Auto-Install" column of our metrics tables, please use the installers provided in the tools directory.

Some real model-backed tests and metrics also need checkpoint assets that are too large to keep in the package. Prepare those assets in a repo-visible cache before running the full real-model checks:

PYTHON=python tools/setup_huggingface_cache.sh

This populates versa_cache/huggingface and versa_cache/discrete_speech_metrics. To reuse an existing local Hugging Face cache without network access, run:

SOURCE_HF_CACHE="$HOME/.cache/huggingface/hub" \
VERSA_HF_LOCAL_ONLY=1 \
PYTHON=python \
tools/setup_huggingface_cache.sh

Installation Notes

Some optional metric backends emit warnings during setup or first use. ESPnet may print a flash_attn warning when Flash Attention is not available; VERSA can still run metrics that do not require that backend. FADTK is only needed for FAD/KID-style metrics and can be installed with tools/install_fadtk.sh when those metrics are selected.

SongEval is an optional, reference-free metric for full songs. Its upstream toolkit predicts coherence, musicality, memorability, structural clarity, and vocal naturalness on a 1--5 scale. Install its Python dependencies and pinned upstream assets explicitly before scoring:

PYTHON=python tools/install_songeval.sh
python versa/bin/scorer.py \
    --score_config egs/separate_metrics/songeval.yaml \
    --pred path/to/generated_wav.scp \
    --output_file songeval.jsonl \
    --io soundfile \
    --use_gpu

On first use, VERSA downloads a pinned SongEval checkout into versa_cache/SongEval and MuQ into versa_cache/huggingface. For a fully local run, set model_dir, muq_model, and offline: true in the YAML.

MAPSS is an optional multi-source metric for source-separation systems. Its backend supports Python 3.10--3.12 and constrains transformers<4.53, so install it in a compatible environment instead of adding it to the VERSA base install:

PYTHON=python tools/install_mapss.sh
versa-score \
    --score_config egs/separate_metrics/mapss.yaml \
    --pred_sources estimates/source1.scp estimates/source2.scp \
    --gt_sources references/source1.scp references/source2.scp \
    --output_file mapss.jsonl \
    --io soundfile \
    --use_gpu

The SCP lists are positional: predicted source i must estimate reference source i, and every list must contain the same mixture keys. VERSA writes the diagnostic per-source means to JSONL and retains MAPSS's frame-level PS, PM, and confidence tables under versa_cache/mapss/results. The frame-level outputs should be retained for scientific reporting; the PS convenience mean is not the paper's formal utterance-level pooling protocol.

If NLTK downloads fail with a certificate verification error, point Python at the certificate bundle used by certifi before running the tests:

export SSL_CERT_FILE=$(python -c "import certifi; print(certifi.where())")