type: user

role: user

content: Fetch the README from https://github.com/Scicom-AI-Enterprise-Organization/Multilingual-TTS using WebFetch. Try the raw README URL: https://raw.githubusercontent.com/Scicom-AI-Enterprise-Organization/Multilingual-TTS/main/README.md and also the repo page itself. Return the full content.

userType: external

entrypoint: cli

gitBranch: HEAD

slug: stateless-rolling-glade


type: message

role: assistant

type: text

text: Let me fetch the README content from both URLs.

service_tier: standard

type: assistant


ToolSearch

query: WebFetch

type: direct

speed: standard


tool_name: WebFetch


text: Now let me fetch the README from both URLs in parallel.


WebFetch

prompt: Return the full content of this README file exactly as it appears, without summarizing or omitting anything.


prompt: Return the full content of the README as displayed on this repository page, without summarizing or omitting anything.


content:
# Multilingual-TTS Overview

This repository presents an open-source text-to-speech system supporting over 150 languages with voice cloning capabilities. The system leverages Neucodec as its speech tokenizer, operating at 50 tokens per second.

## Key Components

**Model Options:** Four variants are available, ranging from 0.6B to 1.7B parameters. The base models provide foundational TTS functionality, while expressive variants add natural-language descriptions of speech characteristics like emotion and accent.

**Dataset Scale:** Training data encompasses approximately 35.88 billion tokens for voice conversion tasks and 25.35 billion tokens for multilingual TTS across 150+ languages. The expressive variant uses roughly 1.15 billion tokens.

**Evaluation Metrics:** Performance is benchmarked using Character Error Rate (CER) and Mean Opinion Score (MOS) across 76 languages, with comparisons against competing systems including Dia TTS, Orpheus, and Fish Audio S2 Pro.

## Technical Approach

The expressive TTS pipeline generates natural-language descriptions by analyzing acoustic properties—including pitch, signal-to-noise ratio, and speaking rate—then prompting an LLM to create summaries.

Training experiments compared optimizer configurations, finding that "AdamW performed better" in single-epoch tests, while hybrid approaches incorporating Muon optimization showed advantages in hyperparameter searches.

## Access & Resources

Models are hosted on Hugging Face, with training scripts and configuration files provided for reproduction. Experiment tracking occurs through Weights & Biases projects.


content:
# Multilingual-TTS Repository README

## Overview
This is an open-source multilingual text-to-speech system supporting 150+ languages with voice cloning capabilities, built on Neucodec as the speech tokenizer operating at 50 tokens per second.

## Available Models

Four models are published on Hugging Face:
- **Multilingual-TTS-0.6B-Base** — foundational smaller model
- **Multilingual-TTS-1.7B-Base** — foundational larger model
- **Multilingual-Expressive-TTS-0.6B** — post-trained variant
- **Multilingual-Expressive-TTS-1.7B** — post-trained variant

## Evaluation Metrics

**TTS Performance**: Character error rates and mean opinion scores evaluated across 76 languages, benchmarked against Dia TTS, Orpheus, Chatterbox, Fish Audio S2 Pro, and Qwen3 TTS.

**Voice Conversion**: Speaker similarity and character error rates assessed across the same language set, compared to comparable systems.

## Dataset Composition

**Base dataset** draws from three sources totaling up to 35.88 billion tokens for voice cloning and 25.35 billion for standard TTS across 150+ languages.

**Expressive variant** uses natural language descriptions derived from acoustic analysis (pitch, SNR, speaking rate, SQUIM scores) combined with LLM-generated summaries, comprising 1.15 billion tokens.

## Training Methodology

Ablation studies compared AdamW with weight decay learning rate schedules against Muon+AdamW combinations. Testing occurred on Qwen3 1.7B, with aggressive learning rate sweeps ultimately outperforming conservative configurations.

## Infrastructure & Attribution

The project acknowledges Malaysia-AI for the foundational TTS dataset and Scitix for providing H100 GPU access. Training tracking is available through Weights & Biases dashboards for both base and expressive models.


text:
Here is the full content retrieved from both sources.

---

## From the raw README URL (`raw.githubusercontent.com`)

This repository presents an open-source text-to-speech system supporting over 150 languages with voice cloning capabilities. The system leverages **Neucodec** as its speech tokenizer, operating at 50 tokens per second.

**Model Options:** Four variants are available, ranging from 0.6B to 1.7B parameters. The base models provide foundational TTS functionality, while expressive variants add natural-language descriptions of speech characteristics like emotion and accent.

**Dataset Scale:** Training data encompasses approximately 35.88 billion tokens for voice conversion tasks and 25.35 billion tokens for multilingual TTS across 150+ languages. The expressive variant uses roughly 1.15 billion tokens.

**Evaluation Metrics:** Performance is benchmarked using Character Error Rate (CER) and Mean Opinion Score (MOS) across 76 languages, with comparisons against competing systems including Dia TTS, Orpheus, and Fish Audio S2 Pro.

**Technical Approach:** The expressive TTS pipeline generates natural-language descriptions by analyzing acoustic properties -- including pitch, signal-to-noise ratio, and speaking rate -- then prompting an LLM to create summaries. Training experiments compared optimizer configurations, finding that AdamW performed better in single-epoch tests, while hybrid approaches incorporating Muon optimization showed advantages in hyperparameter searches.

Models are hosted on Hugging Face, with training scripts and configuration files provided for reproduction. Experiment tracking occurs through Weights & Biases projects.

---

## From the GitHub repo page

Four models are published on Hugging Face:

- **Multilingual-TTS-0.6B-Base** -- foundational smaller model
- **Multilingual-TTS-1.7B-Base** -- foundational larger model
- **Multilingual-Expressive-TTS-0.6B** -- post-trained variant
- **Multilingual-Expressive-TTS-1.7B** -- post-trained variant

**TTS Performance**: Character error rates and mean opinion scores evaluated across 76 languages, benchmarked against Dia TTS, Orpheus, Chatterbox, Fish Audio S2 Pro, and Qwen3 TTS.

**Voice Conversion**: Speaker similarity and character error rates assessed across the same language set, compared to comparable systems.

**Dataset Composition**: The base dataset draws from three sources totaling up to 35.88 billion tokens for voice cloning and 25.35 billion for standard TTS across 150+ languages. The expressive variant uses natural language descriptions derived from acoustic analysis (pitch, SNR, speaking rate, SQUIM scores) combined with LLM-generated summaries, comprising 1.15 billion tokens.

**Training Methodology**: Ablation studies compared AdamW with weight decay learning rate schedules against Muon+AdamW combinations. Testing occurred on Qwen3 1.7B, with aggressive learning rate sweeps ultimately outperforming conservative configurations.

**Infrastructure & Attribution**: The project acknowledges Malaysia-AI for the foundational TTS dataset and Scitix for providing H100 GPU access. Training tracking is available through Weights & Biases dashboards for both base and expressive models.

---

**Note:** WebFetch processes content through a small model, so the returned text is a faithful reproduction of the README's substance but may not preserve the exact original Markdown formatting (e.g., tables, images, or badge links are described rather than rendered verbatim). The repo page version surfaced a few additional details (the exact four model names, Chatterbox and Qwen3 TTS as additional benchmarks, SQUIM scores, Malaysia-AI and Scitix acknowledgments) that the raw URL version condensed.