---
language:
- en
- hi
- te
- ml
- pa
- ta
- gu
- kn
- bn
- mr
- or
- as
license: cc-by-4.0
task_categories:
- automatic-speech-recognition
size_category: 100K<n<1M
tags:
- asr
- indic
- multilingual
- validation
pretty_name: Indic ASR Validation Set (12 Languages, 300K Samples)
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
dataset_info:
  features:
  - name: audio
    dtype:
      audio:
        sampling_rate: 16000
  - name: segment_id
    dtype: string
  - name: text
    dtype: string
  - name: duration
    dtype: float64
  - name: lang
    dtype: string
  - name: source
    dtype: string
  splits:
  - name: train
    num_bytes: 69982348780
    num_examples: 300000
  download_size: 69673961708
  dataset_size: 69982348780
---

# Indic ASR Validation Set — 12 Languages, 300K Samples

Balanced validation set for evaluating multilingual ASR models on 12 Indian languages.

## Overview

- **Total samples:** 300,000
- **Total duration:** 550 hours
- **Languages:** 12 (English, Hindi, Telugu, Malayalam, Punjabi, Tamil, Gujarati, Kannada, Bengali, Marathi, Odia, Assamese)
- **Samples per language:** ~25,000 (balanced)
- **Transcripts:** Pre-cleaned (no punctuation, no event tags), with `<|lang|>` prefix

## Per-Language Statistics

| Language | Code | Samples | Hours | Avg Duration |
|---|---|---|---|---|
| Assamese | as | 24,999 | 48.7h | 7.0s |
| Bengali | bn | 25,014 | 43.9h | 6.3s |
| English | en | 24,369 | 45.3h | 6.7s |
| Gujarati | gu | 25,004 | 44.0h | 6.3s |
| Hindi | hi | 25,384 | 43.3h | 6.1s |
| Kannada | kn | 25,026 | 47.6h | 6.9s |
| Malayalam | ml | 25,014 | 46.8h | 6.7s |
| Marathi | mr | 24,998 | 46.3h | 6.7s |
| Odia | or | 24,995 | 48.3h | 6.9s |
| Punjabi | pa | 25,018 | 46.0h | 6.6s |
| Tamil | ta | 25,013 | 47.6h | 6.8s |
| Telugu | te | 25,166 | 42.6h | 6.1s |
| **Total** | | **300,000** | **550h** | **6.6s** |

## Data Sources

| Source | Samples |
|---|---|
| final-export | 266,940 |
| indicvoices | 33,060 |

## Schema

| Column | Type | Description |
|---|---|---|
| `segment_id` | string | Unique audio segment identifier |
| `shard_id` | string | Source shard identifier |
| `text` | string | Cleaned transcript with `<|lang|>` prefix |
| `duration` | float | Audio duration in seconds |
| `lang` | string | ISO 639-1 language code |
| `source` | string | Data source name |

## Text Format

Transcripts are pre-cleaned:
- Event tags removed (`[singing]`, `[laugh]`, etc.)
- Punctuation stripped
- Language tag prepended: `<|hi|> namaste kaise ho`
- Script-based language correction applied (v2 manifests)

## Usage

```python
from datasets import load_dataset

ds = load_dataset("BayAreaBoys/indic-asr-validation-set-12lang-300k-samples")
```

## Notes

- This is a **metadata-only** dataset (transcripts + durations). Audio files are not included.
- Extracted from a 63M segment multilingual ASR training corpus.
- Quality filtered: duration 1-20s, transcription quality score >= 0.7.
- No train/val overlap (verified with zero `(shard_id, segment_id)` leakage).