o i I@sdZddlmZddlZddlZddlZddlmZmZddl m Z ddl Z ddl m mZddlmZddlmZddlmZeeZd Zd Zd ZeeeZed Zd ZdZ eGdddZ!GdddZ"dS)uHot GPU encoder: keeps XCodec2 warm and processes batched segments. XCodec2-only mode for SFT data encoding. BiCodec removed entirely. The encoder stays loaded for the entire worker lifetime — segments flow in, tokens flow out. Chunking rule (zero data loss) — IDENTICAL to pretraining pipeline: - Segments <= 6s: single chunk, pad to hop boundary. - Segments > 6s: overlapping 6s windows (stride 5.8s, overlap 0.2s). If the would-be last chunk is shorter than MIN_TAIL, it is absorbed into the previous chunk (extending it to 6-8.8s). The model handles variable lengths natively (Conv1d + relative position embeddings). - After encoding, tokens are stitched with a deterministic center-cut rule: drop HALF_OVERLAP tokens at each internal boundary. Ref: XCodec2 training: min_audio_length=96000 (6s random crops). Inference: full-length audio, pad to multiple of 320. CodecEncoder hop = prod([2,2,4,4,5]) = 320, so 50 tokens/sec. ) annotationsN) dataclassfield)Path) TokenBatch) CodecConfig)Segmenti@2g?g@iwc@sReZdZUdZded<ded<ded<ded<ded <d Zded <d Zded <dS)EncodedSegmentaResult of encoding one audio segment through XCodec2. For multi-chunk segments (>6s), tokens are stitched from overlapping windows with center-cut boundary trimming. The stored tokens are one contiguous sequence covering the full segment. int segment_idxfloatstart_send_s duration_s torch.Tensorxcodec2_tokensencode_time_ms num_chunksN)__name__ __module__ __qualname____doc____annotations__rrrr8/home/ubuntu/bench-codecs/codecbench/pipeline/encoder.pyr 0s  r c@seZdZdZd6d7ddZd8d d Zd9ddZed:ddZed;ddZ e dd(d)Z d?d,d-Z .d@dAd2d3Z .d@dBd4d5Zd.S)C HotEncoderzKeeps XCodec2 loaded and warm on GPU. Call load() once at startup, then encode_segments() for each batch. Model is never unloaded between processing units. cudacfgrdevicestrcCs(||_||_d|_d|_d|_d|_dS)NFrr)_cfg_device_xcodec_loaded_total_encode_time_total_segments)selfr!r"rrr__init__Hs  zHotEncoder.__init__returnNonecCsddlm}td||jjd|_|jjr||jjn|jj |j dd|j_ |jj |jj dtjd}td |d|_d S) zLoad XCodec2 and warm it up.r)FastXCodec2CodeczLoading XCodec2...)model_idr"T) batch_sizeg.Az&XCodec2 loaded and warm. VRAM: %.0f MBN)codecbench.codecs.xcodec2_fastr.loggerinfor$xcodec2_model_idr&xcodec2_custom_ckpt_load_custom_xcodec_ckptloadr% _use_compilewarmupxcodec_batch_sizetorchr memory_allocatedr')r*r.vramrrrr8Ps    zHotEncoder.load ckpt_pathcCsdddlm}ddlm}m}m}td|||j j }t j |ddd}| d| d |}i} |D]\} } | d rE| d d d n| } | | | <q5|j| dd \} } | rdtdt| | dd| rstdt| | dd||j||j||j|jd|j_|jj|jdt jjj_dt jj_||j||j_ |j|j_tddS)z6Load a custom XCodec2 checkpoint (fine-tuned weights).r) XCodec2Model)GPUMelExtractor_apply_layer_truncation_apply_sdpa_patchz%Loading custom XCodec2 checkpoint: %scpuF) map_location weights_only state_dictmodelzmodel.r)strictzCustom ckpt missing %d keys: %sNz&Custom ckpt has %d unexpected keys: %sr0Tz7Custom XCodec2 checkpoint loaded with all optimizations)!xcodec2.modeling_xcodec2r@r2rArBrCr3r4from_pretrainedr$r5r<r8getitems startswithreplaceload_state_dictwarninglenevaltor%semantic_modelfeature_extractorr&_mel_extractorbackendsr matmul allow_tf32cudnn_model)r*r?r@rArBrCrHckptrGcleanedkvmissing unexpectedrrrr7as2        z#HotEncoder._load_custom_xcodec_ckptboolcCs|jSN)r'r*rrrloadedszHotEncoder.loadedrcCs|jdkrdS|j|jS)Nrr)r)r(rgrrr avg_encode_mss  zHotEncoder.avg_encode_mswavrcCs&|jd}t|t}t|d|fS)aPad waveform to the next multiple of CODEC_HOP (320). Matches the original repo's padding convention for variable-length inference. For standard 6s chunks (96000), 96000 % 320 == 0 so this adds the unconditional +320 that encode_code uses. r)shape CODEC_HOPFpad)rjTrorrr _pad_to_hops  zHotEncoder._pad_to_hopsegments list[Segment]?tuple[list[Segment], list[int], list[tuple[int, int, Segment]]]c st|jj|jjtt|jj}|}tt|jj}|jj}g}g}g}|D]} | jjd} t|} | krX| t | j | j | | jd| | | | d| fq*g} d} | | krq| | | |7} | | ksb| ry| dnd}| |}|dkr||kr| | |n t| D]@\}}|t| dk}|r| }n|}||}| jd||f}| |}| t | j ||| j |||d| |q| | t| | fq*tfdd|D}t|t|krtdt|t|t||||||fS) aWSplit segments into overlapping chunks. Never discards audio. Rule: 1. T <= chunk_samples (6s): single chunk, pad to hop boundary. 2. T > chunk_samples: stride-based 6s windows. If the would-be last chunk has < MIN_TAIL_S of audio, drop it and extend the previous chunk to the segment end (making it 6-8.8s). Returns: flat_chunks: padded chunks ready for the GPU valid_samples: actual audio samples per chunk (before padding) groups: (start_idx, count, original_segment) per parent rk)rraudiorr.c3s|] }|krdVqdS)rNr).0vs chunk_samplesrr z6HotEncoder._split_to_overlap_chunks..zBOverlap split: %d segments -> %d chunks (%d standard, %d extended))r r$ chunk_seconds target_sr OVERLAP_S MIN_TAIL_SrurlrTappendrrrrq enumeratesumr3debug)r*rroverlap_samplesstridemin_tailsr flat_chunks valid_samplesgroupssegrp group_startstartsposlast_end remaindercisis_lastendactual chunk_wavpadded n_extendedrrxr_split_to_overlap_chunkssl             z#HotEncoder._split_to_overlap_chunksencodedlist[EncodedSegment]valids list[int] original_segrseg_idxr r c CsZt}tdd|D}t|dkr2|d}|dt}t||j|j|j|jddd|f|ddSg} t|} t t ||D]Z\} \}} | t}|jddd|f} | dkrk| | dddt ||dfq?| | dkr| | ddt ||dfq?| | ddt ||t |||dfq?t||j|j|jtj| dd|| dS)aCenter-cut stitch overlapping chunks into one contiguous segment. Rule (deterministic, identical to pretraining pipeline): First chunk: keep tokens[ 0 : valid_tok - H ] Middle chunk: keep tokens[ H : valid_tok - H ] Last chunk: keep tokens[ H : valid_tok ] Single chunk: keep tokens[ 0 : valid_tok ] Where H = HALF_OVERLAP = 5 tokens (0.1s). css|]}|jVqdSrf)rrverrrrz sz+HotEncoder._stitch_group..rrN)r rrrrrrdim) HALF_OVERLAPrrTrmr rrrrrziprmaxminr<cat)r*rrrrHtotal_msecvtx_partsnirwxtokrrr _stitch_groupsB   & "0 zHotEncoder._stitch_grouprEncodedSegment | Nonec CsZt}|j|j}|jdkr|d}z|j||j j }t j Wn0t j jtfyU}z td|j|jt|ddt j tWYd}~dSd}~ww|duretd|j|jdSt|d}|j|7_|jd7_ztd|j|j|j|jdd|d WSttt fy}z td |WYd}~dSd}~ww) z3Encode a single chunk (any length) through XCodec2.r rz.CUDA error in _encode_one (seg %.1f-%.1fs): %sNxz,XCodec2 returned None for segment %.1f-%.1fsrr rrrrrz.Failed to extract tokens from codec output: %s)!time perf_counterrurVr%ndim unsqueezer&encoder$r}r<r synchronizeOutOfMemoryError RuntimeErrorr3rSrrr# empty_cachegccollectr(r)r rtokensrDAttributeError IndexErrorKeyError)r*rt0rjx_tbr elapsed_msrrr _encode_one4s@      zHotEncoder._encode_oneNchunksxcodec_batch_size_override int | Nonec Csg}t|p|jjd}tdt||D]}||||}t}g}|D]} | j|j } | j dkr8| d} | | q%t j|dd} |j| |jj} t jt|d} |j| 7_|jt|7_| durtd|tt|D]}| dq{qzD| jjdt|krtd| jjdd t|t|D]#\}} | t||| j| j| j| j||d | t|d qWqt!tt"fy}ztd |tt|D]}| dqWYd}~qd}~ww|S) zFEncode a batch of same-length chunks (standard 6s path). XCodec2 only.rrr rrNz+Batch encode returned None (batch_start=%d)zXCodec batch output too small: z < rz.Failed extracting tokens from batch result: %s)#rr$r;rangerTrrrurVr%rrrr<rr&rr}r rr(r)r3rSrrlrrr rrrrDrr)r*rrresultsxbsrbatchrx_wavsrrjx_inputrrjrrrr _encode_batchWsX           zHotEncoder._encode_batchc s0|sgS|\}}g}g}t|D]"\}}|jjdttjdkr1||q||qdgt } |rfdd|D} z j | |d} WnBtj j t fy} z2dt| vsndt| vrtdt | tj tfd d| D} nWYd} ~ nd} ~ wwt|| D]\}} | | |<q|D] }|| |<qtd d | D}|rtd |t | |t | krgSg}t|D]B\}\}}}| |||}||||}td d |Drtd||j|jtdd |D|qӈ||||}||q|S)aEncode audio segments with XCodec2. Zero data loss. Flow (identical to pretraining pipeline): 1. Split >6s segments into overlapping chunks (stride 5.8s). Short remainders are absorbed into the previous chunk. 2. Encode standard chunks batched; extended chunks B=1. 3. Trim tokens to actual audio length (no padding tokens stored). 4. Stitch overlapping chunks per parent (center-cut rule). Returns one EncodedSegment per original segment. rkNcsg|]}|qSrr)rvr)rrr sz.HotEncoder.encode_segments..)rz out of memoryzillegal memoryz;CUDA error on batch encode (chunks=%d), falling back to B=1csg|]}|qSr)r)rvcrgrrrscs|] }|durdVqdSNrrrrrrrzr{z-HotEncoder.encode_segments..z%d/%d chunks failed to encodecss|]}|duVqdSrfrrrrrrzsz5Skipping segment %d (%.1f-%.1fs): %d/%d chunks failedcsrrrrrrrrzr{)rrrurlrqr<zerosSTANDARD_CHUNKrrTrr rrr#lowerr3rSrrrrrranyrrr)r*rrrrr std_indices ext_indicesrrw encoded_flat std_chunks std_resultsrenc failed_chunksrrstartcountorig_seg group_encgroup_vsstitchedr)rr*rencode_segmentssn          zHotEncoder.encode_segments)r )r!rr"r#)r,r-)r?r#r,r-)r,re)r,r)rjrr,r)rrrsr,rt) rrrrrrrr r,r )rrr,rrf)rrsrrr,r)rrrsrrr,r)rrrrr+r8r7propertyrhri staticmethodrqrrrrrrrrrrAs$   !    _ ;$ :r)#r __future__rrloggingr dataclassesrrpathlibrr<torch.nn.functionalnn functionalrncodecbench.codecs.basercodecbench.pipeline.configrcodecbench.pipeline.vadr getLoggerrr3rmTOKENS_PER_SECr~r OVERLAP_TOKENSrrrr rrrrrs.