o q^iz @sdZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z mZddlZddlZedZde efddZd ede ee ffd d ZGd d d eZGdddeZdLdedede ee ffddZdMdededede eee ffddZdedefddZdd Zdedefd!d"Zda dNd$d%Z!de efd&d'Z"de efd(d)Z#dede ee ee fffd*d+Z$d,edefd-d.Z%d/edefd0d1Z& 2 dOd3eded4ed5e'd6e ee fde ee ee fff d7d8Z(dPd,ed4eded5e'de ef d9d:Z) ; 2 dQdedZ*  dRdeded4ed?ed@edAe'dBedCedDed5e'dEe+dFedGede ee ffdHdIZ,deded4ed?edBedCedDed5e'dEe+de ee ffdJdKZ-dS)SuAudio download and preprocessing with robust error handling. === v7.0 OPTIMIZATION: Download-Time Resample === Uses ffmpeg directly to create both 16kHz processing audio AND original quality audio in a single pass, eliminating Python resampling overhead. Previous: yt-dlp → temp.wav → torchaudio load → Python resample → save both Now: yt-dlp → ffmpeg pipe → dual output (16kHz + original) in one pass Saves ~10-15s per video by avoiding torchaudio resampling. N)Path)TupleDictAnyOptionalListIterablezFastPipelineV6.DownloadreturncCs tjdS)a  Get proxy URL for yt-dlp from environment variable. Set YTDLP_PROXY environment variable with your proxy URL, e.g.: export YTDLP_PROXY="http://user:pass@proxy.example.com:7777" Returns: Proxy URL string or None if not configured. YTDLP_PROXY)osenvirongetrrN/home/ubuntu/.cursor/worktrees/maya3__SSH__216.81.248.184_/nmo/src/download.pyget_ytdlp_proxys r file_pathc Csz;ttdr%ztt|}|j|j|j|jdWWSty$Ynwtt|\}}|jd}||||dWStyQ}z t d|d|d}~ww)aL Get audio file info (sample_rate, num_frames, duration). Compatible with all torchaudio versions by using torchaudio.load and inferring metadata from the returned tensor. Args: file_path: Path to audio file Returns: Dict with 'sample_rate', 'num_frames', 'duration' keys info) sample_rate num_framesdurationzCould not get audio info for : N) hasattr torchaudiorstrrr Exceptionloadshape RuntimeError)rrwaveformrrerrrget_audio_info)s*    r!c@eZdZdZdS)VideoValidationErrorz*Raised when video fails validation checks.N__name__ __module__ __qualname____doc__rrrrr#Qr#c@r") DownloadErrorz)Raised when download fails after retries.Nr$rrrrr*Vr)r*$@video_idtimeoutc Csd|}dddd|d}t}|r||d<zJt|:}|j|dd}|rM|dd||d d |d p9g|d d |ddWdWSWdWiS1sYwYWiSty~}ztd|d|WYd}~iSd}~ww)a Fetch YouTube metadata (chapters, title, duration) without downloading. === v3: Proxy support for rate limit avoidance === Set YTDLP_PROXY env var for high-volume metadata fetching. Args: video_id: YouTube video ID (11 chars) timeout: Max seconds to wait for metadata Returns: Dict with chapters, title, duration, etc. Empty dict on failure. https://www.youtube.com/watch?v=TF)quiet no_warnings extract_flat skip_downloadsocket_timeoutproxydownloadtitlezVideo rrchaptersuploaderUnknown upload_date)r7rr8r9r;Nz%Could not fetch YouTube metadata for r)ryt_dlp YoutubeDL extract_infor rloggerdebug)r,r- youtube_urlydl_optsr4ydlrr rrrfetch_youtube_metadata[s>          rD>@ @ video_url min_duration max_durationc Cstd|ddddd}t}|r||d<zt|}|j|dd}|dur7ddifWdWS|d d krP|d sPdd |fWdWS|d d}||krndd|d|d|fWdWS||krdd|d|d|fWdWS|ddkrdd|fWdWS|dddkrtd|dd|drdd|fWdWS|dd}td|d|ddd |fWdWS1swYWdStjj y:} zBt | } d!| vrdd"ifWYd} ~ Sd#| vrddifWYd} ~ Sd$| vr(dd%ifWYd} ~ Sdd&| ifWYd} ~ Sd} ~ wt yV} zdd't | ifWYd} ~ Sd} ~ ww)(a Validate YouTube video before processing. Checks: - Video exists and is accessible - Video duration is within acceptable range - Video has audio track - Video is not age-restricted or private Args: video_url: YouTube URL to validate min_duration: Minimum video duration in seconds (default: 30s) max_duration: Maximum video duration in seconds (default: 4 hours) Returns: (is_valid, message, info_dict) u🔍 Validating video: TF)r/r0r1r2r4r5Nz#Video info extraction returned NoneacodecnoneformatszVideo has no audio trackrrzVideo too short: zs (min: s)zVideo too long: zs (max: availabilityprivatezVideo is private age_limitu*⚠️ Video is age-restricted (age_limit=)is_livezLive streams are not supportedr7r:u✅ Video validated: '' (OKzVideo unavailablezVideo unavailable or deletedz Private videozSign inz2Video requires sign-in (age-restricted or private)zDownload error: zValidation failed: ) r?rrr<r=r>r warningutilsr*rr) rGrHrIrBr4rCrrr7r error_msgrrrvalidate_videosf    ("    rXtotal_durationcCsd}d}d}||kr |S|S)u Calculate dynamic intro skip based on video duration. Strategy (when no chapter info available): - Video ≤ 30 min (1800s): Skip 180s (3 min) - ~10% intro ratio - Video > 30 min: Skip 300s (5 min) - smaller % for longer content This is ONLY used when no intro chapter is detected. Chapter-based detection always takes priority. Args: total_duration: Total video duration in seconds Returns: Intro skip duration in seconds gf@gr@r)rYTHRESHOLD_30MIN SKIP_SHORT SKIP_LONGrrrget_dynamic_intro_skips r^c sgd}gd}d}d}d}d}|rt|dkr|d} | dd} | dd} | }d }| }td | d | d d t|dkr|d} | dd} | }| dd}d}|D] }||vrc|}nqY|r|}d|}| d| }td|d| d|d d t|dkr|d}|dd|dd}|d|}tfdd|Dr||dkr||}td|dd|d d n+t|ddr|jdkrt|}|dkrdnd}d|}d}td |d!d"|d#||||fS)$u Detect intro/outro sections from YouTube chapters. === v3: SIMPLIFIED INTRO DETECTION === === LOGIC === 1. If chapters available: - Always skip 1st chapter (assumed to be intro) - If 2nd chapter name matches intro keywords, skip it too 2. If no chapters: - Video ≤30 min: skip 3 min (180s) - Video >30 min: skip 5 min (300s) Returns: (intro_seconds, outro_seconds, skip_reason, matched_title): Trim amounts + metadata )Uintro introductionopening advertisementsponsoradpromo sponsored sponsorshipzbrought to you promotionalपरिचयuभूमिकाuशुरुआतप्रस्तावनाuविज्ञापनप्रायोजकparichaybhumikashuruaat prastavanavigyapanprayojakuపరిచయంuప్రారంభంuప్రకటనuస్పాన్సర్ parichayam prarambham prakatanauಪರಿಚಯu ಆರಂಭuಜಾಹೀರಾತುuಪ್ರಾಯೋಜಕ parichayaarambhajahiratuuஅறிமுகம்uதொடக்கம்uவிளம்பரம்uஸ்பான்சர்arimukam thodakkam vilambaramuപരിചയംuആമുഖംuതുടക്കംuപരസ്യംuസ്പോൺസർrraamukham thudakkamparasyamuપરિચયu આરંભuજાહેરાતuસ્પોન્સરrlaarambhjaheratuপরিচয়uভূমিকাu শুরুuবিজ্ঞাপনuস্পন্সরporichoyrmshurubiggaponrirjuसुरुवातuजाहिरातrkrlrosuruvatjahiratuਜਾਣ-ਪਛਾਣuਸ਼ੁਰੂਆਤuਇਸ਼ਤਿਹਾਰuਸਪਾਂਸਰz jaan-pehchaanrnishtihar)outroendingcreditsrcrdreendcard conclusionclosing subscribezlike and subscribeuसमापनu अंतuसब्सक्राइबముగింపుu!సబ్స్క్రైబ్ruமுடிவுuഅവസാനംuસમાપનu শেষNrr7end_time first_chapteru 📖 Skipping 1st chapter: 'z' (0s - .1frMrzfirst_two_chapters:z + u, 📖 Also skipping 2nd chapter (keyword 'z'): 'rS start_timec3s|]}|vVqdS)Nr).0keyword chapter_titlerr `sz3detect_intro_outro_from_chapters..,u 📖 Chapter-based outro: 'auto_intro_skipTrZu≤30minz>30minzdynamic:u, ⏱️ Dynamic intro skip (no chapters): .0fz s (video rQ) lenr r?rloweranygetattrintro_skip_secondsr^)r8rYconfigintro_keywordsoutro_keywords intro_skip outro_skip skip_reason matched_titlerfirst_chapter_titlefirst_chapter_endsecond_chaptersecond_chapter_titlesecond_chapter_title_lowersecond_chapter_endmatched_keywordr last_chapter chapter_start chapter_endduration_thresholdrrr detect_intro_outro_from_chapterss^!               rcCst|dkrtdd|Dr|Sd|vr!|ddddSd|vr1|ddd dSd |vrA|d dd dSd |vsId |vri|d d}|d d}d|vre|ddd}|S|}|St|ddS)z=Extract video ID from various URL formats (YouTube, R2, etc). css |] }|p |dvVqdS)z-_N)isalnum)rcrrrrusz#extract_video_id..zwatch?v=r&rz youtu.be/?z/shorts/r2.cloudflarestorage.comzs3.amazonaws.com/r.N)rallsplitrsplithashlibmd5encode hexdigest)rGpathfilenamer,rrrextract_video_idqs rR2ClientcCsZtdur+ddlm}m}m}|}tjdd}||j|j |j |j |d}||ddatS) zNLazy-init the R2 client used for reading source videos (bucket_type='source').Nr)rget_r2_config_from_envR2ConfigR2_SOURCE_PREFIXr) endpoint_url access_key_idsecret_access_keybucketprefixsource)r bucket_type) _R2_SOURCE_CLIENT src.r2_clientrrrr r r rrrr)rrrrbase source_prefixcfgrrr_get_r2_source_clients rcCsg}tjdd}|r|dd|dDt|ddp!d}|r)|||gdg}t}|D]$}|}|sBd}n | drI|n|d}||vr[| |||q7|S) z` Build a small set of candidate prefixes for source objects without listing the bucket. R2_SOURCE_PREFIXESrcSg|] }|r|qSrstrip)rprrr z,_iter_r2_source_prefixes..,r2_source_prefix)rpodcasts podcasts/videosvideos/r) r r r rextendrrappendsetendswithadd)rprefixes env_prefixes cfg_prefix normalizedseenrp_normrrr_iter_r2_source_prefixess(   rcCsg}tjdd}|r|dd|dDt|dd}|rFt|tr5|dd|dDnt|t t frF|d d|D|gd g}t }|D]$}|}|s_d}n | d rf|nd |}||vrx| |||qT|S) zOReturn candidate extensions to try for source objects (small list, no listing).R2_SOURCE_EXTENSIONSrcSrrrrr rrrrrz._iter_r2_source_extensions..rr2_source_extensionsNcSrrrrrrrrrcSs$g|]}t|rt|qSr)rrrrrrrs$)r.webm.mp4.mkv.m4a.mp3.opusr)r r r rrrr isinstancerlisttupler startswithrr)rextsenv_extscfg_extsrrr e_normrrr_iter_r2_source_extensionss.    rc Cst|}dttdttdttfdd}|t|gd}|t|gd}d}|D] }|D]}|||} | }|| } | rJ| | fSq/q+td |jd |pU|) a Resolve a video_id to an object key in the R2 source bucket. We avoid bucket listing for performance/cost reasons. Instead we try a small set of likely prefixes/extensions via HEAD requests only. Returns: (remote_key, head_object_response) items preferredr cSsbg}t}|D]}||vr||vr||||q|D]}||vr.||||q|S)z?Return items with preferred values moved to the front (stable).)rrr)rroutrritrrr _prioritizes    z._resolve_r2_source_object.._prioritize)rrr)rrrrrrrNzR2 source not found: s3://r)rrrrr head_objectr*r) r,rclientrrrlast_keyrextkeyheadrrr_resolve_r2_source_objects. " rurlcCsd|vp d|vo d|vS)z'Check if URL is an R2/S3 presigned URL.rzs3.z amazonaws.comr)rrrr is_r2_urlsr presigned_urlcCs6zddlm}||}|jd}|WS|YS)z Extract clean R2 bucket/key path from a presigned URL. Example: Input: https://xxx.r2.cloudflarestorage.com/test/podcasts/VIDEO.webm?X-Amz-... Output: test/podcasts/VIDEO.webm r)urlparser) urllib.parserrlstrip)rrparsedrrrrextract_r2_path#s  rr2_url output_dir max_retriessupabase_metadatac9Cs td|t}||d}||d}||d} |r|rtd|zpg} |d?d@}!|dAr|dAddBn|}"| rf|!rftdCt| dDg}#t| D]:\}$}%|%dE|%dFd}&|$dt| kr>| |$ddE| |$ddF|"}'n|"}'|#|%d=dG|&|'dHqt|#|"|\}}}}tdI|d:dJ|nJtdK|dkrt|}dL}tdM|dNd;n/tdO|dkrt|dPd rt|dQd d krt t||dR}dS}tdM|dNd;t|dQd dkr||j!krtdT|j!d:dU|d:dV|j!}dW}|}(t"|||(dX})|)|(}*|dBks|*dYkrd}(|dkr|ndZ}*d}d[}|j#}+dddt| d\t|(d]t|*d^d_d`t|+ddt|d\t|(d]t|*d^d_dddad dbdct|g},tdd|de|dNdVz t j |,d d d df}Wnt j$y^}z tdg|jddd}~ww|shtdh|sqtdizHt%t|}-t%t|}.|-d3}/|.d3}0t&|/|0}1|1dRkrt'dj|/dkdl|0dkd;tdm|1dkdn|/}2tdo|/dkdp|1dqdVWn%tyty}zt dr||*}2WYd}~nd}~wwz| (WnYd}3i}4zRddsl)m*}5tt|\} } |5| +d,| }3tdt|3j-du|3j.dNdv| |3j/t0|3j1dt0|3j.d|3j2|3j-t0|3j3d%|3j4|3j5t0|3j6dwt0|3j7dwdx }4~ Wnty[}zt dy|WYd}~nd}~wwt|}6tdz|6d:d{|2dNd||r{|d>pzgng}7t8|}8t|id}|d~d|d=|d|d|d|d|d|2dt|d>|7d5|+d|dt|d|8dd dd dd |4||4dd@ dfS)a Download audio directly from R2 presigned URL using ffmpeg. === v2: R2 DIRECT DOWNLOAD === For pre-cached videos in R2, we bypass yt-dlp entirely and use ffmpeg to download and convert in a single pass. Creates: - {video_id}_trimmed.wav (16kHz mono for processing) - {video_id}_original.flac (original quality mono for export) Returns: (audio_path, metadata): Path to processed audio and basic metadata u⚡ R2 direct download: _trimmed.wav_original.flac_raw.wav✅ Using cached audio: r.z R2 Source: rrTflacr2)r,rAr7original_duration intro_skippedprocessed_durationrr8roriginal_sample_rateoriginal_audio_pathoriginal_audio_preserved audio_format source_typezCache validation failed: , re-downloadingNz R2 download attempt rffmpeg-y-iz-vn-ac1z-fwavr)capture_outputtextr-rzffmpeg failed: ii'z$Downloaded file too small or missingzDownload timed outz R2 download timeout, attempt failed: zR2 download failed after attempts: z)Could not get audio info via torchaudio: z, trying ffprobeffprobez-verrorz -show_entriesz"format=duration:stream=sample_ratez-ofjson)r1r2formatrstreamsr逻zffprobe also failed:  📊 Raw audio: Hz, rsrKr7r8 has_chaptersFsource_duration_min<u! 📖 Using Supabase chapters (z chapters)secondsrr)r7rru" 📖 Chapter-based skip: intro=z s, reason=u3 📊 No chapters in Supabase, using dynamic skipzdynamic:no_supabase_chaptersu ⏱️ Dynamic skip: ru2 ⚠️ No Supabase metadata, using dynamic skiprr皙?zdynamic:no_metadata#⚙️ Manual intro skip override: s (was: rMmanual_override i?Bznone:too_short_or_unknown-ss-t-map0:a-ar-c:a-compression_level5' ⚡ FFmpeg dual output: 16kHz WAV + zHz FLAC (skip: r1r2checkFFmpeg dual output failed: z"FFmpeg did not create trimmed filez#FFmpeg did not create original file" ❌ DURATION MISMATCH! trimmed=.2f s, original=#Duration mismatch between outputs: s differenceu ✅ Duration verified: s (diff=.3f& ⚠️ Could not verify durations: analyze_spectral_quality 🔬 Spectral: z (rolloff=Hz) claimed_sample_rateeffective_sample_ratedetected_nyquistrolloff_frequency is_upsampledoriginal_format confidenceis_suitable_for_ttsquality_issueshigh_freq_energy_ratiospectral_flatness Spectral analysis skipped: u ✅ R2 download complete: zs, zs audior,rAr"r#intro_skip_reasonintro_skip_matched_titler$rr%r& source_urlr'r(r)rg)spectral_qualityaudio_native_sample_rateis_native_quality)9r?rtimeexistsrrrrrrUrange subprocessrun returncoder*stderrstatst_sizeTimeoutExpiredsleepr8loadsstdoutfloatr intr enumeraterrr^rminrmaxrCalledProcessErrorr!absr7unlinksrc.spectral_analysisr^squeezenumpyoriginal_format_guessrfrdroundrergrirjrkrlrmr)9rr,rrrrstart trimmed_file original_fileraw_file proc_waveformproc_sr orig_waveformorig_srr last_errorattemptffmpeg_downloadresultr original_sractual_duration probe_cmd probe_resultr8 probe_datae2rrrr video_titler8r?source_durationnormalized_chaptersich start_secend_secrr trim_duration target_sr ffmpeg_dual trimmed_info original_infotrimmed_durationoriginal_duration_check duration_diffr$rr spectral_infor^elapsed sb_chaptersr2_pathrrr_download_from_r25s          "    &      *                         rc Cs|td|t}||d}|r(|jdkr(td||Sdt||ddddd d d }t}|rB||d <t|D]l}z>t |} | j |dd Wd n1s`wY|rtdt|dd|jddd|WSWqFt y} z t d|dd| ||dkrtd|WYd } ~ qFd } ~ wwtd|dd S)z,Download video for visualization (720p MP4).u*🎥 Downloading video for visualization: ri@Bu✅ Using cached video: z2bestvideo[height<=720]+bestaudio/best[height<=720]z.%(ext)smp4Tr)r9outtmplmerge_output_formatr/r0retriesfragment_retriesr4r5Nu✅ Video download: rz s | Size: g.AMBzVideo download attempt rr3r4u ❌ Video download failed after z attempts)r?rrurvr|r}rrrwr<r=r>rrUrr7) rrr,rr output_filerBr4rrCr rrr download_video_for_visualizationsD   .  rTvalidatec(Csptd|dddt}t|}t|j|}|jdddt|dd}|d krt|st |} t ||\} } t t|d d } | j | | d } td | j d| d| dddddt| |||||d\}}t|}||d<| j |d<| |d<| d|d<td|dd|dddd||fSt|rtdt||||||d\}}t|}||d<td |dd|dddd||fSi}|rt|\}}}|std!||s5td"ddd#d$}zt|}|j|d#d%}Wdn 1swYWn ty4}ztd&|d'gd(}WYd}~nd}~ww||d)}||d*}t|d+d#}||d,}|r;d}|r^|n|re|nd}|rt|std#}td-|r;td.|ztt|\}}|jd/|d0krtd1||r|r|n]|}|r|rz tt|} | d2}WnYt||d3||d4d5|jd/|d6|jd/|t||d7g|||rt|nd|du|rt|d8rd9nd:d; fWSWn3ty:}z&td<|d=z||r*|r*|WnYWYd}~nd}~ww|d>d}!|rJ|d7gng}"t|"|!|\}#}$}%}&|j dkrt|#|j krttd?|j dd@|#ddA|j }#|rt!||||||j"|#|$|!|||%|&dB }'nt#||||||#|$|!||dC }'t|}tdD|dd|'ddd||'d<t||'fS)Eu; Download audio from YouTube URL or R2 presigned URL and prepare for processing. === v7.0 OPTIMIZATION: Download-Time Resample === Uses ffmpeg directly for dual-output in single pass: - 16kHz mono WAV for processing pipeline - Original quality mono WAV for high-quality export Previous: yt-dlp → temp.wav → torchaudio load → Python resample → save both Now: yt-dlp → single ffmpeg → dual output (NO Python resampling!) === v2: R2 URL SUPPORT === When video_url is an R2 presigned URL, we use ffmpeg directly instead of yt-dlp. === v3: Supabase Metadata === When supabase_metadata is provided (from distributed queue), use it for: - Title, chapters, duration (no yt-dlp metadata fetch needed) - Chapter-based intro/outro skip Args: video_url: YouTube URL or R2 presigned URL config: Pipeline configuration validate: Whether to validate video before download (default: True) max_retries: Number of download retries (default: 3) supabase_metadata: Pre-fetched metadata from Supabase (chapters, title, etc.) Returns: (audio_path, metadata): Path to processed audio and video metadata Raises: VideoValidationError: If video fails validation DownloadError: If download fails after retries u📥 Downloading: NPz...T)parentsexist_ok input_sourceyoutuber!r2_presign_ttl_seci)expires_in_secu📦 R2 source resolved: s3://r ( ContentLengthrirz MB))rr,rrrrdownload_wall_time_secr2_source_bucket r2_source_keyr2_source_size_bytesu✅ Download (R2): zs | Duration: r$rr>u5📦 R2 source detected, using ffmpeg direct downloadu✅ Download (R2 URL): zVideo validation failed: u📋 Fetching video metadata...F)r/r0r1r5zCould not fetch metadata: r:)r7r8rrpreserve_original_audio _original.wavuF⚠️ Cached trimmed file exists but original missing, re-downloadingrrrGu?⚠️ Cached file seems corrupted or too short, re-downloadingrr.r7Cachedrr8z.flacr r0) r,rAr7r"r#r$rr8rr%r&r'r(u#⚠️ Failed to load cached file: r*rrDrErM) rGr,rrrrrrr"rrrr) rGr,rrrrrr"rru✅ Download: )$r?rrurrrmkdirrrrrrgenerate_presigned_get_urlrr rrXr#r<r=r>rrUrvrrrrrr!rrr!_download_with_ffmpeg_dual_outputr_download_standard)(rGrrrrrr,rrr  remote_keyr ttlr audio_pathmetadatarris_validmessage ydl_info_optsrCr rrpreserve_originallegacy_original_file cache_validactual_originalrsrr orig_infor"r8rrrrrrrrdownload_audios<(   $  $                  rrrrrrr"rrrc +Cstd||d} dt||ddddddd gd d gd d d }t}|r.||d<d}t| D]}zltd|dd| t|}|j|dd}Wdn1s[wY| s| |dD]+}|j dvr|j dkrt j ddt|d d t| dgddd| n|| nql| rWn2tdty}z"|}td|dd||| dkrtd|WYd}~q4d}~ww| std| d|ztt| }|d }|d!}Wn ty}ztd"|d#d$}|}WYd}~nd}~wwtd%|d&|d'd(d}zBd)d*lm}d)dl}|t| \}}||d)|}td+|jd,|jd-|jd.d/|jr`td0d1|jWnty|}ztd2|WYd}~nd}~ww|d)kr|nd)}|d)kr||n|}||} ddt| d3t|d4t| d5d6d7t|d d dt|d3t|d4t| d5d6d d d8d9d:d;dt|g}!td<|d=|d'd>|d'd?z t j |!dddd@}"Wnt j y}zt!dA|j"tdB|d}~ww|s tdC||stdD|zUtt|}#tt|}$|#d!}%|$d!}&t#|%|&}'|'dEkrWt!dF|%dGdH|&dGdI|'dGd?t!dJtdK|'dGdL|%}(tdM|%dGdH|&dGdI|'dNd?Wn%tyuty}ztdO|| }(WYd}~nd}~wwz| WnY| |dD]})z|) WqYqtdP|j$dQ|dR|j$dQ|dS i}*|dur||jt%|j&d)t%|jd)|j'|jt%|j(d|j)|jt%|j*dTt%|j+dTdU }*idV|dWdX|dY| r | ,dYdZndZd[|d\|d]|(d^t|d_| r%| ,d_gngd |d`|dat|db|dcdddd9dedfdgdhdi|*||*,djdk | | dlS)maq === v7.2 OPTIMIZATION: Download-Time Dual Output with FLAC === Single ffmpeg command creates both outputs: 1. 16kHz mono WAV (for processing) - with intro/outro trim 2. Original quality mono FLAC (for export) - LOSSLESS, ~50% smaller than WAV FLAC Benefits: - 100% lossless (bit-perfect audio quality) - ~50% smaller than WAV (saves storage costs) - Fast decode (CPU-efficient for training) - Compression level 5 (balanced speed vs size) Key insight: ffmpeg can output to multiple files from single input stream, encoding each to different formats in one pass. u9⚡ Using optimized dual-output download with FLAC (v7.2)rbestaudio/bestz _raw.%(ext)sTFFmpegExtractAudior00r preferredcodecpreferredqualityr.r/rr9rr/r0postprocessorspostprocessor_argsrrr4N Download attempt rrr5z_raw.*.wavrrrrrr+r-r,r1rSzDownloaded file not foundr3r4Download failed after r5rrzCould not get audio info: z, using metadata durationr;r<r=rr>rr]r_z (effective=z Hz, rolloff=rr`u ⚠️ Quality: z, rnrIrJrKrLrMrNr rOrPrQzHz FLAC (trim: zs-rMrRzFFmpeg failed: rTz$FFmpeg did not create trimmed file: z%FFmpeg did not create original file: rCrUrVrWrZz> This indicates FFmpeg trim was not applied to both outputs!rXrYu" ✅ Duration verified: trimmed=r[r\u ✅ Dual output complete: rz Hz WAV), zHz FLAC)rarbr,rAr.r7r:r"r#r$rr8r%r&rqr'r(audio_compressionlosslessr)rrrrgF)rsrtrorp)-r?rrrrwr<r=r>rvglobsuffixrxryrrenamer*rrUrurr!rr^rrrrrrdrfrkjoinrr7r{rnamerrergrirjrlrmr )+rGr,rrrrrrr"rrrrrrBr4rrrCdl_infoext_filer audio_inforrrrr^r raw_waveform_rrr ffmpeg_cmdrrrrrrr$temprrrrrs               "      $ ( (             rc * Cs||d} ||d} ||d} t|dd} t|dd}| rDdt| d d d d d d g|dkr7ddgndt|ddgddd}ndt| d d d d d d gdt|jddgddd}d}t|D]}zotd|dd|t|}|j|d dWdn1swY| s| |dD]0}|j dvr|j dkrt j ddt|dt|jddt| dg d d d|n|| nq| rWn1Wqcty}z"|}td|dd|||dkrtd |WYd}~qcd}~ww| std!|d"|z tt| \}}Wnty;}z| r0| td#|d}~ww|}|jd|}|}d}| r||jkrtd$|d%| jt| }ddlm}|j||jd&}||}|j}td'|d(|d)krtd*|d+d,| r| d-gng}t|||\}}}}|jdkr||jkrtd.|jd+d/|d+d0|j}|}t||}|dkrt||nd} |dkr|jd| n|jd}!|dks| dkr|dd||!f}|jddkr |j dd d1}|d  !"}"|"d2kr$td3|"d4d5t#t|||d}#| r| rzktt| \}$}%|$jddkrP|$j dd d1}$t||%}&|dkrat||%nd}'|dkro|$jd|'n|$jd}(|$dd|&|(f}$t#t| |$|%t| }#td6| jd7|%d8|$jd|%d+d0Wnty}ztd9|WYd}~nd}~ww| r| | |dD]})z|)WqYqid:|d;d<|d=| r| d=d>nd>d?|d@|dA|jd|dBt|d-|dC|dD| r|n|dE|#dF|#dudG| r(| dGdHndHdI| r3| dInddJ| r?| dKd nd dL| rL| dLdHSdHS)Mz=Standard download without original preservation (16kHz only).z _temp.%(ext)sz _temp.wavrrForiginal_audio_sample_raterrTrr0rrr.r/rMrrNrrrr5z_temp.*rrr+r-r,rr3r4rr5zAudio file corrupted: u# 🎵 Preserving original audio: uHz → ) orig_freqnew_frequ! 📊 Created processing copy: HzrHu⚠️ Audio very short: rr>r8rDrErM)dimkeepdimg-C6?u'⚠️ Audio appears to be silent (RMS=z.6frQu ✅ Saved original quality: rr=u) ⚠️ Failed to save original audio: r,rAr.r7r:r"r#r$rrr%r&r'rJunknownabrchannelsaudio_channelsr9)$rrrrwr?rr<r=r>rvrrrxryrrrrUrurr*rrrrtorchaudio.transforms transformsResampler rrrmeansqrtitemsave)*rGr,rrrrrr"rr temp_filedownloaded_filerroriginal_sr_targetrBrrrCrr rrrrr&T resamplerr8rr# intro_samples outro_samples end_samplermsoriginal_audio_savedrrorig_intro_samplesorig_outro_samplesorig_end_samplerrrrrsD                 "  "0      r)r+)rErF)r r)rN)r)TrN)NN).r(rur loggingrrxpathlibrtypingrrrrrrrr< getLoggerr?rrr!rr#r*rrDboolrXr^rrrrrrrrrrrrrdictrrrrrrs    ( &.Q| !"!3  $L,  |