o i&@slddlZddlZddlZddlZddlmmZddl m Z ddl m Z GdddejjZd ddZdS) N)parse) to_devicecsTeZdZdZdfdd ZddZdd Zd d Zd d ZddZ dddZ Z S)CTCaCTC module :param int odim: dimension of outputs :param int eprojs: number of encoder projection units :param float dropout_rate: dropout rate (0.0 ~ 1.0) :param str ctc_type: builtin :param bool reduce: reduce the CTC loss into a scalar builtinTcst||_d|_tj|||_tj||_ d|_ t tj t dkr(|nd|_ ||j kr:td|j d|j dkrO|rCdnd}tjj|dd|_n,|j d krc|rXdnd}tjj|d |_n|j d krsd d lm}|j|_ntd|j d|_||_dS)Nz1.7.0rzCTC was set to z due to PyTorch version.sumnoneT) reduction zero_infinitycudnnctc)rgtnctcr)GTNCTCLossFunctionz*ctc_type must be "builtin" or "gtnctc": {})super__init__ dropout_ratelosstorchnnLinearctc_loDropoutdropoutprobsV __version__ctc_typeloggingwarningCTCLossctc_loss#espnet.nets.pytorch_backend.gtn_ctcr apply ValueErrorformat ignore_idreduce)selfodimeprojsrrr%reduction_typer  __class__S/home/ubuntu/.local/lib/python3.10/site-packages/espnet/nets/pytorch_backend/ctc.pyrs2           z CTC.__init__cCs|jdvr3|d}tjjjdd|||||}Wdn1s%wY||d}|S|jdkrQdd|D}tjj j|dd }||||d d St ) N)rr T) deterministicr cSsg|]}|qSr,)tolist).0tr,r,r- CzCTC.loss_fn..dimrr) r log_softmaxrbackendscudnnflagsrsizer functionalNotImplementedError)r&th_pred th_targetth_ilenth_olenrtargets log_probsr,r,r-loss_fn8s   z CTC.loss_fnc sfdd|D}|}jdkr|dd}jdkrAt|tdd|D}|}t|} ||||_ nKd_ t t j |t jd }t t j d d |Dt jd }t|}|j}jd krut||}jdkr||}t| ||||j|d _ tjjd dt|dtjjddt|djrLjj _ tdttj j S)acCTC forward :param torch.Tensor hs_pad: batch of padded hidden state sequences (B, Tmax, D) :param torch.Tensor hlens: batch of lengths of hidden state sequences (B) :param torch.Tensor ys_pad: batch of padded character id sequence tensor (B, Lmax) :return: ctc loss value :rtype: torch.Tensor csg|] }||jkqSr,)r$)r2yr&r,r-r4TszCTC.forward..r rr0rcSsg|]}t|qSr,)len)r2sr,r,r-r4\r5Ndtypecss|]}|dVqdS)rN)r<)r2xr,r,r- dszCTC.forward..r z input lengths:  z output lengths: z ctc loss:)rrr transposerr LongTensorlongcatrEr from_numpynpfromiterint32cpuintrKtorinfor+__name__joinstrsplitr%rfloat) r&hs_padhlensys_padysys_hatolensys_truerKr,rGr-forwardIsV         z CTC.forwardcCstj||dd|_|jS)zsoftmax of frame activations :param torch.Tensor hs_pad: 3d tensor (B, Tmax, eprojs) :return: log softmax applied 3d tensor (B, Tmax, odim) :rtype: torch.Tensor r.r6)Fsoftmaxrrr&rar,r,r-rjsz CTC.softmaxcCtj||ddS)zlog_softmax of frame activations :param torch.Tensor hs_pad: 3d tensor (B, Tmax, eprojs) :return: log softmax applied 3d tensor (B, Tmax, odim) :rtype: torch.Tensor r.r6)rir8rrkr,r,r-r8zCTC.log_softmaxcCrl)zargmax of frame activations :param torch.Tensor hs_pad: 3d tensor (B, Tmax, eprojs) :return: argmax applied 2d tensor (B, Tmax) :rtype: torch.Tensor r.r6)rargmaxrrkr,r,r-rnrmz CTC.argmaxrc Csd dd}||}|d}|||}t|dt|fd}tj|dt|ftjdd}|d|d|d<|d|d|d<tj d|dD]} tj t|D]u} || |ksq| d ksq|| || d krt || d| f|| d| dfg} | | dg} n't || d| f|| d| df|| d| d fg} | | d| d g} t | || || || | f<| t | || | f<q[qQd tj |ddftjd} t |d t|df|d t|d fg} t|dt|d g} | t | | d <tj |dd d d D]} || d| | ddff| | <qg}tj d|dD]} ||| | dfq:|S) aforced alignment. :param torch.Tensor h: hidden state sequence, 2d tensor (T, D) :param torch.Tensor y: id sequence tensor 1d tensor (L) :param int y: blank symbol index :return: best alignment results :rtype: list rcSsZt|d}tj|jddftjd|}tj||gdd}|d}t||d}|S)z1Insert blank token between every two label token.r0rrJ)axisr )rU expand_dimszerosshapeint64 concatenatereshapeappend)labelblank_idblanksr,r,r-interpolate_blanks  z+CTC.forced_align..interpolate_blankgvH7BrJr0)rr)rr0r.r Nr)r8squeezerUrqr<rHint16sixmovesrangearraymaxrnonesrv)r&hrFrxrzlpzy_intlogdelta state_pathr3rI candidates prev_state state_seqoutput_state_seqr,r,r- forced_alignsH   (*"&$zCTC.forced_align)rTr{) r\ __module__ __qualname____doc__rrErhrjr8rnr __classcell__r,r,r*r-r s ">   rTcCst|dd}|dkrt||j|j|j|dS|dkrStj}|jr7t||j|jd|j|d}| ||St |D]}t||j|j||j|d}| |q;|St d |)zReturns the CTC module for the given args and output dimension :param Namespace args: the program args :param int odim : The output dimension :param bool reduce : return the CTC loss in a scalar :return: the corresponding CTC module num_encsr0)rr%rz0Number of encoders needs to be more than one. {}) getattrrr(rrrr ModuleList share_ctcrvrr"r#)argsr'r%r ctcs_listctcidxr,r,r-ctc_fors<     r)T)rnumpyrUr~rtorch.nn.functionalrr=ripackaging.versionrr&espnet.nets.pytorch_backend.nets_utilsrModulerrr,r,r,r-s  Z