o }oi F @sddlZddlZddlmZmZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZmZdd lmZdd lmZz ddlmZdZ Wn e!e"fyqdZ YnwdgZ#GdddeZ$dS)N)ListOptional) instantiate)Trainer) DictConfig)CrossEntropyLoss)ClassificationReport)TokenClassifier) tensor2list)HeteronymClassificationDataset)get_heteronym_spansget_wordid_to_phonemes read_wordids)PretrainedModelInfo)logging)NLPModelTFHeteronymClassificationModelc seZdZdZdAdedeffdd ZddZd d Zd d Z fd dZ dBddZ ddZ ddZ ddZdefddZdedeeedeefddZe ! dCd"eed#ed$edeefd%d&ZedDd(ed#ed$efd)d*Ze ' ! dEd+ed,ed#ed$edeef d-d.Zd/eefd0d1Zd2eefd3d4Zd5eefd6d7Zded8efd9d:Zd(ed,ed#ed$ed;dZed;ee fd?d@Z!Z"S)Frz This is a classification model that selects the best heteronym option out of possible dictionary entries. Supports only heteronyms, no OOV. Ncfgtrainerc sJ|j|_|d|j|_t|j\|_|_dd|jD|_t|j |_ |j j dur_d}t |d}tt|jD] }||j|dq=Wdn1sTwY|d|tj||d|jd d|_t|j}t|j||jjj|jjjd |jjj|jjjd |_td d |_t |dd|jd|_!d|_"d|_#dS)NwordidscSsi|]\}}||qSr).0kvrrl/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/tts/g2p/models/heteronym_classification.py 6z9HeteronymClassificationModel.__init__..z/tmp/label_ids.csvw zclass_labels.class_labels_file)rrlangF) hidden_size num_classes num_layers activation log_softmaxdropoutuse_transformer_init) logits_ndimmacroT)r!modedist_sync_on_step label_ids)$max_seq_lengthregister_artifactrrheteronym_dict wordid_to_idxitems idx_to_wordidlistkeyssupported_heteronyms class_labelsclass_labels_fileopenrangelenwritesuper__init___cfggetrr r head num_fc_layersr# fc_dropoutr& classifierrlossrclassification_reportwordid_to_phonemes_filewordid_to_phonemes)selfrrlabel_ids_filefidxr! __class__rrr=2s>       z%HeteronymClassificationModel.__init__cCs2|j|||d}t|tr|d}|j|d}|S)N input_idsattention_masktoken_type_idsr) hidden_states) bert_model isinstancetuplerC)rHrOrPrQrRlogitsrrrforwardYs  z$HeteronymClassificationModel.forwardcCsP|j|d|dt|dd}d|vr"|j||dd}||fSd}||fS)NrOrPrNtargets)rVlabels)rWtorch zeros_likerD)rHbatchrVrDrrr make_stepbs z&HeteronymClassificationModel.make_stepcCs||\}}|d||S)z Lightning calls this inside the training loop with the data from the training dataloader passed in as `batch`. train_loss)r]log)rHr\ batch_idxrDrVrrr training_steprs z*HeteronymClassificationModel.training_stepcs tSN)r<on_train_epoch_endrHrLrrrc|s z/HeteronymClassificationModel.on_train_epoch_endvalcCs||\}}|d}|d}||dk}||d|tj|dd|dk}|||\} } } } |d|d| d | d | i} |d krL|j| | S|d krV|j| | S) z Lightning calls this inside the validation loop with the data from the validation dataloader passed in as `batch`. subtokens_maskrXi_lossaxisrtpfnfpretest)r]r_rZargmaxrEvalidation_step_outputsappendtest_step_outputs)rHr\r`splitval_lossrVrfrX tag_predsrkrlrm_rDrrrvalidation_steps   z,HeteronymClassificationModel.validation_stepcs|jjrdnddkrtfdd|jD}ndkr/tfdd|jD}|j\}}}}d dd| dD}t d|t d |d d |j d |d d| d|| d|| d|||dd dddd d}||dd dddd d}| dtt|g| dtt|g|jdkr|jdSdkr|jdSdS)Nrnrecg|] }|dqSrgrrxrsrr zHHeteronymClassificationModel.on_validation_epoch_end..crxryrrzr|rrr}r~rcSs"g|] }|dsd|vr|qS)z 0z100.00 100.00 100.00)endswithrzrrrr}sz _report: z_f1: z.2f%rgT)prog_bar _precision_f1_recallr)rz macro avgmicroz micro avg _f1_macro _f1_micro)rtestingrZstackrpmeanrrrEcomputejoinrsrinfor_indexreplacestripTensorfloatresetclear)rHavg_loss precisionrecallf1reportf1_macrof1_microrr|ron_validation_epoch_ends6 00 z4HeteronymClassificationModel.on_validation_epoch_endcCs|||dS)zx Lightning calls this inside the test loop with the data from the test dataloader passed in as `batch`. rn)rw)rHr\r`rrr test_stepsz&HeteronymClassificationModel.test_stepcCs|S)z Called at the end of test to aggregate outputs. Args: outputs: list of individual outputs of each test step. )rrdrrron_test_epoch_endsz.HeteronymClassificationModel.on_test_epoch_endrFcCsN|dus tj|st|ddS||_t|j|_td|dS)Nz, not found, skip setting wordid_to_phonemes.z"Wordid to phonemes file is set to ) ospathexistsrwarningrFr rGr)rHrFrrrset_wordid_to_phonemess  z3HeteronymClassificationModel.set_wordid_to_phonemestext start_end predictionsc Csd}d}t|D]8\}}|\}} ||} |jdus| |jvr%d| d} n|j| } ddd| D} ||||| 7}| }q|t|krO|||d7}|S)Nrr[]cSsg|]}d|dqS)|rrprrrr}zBHeteronymClassificationModel._process_sentence..) enumeraterGrr:) rHrrrtext_with_heteronym_replacedlast_idx heteronym_idx cur_start_end cur_startcur_endcur_predrrr_process_sentences  z.HeteronymClassificationModel._process_sentencer sentences batch_size num_workerscCs6t|tr|g}t|t|}t||j\}}t|t|kr't|kr7nntdt|dt|d}t|d&}t|||D]\} } } | | | d} | t j | dddqEWd n1sgwY|j |||d } |d ur}| |g}t|D]\}}||j|||| |d q| |fS) a Replaces heteronyms, supported by the model, with the phoneme form (if wordid_to_phonemes_file) or with predicted wordids. Args: sentences: Sentences to use for inference batch_size: batch size to use during inference. Bigger will result in better throughput performance but would use more memory. num_workers: number of workers for DataLoader wordid_to_phonemes_file: (Optional) file with mapping between wordid predicted by the model to phonemes Returns: preds: model predictions output: sentences with heteronym replaced with phonemes (if wordid_to_phonemes_file specified) zLNumber of sentences should match the lengths of provided start-end indices, z != z/tmp/manifest.jsonr)text_graphemesrheteronym_spanF ensure_asciirN)manifestrrrrr)rTstrminr:r r/ ValueErrorr8zipr;jsondumps _disambiguaterrrqr)rHrrrrFr heteronyms tmp_manifestrJ cur_sentencecur_start_endscur_heteronymsitem all_predsoutputsent_idxsent_start_endrrr disambiguates< $   z)HeteronymClassificationModel.disambiguaterrc sg}j}zztjr dnd}|j||||d}|D]S} | d} | d|| d|d} | \} } ttj | dd | d k} d d t| D}d }|D]}| |||}fd d |D}| |||7}qYq#Wj |d|Sj |dw)Ncudacpu)grapheme_fieldrrrfrOrP)rOrPrhrircSsg|] }tdd|DqS)cSsg|]}|dkr|qS)r)rp_rrrr}9rzIHeteronymClassificationModel._disambiguate...)r:rrrrr}9sz>HeteronymClassificationModel._disambiguate..csg|]}j|qSr)r2rrdrrr}>r)r*) trainingrZr is_availableevalto_setup_infer_dataloaderr]r rorqtrain)rHrrrrrr*deviceinfer_datalayerr\rfrvrVpreds preds_numrnumpreds_rrdrr!s8      z*HeteronymClassificationModel._disambiguateoutput_manifestrc Cs|j||||d}||t|dddc}t|dddL} t|D]?\} } t| } | d} t| dkr?t| dtr?| g} |j | || || d} | | d <|| | d <| tj | d d d q"Wdn1slwYWdn1s{wYt d||S)N)rrrrrzutf-8)encodingrrrr pred_text pred_wordidFrrzPredictions save at )rrr8rrloadsr:rTintrr;rrr)rHrrrrrrFrf_inf_predsrKlinerrrrrdisambiguate_manifestFs,     z2HeteronymClassificationModel.disambiguate_manifesttrain_data_configcC8|r|jjdurtdd|_dS|j|dd|_dS)Nz`Dataloader config or file_path for the train is missing, so no data loader for train is created!rr data_split)datasetrrr _train_dl_setup_dataloader_from_config)rHrrrrsetup_training_dataiz0HeteronymClassificationModel.setup_training_dataval_data_configcCr)NzjDataloader config or file_path for the validation is missing, so no data loader for validation is created!rer)rrrr_validation_dlr)rHrrrrsetup_validation_datarrz2HeteronymClassificationModel.setup_validation_datatest_data_configcCr)Nz^Dataloader config or file_path for the test is missing, so no data loader for test is created!rnr)rrrr_test_dlr)rHrrrrsetup_test_data{rz,HeteronymClassificationModel.setup_test_datarc Csd|vs t|jtstd|d|vst|jts"td|t|j|jj|jj|j|j |j |j dd}t j jj|fd|ji|jS)NrzNo dataset for dataloader_paramszNo dataloader_params for Trr tokenizerr0r/ max_seq_len with_labels collate_fn)rTrrrrrrrrr0r/r-rZutilsdata DataLoaderr)rHrrrrrrrs z:HeteronymClassificationModel._setup_dataloader_from_configreturnztorch.utils.data.DataLoaderc Cs>t|||j|j|j|jjjdd}tjjj||j |d|ddS)NFr)rrshuffler drop_last) r rr0r/model_max_lengthrZrrrr)rHrrrrrrrrrs" z4HeteronymClassificationModel._setup_infer_dataloadercCsgS)z This method returns a list of pre-trained model which can be instantiated directly from NVIDIA's NGC cloud. Returns: List of available pre-trained models. r)clsrrrlist_available_modelssz2HeteronymClassificationModel.list_available_modelsrb)re)rrN)rr)rrrN)#__name__ __module__ __qualname____doc__rrr=rWr]rarcrwrrrrrrrrrZno_gradrrrrrrrrr classmethodrr  __classcell__rrrLrr,sz'  % "  9$ "    )%rrtypingrrrZ hydra.utilsrlightning.pytorchr omegaconfrnemo.collections.common.lossesr2nemo.collections.nlp.metrics.classification_reportr#nemo.collections.nlp.modules.commonr &nemo.collections.nlp.parts.utils_funcsr 6nemo.collections.tts.g2p.data.heteronym_classificationr nemo.collections.tts.g2p.utilsr r rnemo.core.classes.commonr nemo.utilsr%nemo.collections.nlp.models.nlp_modelr NLP_AVAILABLEModuleNotFoundError ImportError__all__rrrrrs.