o iX@s.dZddlZddlZddlZGdddZdS)z| Used for EMA tracking a given pytorch module. The user is responsible for calling step() and setting the appropriate decay Nc@sVeZdZdZdddZdddZdd d Zd d Zd dZddZ ddZ ddZ dS) EMAModulez,Exponential Moving Average of Fairseq ModelsH.?FNcCsv||_||_t||_|jd|pt|_i|_|dur/t d||jj |d|_|jr6| d|_ dS)a, @param model model to initialize the EMA with @param config EMAConfig object with configuration like ema_decay, ema_update_freq, ema_fp32 @param device If provided, copy EMA to this device (e.g. gpu). Otherwise EMA is in the same device as the model. FNzCopying EMA model to device )devicer)decayema_fp32copydeepcopymodelrequires_grad_set skip_keys fp32_paramslogginginfotobuild_fp32_paramsupdate_freq_counter)selfr ema_decayrrr rU/home/ubuntu/.local/lib/python3.10/site-packages/funasr/models/data2vec/ema_module.py__init__s     zEMAModule.__init__cCsh|jstd|dur|j}dd}|D]}||jvr(|j|||q||||j|<qdS)z Store a copy of the EMA params in fp32. If state dict is passed, the EMA params is copied from the provided state dict. Otherwise, it is copied from the current EMA model parameters. zgbuild_fp32_params should not be called if ema_fp32=False. Use ema_fp32=True if this is really intended.NcSst|r |S|SN)torchis_floating_pointfloat)trrr _to_float=sz.EMAModule.build_fp32_params.._to_float)r RuntimeErrorr state_dictr copy_)rrr param_keyrrrr-s  zEMAModule.build_fp32_paramscCs&|jj|dd|r||dSdS)z*Load data from a model spec into EMA modelFstrictN)r load_state_dictr)rrrrrrrestoreFszEMAModule.restorecCs ||_dSrr)rrrrr set_decayLs zEMAModule.set_decaycCs|jSrr&)rrrr get_decayOszEMAModule.get_decayc Cs0|j}i}|jr |jn|j}|D]x\}}t|tr qz||}Wnty?|j dkr8| nt |}Ynw|j|jkrRtdd|j|jd|vrWq||jvsfd|vrw|jtjkrw|j|jd }|||n|||j|j|jdd|d|||<q|j|dd d S) z6One update of the EMA model based on new model weightszs