o rri> @s:dZddlZddlZd ddZedZ dd d ZdS)z Ways to make the model stronger.NcCs|dksJ|jd|jdksJ|jd}tj|||j|jd}t|D]}||}|jddd}|d|}q(| S)zXThis is the power method. batch size is used to try multiple starting point in parallel.rr)devicedtypeT)dimkeepdimg|=) rshapetorchrandnrrrangemmnormmean)mnitersbsrb_nr r>/home/ubuntu/.local/lib/python3.10/site-packages/demucs/svd.pypower_iteration s   ri皙?rFTc Cszd} t|kr dS|D]} | jddD]\} } | d|kr#q|r>t| tjjtjj fr>| dvr>| dd } | dkrM| t| d } n| d kr\| t| d } n | dkrcq|rfq| d kssJ| | jf|rtj| dd dd }n/|r| j\}}||kr| | }n| | }t||| }nt| ||ddd }| |7} qq| |S) a Penalty on the largest singular value for a layer. Args: - model: model to penalize - min_size: minimum size in MB of a layer to penalize. - dim: projection dimension for the svd_lowrank. Higher is better but slower. - niters: number of iterations in the algorithm used by svd_lowrank. - powm: use power method instead of lowrank SVD, my own experience is that it is both slower and less stable. - convtr: when True, differentiate between Conv and Transposed Conv. this is kept for compatibility with older experiments. - proba: probability to apply the penalty. - conv_only: only apply to conv and conv transposed, not LSTM (might not be reliable for other models than Demucs). - exact: use exact SVD (slow but useful at validation). - bs: batch_size for power method. rgF)recursei)rrrr) compute_uv) penalty_rngrandommodulesnamed_parametersnumel isinstancer nnConvTranspose1dConvTranspose2dr transpose contiguousviewlenrsvdpowmaxr tr svd_lowrank)modelmin_sizerrpowmconvtrproba conv_onlyexactrtotalrnamepestimatearrrrr svd_penaltys@        r<)rr) rrrFTrFFr)__doc__rr rRandomrr<rrrrs