o TiZ @sVddlZddlZddlmZddlmZddlmZmZddl Z ddl Z ddl m Z mZd#deded efd d Zd$d e jd e jdeed e jfddZd%d e jd e jded e jfddZd%d e jd e jded e jfddZGdddZGdddZ    d&deeefdeeee je jfdee jdee jd ee jd e jf d!d"ZdS)'N)Mapping)contextmanager)OptionalUnion)is_torch_npu_availableis_torch_xpu_available/nestedsepreturncs4dtdtdtddffdd i}|d||S) z>Flatten dictionary and concatenate nested keys with separator.nestprefixintor Ncs`|D])\}}|vrtdd|dt|tr'||||q||||<qdS)Nz separator 'z' not allowed to be in key '')items ValueError isinstancer)r r rkvrecurser .recurse)dictstr)r r flatrrr flatten_dicts  rvaluesmaskaxiscCs8|dur||j|d|j|dS|||S)z,Compute mean of tensor with a masked values.N)r )sum)rrr rrr masked_mean+sr"TunbiasedcCsTt||}||}t|d|}|r(|}|dkrtd||d}||}|S)z.Compute variance of tensor with masked values.rzThe sum of the mask is zero, which can happen when `mini_batch_size=1`;try increase the `mini_batch_size` or `gradient_accumulation_steps`)r"r!r)rrr#meancentered_valuesvariancemask_sumbessel_correctionrrr masked_var3s  r+ shift_meancCs<t||t||}}||t|d}|s||7}|S)z!Whiten values with masked values.g:0yE>)r"r+torchrsqrt)rrr,r&varwhitenedrrr masked_whitenFs r1c@s0eZdZdZdedefddZdefddZd S) LengthSamplerz Samples a length min_value max_valuecCstt|||_dSN)listranger)selfr3r4rrr__init__TszLengthSampler.__init__r cCstj|jSr5)nprandomchoicer)r8rrr__call__WszLengthSampler.__call__N)__name__ __module__ __qualname____doc__intr9r=rrrrr2Osr2c@s eZdZdZeeddZdS) PPODecoratorsFccsdV|jr?trttjtdStr+ttjtdStj rAttj tdSdSdSr5) optimize_device_cachergccollectr-xpu empty_cachernpucuda is_available)clsrrrempty_device_cache^s"        z PPODecorators.empty_device_cacheN)r>r?r@rD classmethodrrMrrrrrC[s rCshape generatordevicedtypelayoutc s>|d}p tj|ptd}durYttsjjndjj}||jkrE|dkrEd|dkrDtd|d|d|dtn||jkrY|d krYt d |d |d ttrht d krhdttrdd dfddt |D}tj |dd |}|Stjd |}|S)zA helper function to create random tensors on the desired `device` with the desired `dtype`. When passing a list of generators, you can seed each batch size individually. If CPU generators are passed, the tensor is always created on the CPU. rcpuNmpszBThe passed generator was created on 'cpu' even though a tensor on zB was expected. Tensors will be created on 'cpu' and then moved to zk. Note that one can probably slighly speed up this function by passing a generator that was created on the z device.rJzCannot generate a z! tensor from a generator of type .r%)r%c s$g|]}tj|dqS)rPrQrRrS)r-randn).0irRrPrS rand_devicerOrr sz randn_tensor..)dimrW)r-stridedrQrr6typewarningswarn UserWarningrlenr7cattorX)rOrPrQrRrS batch_sizegen_device_typelatentsrr[r randn_tensorqs>   rj)rr5)T)NNNN) rEracollections.abcr contextlibrtypingrrnumpyr:r- transformersrrrrrTensorboolr"r+r1r2rCtupler6 GeneratorrQrRrSrjrrrrs>  &""