o ci+ @s dZddlZddlmZddlmZddlZddlmZddlm Z ddl m Z ddl mZe eGdd d Zed d Gd d d ejZed d GdddejZed d GdddejZed d  ddejdededeeefdejjf ddZed d GdddejZdS)a Adapted from https://github.com/karpathy/minGPT Full definition of a GPT Language Model, all of it in this single file. References: 1) the official GPT-2 TensorFlow implementation released by OpenAI: https://github.com/openai/gpt-2/blob/master/src/model.py 2) huggingface/transformers PyTorch implementation: https://github.com/huggingface/transformers/blob/main/src/transformers /models/gpt2/modeling_gpt2.py N) dataclass)Tuple) functional) DeveloperAPI) Deprecatedc@s^eZdZUeed<dZeed<dZeed<dZeed<dZe ed<dZ e ed <dZ e ed <d S) GPTConfig block_size n_layern_headin_embedg? embed_pdrop resid_pdrop attn_pdropN) __name__ __module__ __qualname__int__annotations__r r r r floatrrrrQ/home/ubuntu/.local/lib/python3.10/site-packages/ray/rllib/models/torch/mingpt.pyrs      rF)errorc@seZdZdZddZdS)NewGELUz Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT). Reference: Gaussian Error Linear Units (GELU) paper: https://arxiv.org/abs/1606.08415 c Cs6d|dttdtj|dt|dS)Ng??g@gHm?g@)torchtanhmathsqrtpipow)selfxrrrforward4s"zNewGELU.forwardN)rrr__doc__r#rrrrr+s rc0eZdZdZdeffdd ZdddZZS) CausalSelfAttentionz Vanilla multi-head masked self-attention layer with a projection at the end. It is possible to use torch.nn.MultiheadAttention here but I am including an explicit implementation here to show that there is nothing too scary here. configc st|j|jdksJt|jd|j|_t|j|j|_t|j |_ t|j |_ | dtt|j|jdd|j|j|j|_|j|_dS)Nrbias)super__init__r r nnLinearc_attnc_projDropoutr attn_dropoutr resid_dropoutregister_bufferrtrilonesrviewr!r' __class__rrr,Is   zCausalSelfAttention.__init__Nc CsB|\}}}||j|jdd\}}}||||j||jdd}||||j||jdd}||||j||jdd}||dddt|d} | |j ddddd|d|fdkt d} |durx| |} t j | dd} || } | |} | dd|||} ||| } | | fS) N)dimr*rrz-inf)sizer/splitr r7r transposerr masked_fillr)rFsoftmaxr2 contiguousr3r0) r!r"attention_masksBTCqkvattyrrrr#^s   $2 zCausalSelfAttention.forwardNrrrr$rr,r# __classcell__rrr9rr&Asr&cr%) Blockzan unassuming Transformer blockr'c svtt|j|_t||_t|j|_t t t |jd|jt d|j|jt t |jd|_dS)N)c_fcr0actdropout)r+r,r- LayerNormr ln_1r&attnln_2 ModuleDictdictr.rr1rmlpr8r9rrr,s    zBlock.__init__Nc CsT|j|||d\}}||}|j|j|j|j|}||}||fS)NrF)rYrXr]rVr0rUrT)r!r"rFx_attrMx_ffnrrrr#s $z Block.forwardrOrPrrr9rrR}srRg?gffffff?model learning_rate weight_decaybetasreturnc stt}t}tjjf}tjjtjjf}|D]A\} } | D]8\} } | r+d| | fn| } | dr8| | q| drHt | |rH| | q| drWt | |rW| | qqt |||@}||B}t |dkswJdt |dt |dksJdt |dfdd t|D|d fd d t|Dd d g}tjj|f||d |}|S)ai This long function is unfortunately doing something very simple and is being very defensive: We are separating out all parameters of the model into two buckets: those that will experience weight decay for regularization and those that won't (biases, and layernorm/embedding weights). We are then returning the PyTorch optimizer object. z%s.%sr)weightrz parameters z' made it into both decay/no_decay sets!z3 were not separated into either decay/no_decay set!cg|]}|qSrr.0pn param_dictrr z+configure_gpt_optimizer..)paramsrdcrhrrrirlrrrnro)lrre)setrr-r.rW Embedding named_modulesnamed_parametersendswithadd isinstancer\lenstrkeyssortedoptimAdamW)rbrcrdrekwargsdecayno_decaywhitelist_w_modulesblacklist_w_modulesmnmrkpfpn inter_params union_params optim_groups optimizerrrlrconfigure_gpt_optimizersB       rcs8eZdZdZdeffdd ZddZd d d ZZS) GPTzGPT Transformer Modelr'c stjdus Jj|_tttjtfddt j Dt j d|_ ||j|D]\}}|drUtjjj|ddtdj dq:dS) Ncsg|]}tqSr)rR)rj_r'rrrnroz GPT.__init__..)drophln_fz c_proj.weightrq{Gz?r;meanstd)r+r,rr-r[r\r1r ModuleListranger rWr transformerapply _init_weightsrvrwrinitnormal_rr)r!r'rkrr9rrr,s$     z GPT.__init__cCst|tjr"tjjj|jddd|jdur tjj|jdSdSt|tj r5tjjj|jddddSt|tj rMtjj|jtjj |jdSdS)Nrqrr) ryr-r.rrrrgr)zeros_rtrWones_)r!modulerrrrs    zGPT._init_weightsNFc Cs|\}}}||jksJd|d|j|durD|\}}||kr)||ks+J|ddddddf}|j|jd}d|d}|j|} g} |jjD]} | | |d\} } | | qP|j| } |rl| | fS| S)z input_embeds: [batch_size x seq_len x n_embed] attention_masks: [batch_size x seq_len], 0 don't attend, 1 attend z"Cannot forward sequence of length z, block size is only N)dtyperger^) r?rtorrrrappendr) r! input_embedsrFreturn_attentionsrGrHrI_B_Tr"attsblockrMrrrr#s*       z GPT.forward)NF) rrrr$rr,rr#rQrrr9rrs  r)ra)r$r dataclassesrtypingrrtorch.nnr-rrCray.rllib.utils.annotationsrray.rllib.utils.deprecationrrModulerr&rRrr~ OptimizerrrrrrrsB      ;  A