o ϯi@sddlmZmZddlZddlmZ dedeefddZGdddej Z Gd d d ej Z Gd d d ej Z deefd dZ dS))CallableListNmethod feature_dimscCsr|dkrt|S|dkrt|S|dkrt|ddS|dkr&t|ddS|dkr1t|d dStd |d ) a  Args: method (str): the fusion method to be constructed. Options: - 'concat' - 'temporal_concat' - 'max' - 'sum' - 'prod' feature_dims (List[int]): the first argument of all fusion layers. It holds a list of required feature_dims for each tensor input (where the tensor inputs are of shape (batch_size, seq_len, feature_dim)). The list order must corresponds to the tensor order passed to forward(...). concattemporal_concatmaxcSstj|ddjSNrdim)torchrvaluesxrN/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/layers/fusion.py%sz#make_fusion_layer..sumcStj|ddSr )r rrrrrr'prodcSrr )r rrrrrr)rzFusion z not available.) ConcatFusionTemporalConcatFusion ReduceFusionNotImplementedError)rrrrrmake_fusion_layersrcPeZdZdZdeeffdd ZeddZdee j de j fd d Z Z S) rz Concatenates all inputs by their last dimension. The resulting tensor last dim will be the sum of the last dimension of all input tensors. rcs tt|t||_dSN)super__init___verify_feature_dimr _output_dimselfr __class__rrr4s zConcatFusion.__init__cC|jSzC Last dimension size of forward(..) tensor output. r!r#rrr output_dim9zConcatFusion.output_dim input_listreturncCtj|ddS)a8 Args: input_list (List[torch.Tensor]): a list of tensors of shape (batch_size, seq_len, feature_dim). Returns: Tensor of shape (batch_size, seq_len, sum(feature_dims)) where sum(feature_dims) is the sum of all input feature_dims. r r catr#r,rrrforward@ zConcatFusion.forward __name__ __module__ __qualname____doc__rintrpropertyr*r Tensorr3 __classcell__rrr$rr.s  "rcr) rz[ Concatenates all inputs by their temporal dimension which is assumed to be dim=1. rcs2tt|t||_|jt|ksJdSr)rrr rr!minr"r$rrrRs  zTemporalConcatFusion.__init__cCr&r'r(r)rrrr*Zr+zTemporalConcatFusion.output_dimr,r-cCr.)a, Args: input_list (List[torch.Tensor]): a list of tensors of shape (batch_size, seq_len, feature_dim) Returns: Tensor of shape (batch_size, sum(seq_len), feature_dim) where sum(seq_len) is the sum of all input tensors. r r0r2rrrr3ar4zTemporalConcatFusion.forwardr5rrr$rrMs  "rcsbeZdZdZdeedeejgejfffdd Z e ddZ deejd ejfd d Z Z S) rz Generic fusion method which takes a callable which takes the list of input tensors and expects a single tensor to be used. This class can be used to implement fusion methods like "sum", "max" and "prod". r reduce_fncs8tt|||_t||_|jt|ksJdSr)rrr r@rr!r>)r#rr@r$rrrus  zReduceFusion.__init__cCr&r'r(r)rrrr*r+zReduceFusion.output_dimr,r-cCs|t|S)z Args: input_list (List[torch.Tensor]): a list of tensors of shape (batch_size, seq_len, feature_dim). Returns: Tensor of shape (batch_size, seq_len, feature_dim). )r@r stackr2rrrr3s zReduceFusion.forward)r6r7r8r9rr:rr r<rr;r*r3r=rrr$rrns "rcCs(t|tsJtdd|DsJdS)Ncss|]}|dkVqdS)rNr).0rrrr sz&_verify_feature_dim..) isinstancelistall)rrrrr sr )typingrrr torch.nnnnstrr:rModulerrrr rrrrs !%