o ॵiF@sddlZddlmZddlmZddlZddlZddlm Z ddl m Z m Z e GdddeZd d Ze Gd d d eZe Gd ddeZddZe GdddeZe GdddeZe GdddeZdS)N)Sequence)Mapping) default_group) PREPROCESSORSbuild_preprocessorc@s*eZdZdZd ddZddZdd ZdS) Composea(Compose a data pipeline with a sequence of transforms. Args: transforms (list[dict | callable]): Either config dicts of transforms or transform objects. profiling (bool, optional): If set True, will profile and print preprocess time for each step. NFc Cst|tsJ||_g|_||_|D]<}t|tr:|jdur$t|t}n$zt||}Wnty9t|t}Ynwt |r?n t dt ||j |qdS)Nz.transform must be callable or a dict, but got ) isinstancer profiling transforms field_namedictrrKeyErrorcallable TypeErrortypeappend)selfr r r transformrS/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/preprocessors/common.py__init__s,    zCompose.__init__cCsT|jD]$}|jr t}||}|jr t|dt||dur'dSq|S)Nz time )r r timeprint)rdatatstartrrr__call__/s zCompose.__call__cCs2|jjd}|jD] }|d|7}q |d7}|S)N(z z )) __class____name__r )r format_stringrrrr__repr__=s  zCompose.__repr__)NFr __module__ __qualname____doc__rrr"rrrrrs   rcCst|tjr|St|tjrt|St|tr"t|ts"t|St|t r-t |gSt|t r8t |gSt dt|d)a8Convert objects of various python types to :obj:`torch.Tensor`. Supported types are: :class:`numpy.ndarray`, :class:`torch.Tensor`, :class:`Sequence`, :class:`int` and :class:`float`. Args: data (torch.Tensor | numpy.ndarray | Sequence | int | float): Data to be converted. type  cannot be converted to tensor.)r torchTensornpndarray from_numpyrstrtensorint LongTensorfloat FloatTensorrrrrrr to_tensorEs        r5c@*eZdZdZd ddZddZddZdS) ToTensoru Convert target object to tensor. Args: keys (Sequence[str]): Key of data to be converted to Tensor. Only valid when data is type of `Mapping`. If `keys` is None, all values of keys ​​will be converted to tensor by default. NcC ||_dSNkeysrr;rrrrh zToTensor.__init__cCVt|tr%|jdurt||_|jD]}||vr"t||||<q|St|}|Sr9)r rr;listr5rrkeyrrrrk   zToTensor.__call__cC|jjd|jdSNz(keys=)rr r;rrrrr"xzToTensor.__repr__r9r#rrrrr7^   r7c@s(eZdZdZddZddZddZdS) FilteraThis is usually the last stage of the dataloader transform. Only data of reserved keys will be kept and passed directly to the model, others will be removed. Args: keys (Sequence[str]): Keys of data to be reserved, others will be removed. cCr8r9) reserved_keys)rrKrrrrr=zFilter.__init__cCs6t|tsJi}|jD] }||vr||||<q |Sr9)r rrK)rr reserved_datarArrrrs  zFilter.__call__cCrCrD)rr rKrGrrrr"rHzFilter.__repr__Nr#rrrrrJ|s  rJcCst|tjr |St|tjr|St|tr!t|ts!t|St|t r.tj|tj dSt|t r;tj|tj dSt dt|d)zConvert objects of various python types to `numpy.ndarray`. Args: data (torch.Tensor | numpy.ndarray | Sequence | int | float): Data to be converted. )dtyper'r()r r)r*numpyr+r,rr.asarrayr0int64r2float64rrr4rrrto_numpys     rRc@r6) ToNumpyu!Convert target object to numpy.ndarray. Args: keys (Sequence[str]): Key of data to be converted to numpy.ndarray. Only valid when data is type of `Mapping`. If `keys` is None, all values of keys ​​will be converted to numpy.ndarray by default. NcCr8r9r:r<rrrrr=zToNumpy.__init__cCr>r9)r rr;r?rRr@rrrrrBzToNumpy.__call__cCrCrDrFrGrrrr"rHzToNumpy.__repr__r9r#rrrrrSrIrSc@s.eZdZdZggfddZddZddZdS) RenamezDChange the name of the input keys to output keys, respectively. cCs||_||_dSr9) input_keys output_keys)rrUrVrrrrs zRename.__init__cCsLt|tr$t|j|jD]\}}||vr#||vr#||||<||q |Sr9)r rziprUrVpop)rrin_keyout_keyrrrrs   zRename.__call__cCrCrDrFrGrrrr"rHzRename.__repr__Nr#rrrrrTs  rTc@seZdZddZddZdS)IdentitycCsdSr9rrGrrrrzIdentity.__init__cCs|Sr9r)ritemrrrrr\zIdentity.__call__N)r r$r%rrrrrrr[s r[)rcollections.abcrtypingrrNr+r)modelscope.utils.registryrbuilderrrregister_moduleobjectrr5r7rJrRrSrTr[rrrrs*   6