o ̳i./@sddlmZddlmZmZmZmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZmZddlmZmZmZddlmZdd lmZe d ed Z d>d eededeedeefddZde ee fddfddZ!de de deddeee effddZ"deedefddZ#e  !d?d"e d#ed$eed%e$d&ed'ed(ed)d*e$d+ee efdefd,d-Z%e . d@d/ee efd0ee e&fd1e d&edef d2d3Z'e  4 !dAd5ed6ed7ed8eeegefd9e$d'ed(ed)d:eed;e$defd>> # Load from remote source >>> image = load_image("https://www.wikipedia.org/en/bird.jpg") >>> # Load from local file path >>> image = load_image(Path("/home/user/bird.jpg")) Returns: PIL.Image.Image: The loaded image. r)ImagehttpzFailed to load image from Nz'Failed to open image as PIL Image from ) PILr! isinstancestr startswithr urlopen Exception ValueErroropen)r r!eimagerrr load_image/s  r-content image_tagimagescCs||}t||krtdt|d|d|||}g}t|D]%\}}t|dkr7|d|d|t|dkrJ|d|ddq%|S) a Given a raw text string, split by the specified ``image_tag`` and form into list of dictionaries to be used in the :class:`~torchtune.data.Message` content field:: [ { "role": "system" | "user" | "assistant", "content": [ {"type": "image", "content": }, {"type": "text", "content": "This is a sample image."}, ], }, ... ] Args: content (str): raw message text image_tag (str): string to split the text by images (List["PIL.Image.Image"]): list of images to be used in the content Raises: ValueError: If the number of images does not match the number of image tags in the content Examples: >>> content = format_content_with_images( ... "<|image|>hello <|image|>world", ... image_tag="<|image|>", ... images=[, ] ... ) >>> print(content) [ {"type": "image", "content": }, {"type": "text", "content": "hello "}, {"type": "image", "content": }, {"type": "text", "content": "world"} ] Returns: List[Dict[str, Any]]: list of dictionaries to be used in the :class:`~torchtune.data.Message` content field zNumber of images (z') does not match number of image tags (z) in content: rtext)typer.r,)countlenr)split enumerateappendpop)r.r/r0num_image_tags_in_content split_contentfinal_content_listisubstrrrrformat_content_with_images^s$ -    r?funcscsfdd}|S)z Chain a list of functions together into a single function. Args: *funcs (List[Callable]): list of functions to chain together Returns: Callable: chained function csD]}||}q|SNr)xfnr@rr chained_fns zchain..chained_fnr)r@rErrDrchains rFTthreadFsource transform filter_fnshuffleseed num_workersparallel_method)processrG streamingload_dataset_kwargscKsddlm} m} m} d|vr d|vsJd||d|d<t|fi|} |dur1| |} t\} }|rLt| || d} |rG| j |d} | | }nt | | |||d }| |}t | j |}| ||||d }|S) a Load a HuggingFace dataset (Map or Streaming) and apply a Transform to it. Args: source (str): HuggingFace dataset source. transform (Transform): Transform to apply to the samples of the dataset. filter_fn (Optional[Callable]): Filter function to pass to HuggingFace dataset. shuffle (bool): Whether to shuffle the dataset. Default is True. For streaming datasets, this is passed to HuggingFace dataset as .shuffle(). For map datasets, a DistributedSampler is used. seed (int): Seed for the random number generator in the case of Map style dataset shuffling. Default is 0. num_workers (int): Number of workers to use for loading the dataset. Default is 0 (no parallelism). Setting this greater than 0 will create `parallel_method` workers to perform transforms to the dataset. parallel_method (Literal["process", "thread"]): Method to use for parallelism. Default is "thread". No effect if num_workers is 0. streaming (bool): whether to load a streaming vs map-style dataset. Default False. **load_dataset_kwargs (Dict[str, Any]): Additional Keyword arguments to pass to HuggingFace dataset. See Hugging Face's documentation. Returns: A ``torchdata.nodes`` iterator that can be passed directly to a Loader, or combined with other-datasets in a multi-dataset sampler. r)IterableWrapperParallelMapperSamplerWrappersubsetnamezTfound both 'subset' and 'name' found, you may only specify one, load_dataset_kwargs=N)rank world_size)rL) num_replicasrWrKrLmap_fnrMmethod) torchdata.nodesrRrSrTr9r filterrr rKrrF __getitem__)rHrIrJrKrLrMrNrPrQrRrSrTdatasetrXrWnodesamplerrrrload_hf_datasets8"      rc!CYCLE_UNTIL_ALL_DATASETS_EXHASTEDdatasetsweights stop_criteriacCsddlm}|||||dS)a Given a dictionary of datasets and their corresponding weights, return a dataset that samples from the given datasets according to the specified weights. Args: datasets (Dict[str, DatasetType]): dictionary of datasets weights (Dict[str, float]): dictionary of weights for each dataset. If not stop_criteria (str): stop criteria for the sampler. Default "CYCLE_UNTIL_ALL_DATASETS_EXHASTED". See also: torchdata.nodes.StopCriteria seed (int): seed for the random number generator. Default 0. Returns: A ``torchdata.nodes`` iterator which can be passed to Loader, or further composed with other Nodes. r)MultiNodeWeightedSampler) source_nodesrfrgrL)r]rh)rerfrgrLrhrrrget_multi_datasets rjr`model_transform batch_size collate_fn drop_lastprefetch_factor pin_memoryc Csxddlm} m} m} m} |durt}| ||||d} | | ||d} | | |||d} |r/| | } |dur8| | |} t| S)ax This will configure TorchData Nodes to approximate torch.utils.data.DataLoader. Given a dataset, apply model_transform (eg tokenization), batching, collation, memory pinning, and pre-fetching. Args: dataset (DatasetType): dataset to load. May be a MultiNodeWeightedSampler model_transform (Transform): model transform to apply to the samples of the dataset batch_size (int): batch size collate_fn (Optional[Callable[[Any], Any]]): collate function to apply to the samples of the dataset. If None, use torch.utils.data.default_collate. Default None. drop_last (bool): whether to drop the last batch. Default is True. num_workers (int): number of workers to use for loading the dataset. Default is 0 (no parallelism parallel_method (Literal["process", "thread"]): method to use for parallelism. Default is "thread". prefetch_factor (Optional[int]): number of batches to prefetch. Default is 4. pin_memory (bool): whether to pin memory. Default is False. Returns: A ``torchdata.nodes`` Loader, an Iterable that returns batches. r)BatcherrS PinMemory PrefetcherNrZ)ro)r]rrrSrsrtrr)r`rlrmrnrorMrNrprqrrrSrsrtrarrrget_dataloaders! rurA)NTrrrGF)rdr)NTrrGrkF))pathlibrtypingrrrrrrr r urllibr rer datasets.distributedr torch.utils.datarrtorchtune.data._torchdatarrrtorchtune.modules.transformsrtorchtune.utilsrr2rintrr%r-r?rFboolrcfloatrjrurrrrs (       / ?  F