o Ni-@sdZddlmZddlmZddlmZddlZddlZddlmm Z ddl m Z ddl mZdZdZGd d d ejjZGd d d ejjZed ZddZdS)zCityscapes Datasets.)absolute_import)division)print_functionN) api_utilsa@inproceedings{Cordts2016Cityscapes, title={The Cityscapes Dataset for Semantic Urban Scene Understanding}, author={Cordts, Marius and Omran, Mohamed and Ramos, Sebastian and Rehfeld, Timo and Enzweiler, Markus and Benenson, Rodrigo and Franke, Uwe and Roth, Stefan and Schiele, Bernt}, booktitle={Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2016} } a Cityscapes is a dataset consisting of diverse urban street scenes across 50 different cities at varying times of the year as well as ground truths for several vision tasks including semantic segmentation, instance level segmentation (TODO), and stereo pair disparity inference. For segmentation tasks (default split, accessible via 'cityscapes/semantic_segmentation'), Cityscapes provides dense pixel level annotations for 5000 images at 1024 * 2048 resolution pre-split into training (2975), validation (500) and test (1525) sets. Label annotations for segmentation tasks span across 30+ classes commonly encountered during driving scene perception. Detailed label information may be found here: https://github.com/mcordts/cityscapesScripts/blob/master/cityscapesscripts/helpers/labels.py#L52-L99 Cityscapes also provides coarse grain segmentation annotations (accessible via 'cityscapes/semantic_segmentation_extra') for 19998 images in a 'train_extra' split which may prove useful for pretraining / data-heavy models. Besides segmentation, cityscapes also provides stereo image pairs and ground truths for disparity inference tasks on both the normal and extra splits (accessible via 'cityscapes/stereo_disparity' and 'cityscapes/stereo_disparity_extra' respectively). Ingored examples: - For 'cityscapes/stereo_disparity_extra': - troisdorf_000000_000073_{*} images (no disparity map present) WARNING: this dataset requires users to setup a login and password in order to get the files. cs,eZdZdZej  dfdd ZZS)CityscapesConfigaBuilderConfig for Cityscapes. Args: right_images (bool): Enables right images for stereo image tasks. segmentation_labels (bool): Enables image segmentation labels. disparity_maps (bool): Enables disparity maps. train_extra_split (bool): Enables train_extra split. This automatically enables coarse grain segmentations, if segmentation labels are used. FTc stt|jdddi|||_||_||_||_t|_i|_ d|j d<|jr-d|j d<|jr=d|j d<|jr=d |j d <|jrY|jsLd |j d <d |_ n d|j d <d|j d<d|_ |jrqd|j d<|jrsd|j d<|j ddSdSdS)Nversionz1.0.0)zleftImg8bit_trainvaltest.zip leftImg8bit images_left)zleftImg8bit_trainextra.ziprzimages_left/extra)zrightImg8bit_trainvaltest.zip rightImg8bit images_right)zrightImg8bit_trainextra.zipr zimages_right/extra)zgtFine_trainvaltest.zipgtFinesegmentation_labelsgtFine_labelIds)z gtCoarse.zipgtCoarsezsegmentation_labels/extragtCoarse_labelIds)zdisparity_trainvaltest.zip disparitydisparity_maps)zdisparity_trainextra.ziprzdisparity_maps/extratroisdorf_000000_000073) superr__init__ right_imagesr rtrain_extra_splitset ignored_idszip_root label_suffixadd)selfrr rrkwargs __class__rX/home/ubuntu/.local/lib/python3.10/site-packages/tensorflow_datasets/image/cityscapes.pyrNs8         zCityscapesConfig.__init__)FTFF)__name__ __module__ __qualname____doc__rdisallow_positional_argsr __classcell__rrr r"rCs  rc @sxeZdZdZdZedddddddedd ddddded d ddddded d dddddgZddZddZddZ dS) Cityscapesz#Base class for Cityscapes datasets.aC You have to download files from https://www.cityscapes-dataset.com/login/ (This dataset requires registration). For basic config (semantic_segmentation) you must download 'leftImg8bit_trainvaltest.zip' and 'gtFine_trainvaltest.zip'. Other configs do require additional files - please see code for more details. semantic_segmentationz)Cityscapes semantic segmentation dataset.FT)name descriptionrr rrsemantic_segmentation_extrazRCityscapes semantic segmentation dataset with train_extra split and coarse labels.stereo_disparityz3Cityscapes stereo image and disparity maps dataset.stereo_disparity_extrazJCityscapes stereo image and disparity maps dataset with train_extra split.cCsi}tj|d<tjjddd|d<|jjr!tjjddd|d<|jjr/tjjddd|d<|jjr=tjjddd|d <tjj |t tj |d t d S) Nimage_id)png)shapeencoding_format image_left image_right)r1r2segmentation_label disparity_mapz"https://www.cityscapes-dataset.com)builderr,featureshomepagecitation) tfdsr=TextImagebuilder_configrr rcore DatasetInfo _DESCRIPTION FeaturesDict _CITATION)rr=rrr"_infos0     zCityscapes._infoc Cs<i}|jjD]\}\}}tj|j|||<qtdd|Dr3d}|d|7}t ||jjD]\}\}}tj| |||||<q9t j j t jjdd|Ddt j j t jjdd|Ddg}|jjs|t j j t jjd d|Dd|S|t j j d d d|Dd|S) Ncss |] }tjj| VqdS)N)tfiogfileexists).0zrrr" sz/Cityscapes._split_generators..z@You must download the dataset files manually and place them in: z, cS*i|]\}}|ds|tj|dqS)/extratrainendswithospathjoinrNfeat_dirrWrrr" z0Cityscapes._split_generators..)r+ gen_kwargscSrQ)rRvalrTrYrrr"r[r\cSrQ)rRtestrTrYrrr"r[r\ train_extracSs2i|]\}}|dr|ddtj|dqS)rRr`)rUreplacerVrWrXrYrrr"r[s )rCritemsrVrWrX manual_diranyvaluesAssertionErrorextractr@rDSplitGeneratorSplitTRAIN VALIDATIONrappendTEST) r dl_managerpathssplitzip_file_msgrsplitsrrr"_split_generatorssJ  zCityscapes._split_generatorsc +s|d}tjj|D]pfdd|D}|d}tjj|D]W}tj||}t|}||j j vr8q$||d}|j j rOtj|dd ||d<|j j rdtj|dd ||j j|d <|j jrvtj|d d ||d <||fVq$q dS)Nr cs i|] \}}|tj|qSr)rVrWrXrYcity_idrr"r[sz1Cityscapes._generate_examples..)r0r7r z{}_rightImg8bit.pngr8r z {}_{}.pngr:rz{}_disparity.pngr;)rJrKrLlistdirrcrVrWrX_get_left_image_idrCrrformatr rr) rrpleft_imgs_rootpaths_city_rootleft_city_rootleft_img left_img_pathr0r=rrwr"_generate_examplessF   zCityscapes._generate_examplesN) r#r$r%r&MANUAL_DOWNLOAD_INSTRUCTIONSrBUILDER_CONFIGSrIrvrrrrr"r)sN # 5r)z'([a-z\-]+)_(\d+)_(\d+)_leftImg8bit\.pngcCst|}dj|S)aReturns the id of an image file. Used to associate an image file with its corresponding label. Example: 'bonn_000001_000019_leftImg8bit' -> 'bonn_000001_000019' Args: left_image: name of the image file. Returns: Id of the image (see example above). z{}_{}_{})LEFT_IMAGE_FILE_REmatchr{groups) left_imagerrrr"rz+s rz)r& __future__rrrrVretensorflow.compat.v2compatv2rJtensorflow_datasets.corertensorflow_datasets.public_api public_apir@rHrFrD BuilderConfigrGeneratorBasedBuilderr)compilerrzrrrr"s       ? '