o Ni@sdZddlmZddlmZddlmZddlZddlZddlZddl m m Z ddl mZdZeedfZdZGd d d ejjZGd d d eZGd ddedgdZdddZdS)zCIFAR datasets.)absolute_import)division)print_functionN z@TECHREPORT{Krizhevsky09learningmultiple, author = {Alex Krizhevsky}, title = {Learning multiple layers of features from tiny images}, institution = {}, year = {2009} } c@s@eZdZdZejdZddZe ddZ ddZ d d Z d S) Cifar10z CIFAR-10.3.0.2c CsBtjj|dtjtjtjjtdtjjdddddt dS) NzThe CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.shape  num_classes)idimagelabelrr+https://www.cs.toronto.edu/~kriz/cifar.htmlbuilder descriptionfeaturessupervised_keyshomepagecitation tfdscore DatasetInfor FeaturesDictTextImage_CIFAR_IMAGE_SHAPE ClassLabel _CITATIONselfr&b/home/ubuntu/.local/lib/python3.10/site-packages/tensorflow_datasets/image_classification/cifar.py_info2s  z Cifar10._infoc Cs"t|jdgddgddgdgdS)Nz7https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz)zdata_batch_1.binzdata_batch_2.binzdata_batch_3.binzdata_batch_4.binzdata_batch_5.binztest_batch.binzcifar-10-batches-bin/zbatches.meta.txtrnameurl train_files test_filesprefix label_files label_keys CifarInfor*r$r&r&r' _cifar_infoBszCifar10._cifar_infoc s||jj|j}tj|jt|j|j D]5\}}tj|}t j j |}dd|dD}Wdn1sBwY||jj|_qfdd}tjjtjjd||jdd tjjtjjd ||jdd gS) zReturns SplitGenerators.cSsg|]}|r|qSr&r&).0r*r&r&r' ]sz-Cifar10._split_generators.. Nc3s |D] }tj|VqdS)N)ospathjoin) filenamesf cifar_pathr&r' gen_filenamesasz0Cifar10._split_generators..gen_filenamestrain_) split_prefix filepaths)r* gen_kwargstest_)download_and_extractr3r+r7r8r9r.zipr0r/tfiogfileGFilereadsplitinfornamesrrSplitGeneratorSplitTRAINr,TESTr-) r% dl_manager cifar_info label_key label_file labels_pathlabel_f label_namesr>r&r<r'_split_generatorsQs2  zCifar10._split_generatorsc csl|jj}d}|D]*}t|t|D] \}}tt||}d|||d<||d<||fV|d7}qq dS)aeGenerate CIFAR examples as dicts. Shared across CIFAR-{10, 100}. Uses self._cifar_info as configuration. Args: split_prefix (str): Prefix that identifies the split (e.g. "tr" or "te"). filepaths (list[str]): The files to use to generate the data. Yields: The cifar examples, as defined in the dataset info features. rz{}{:05d}rrN)r3r0 _load_datalendictrEformat) r%r@rAr0indexr8labelsnp_imagerecordr&r&r'_generate_examplests   zCifar10._generate_examplesN) __name__ __module__ __qualname____doc__rrVersionVERSIONr(propertyr3rYrcr&r&r&r'r-s   #rc@s0eZdZdZejdZeddZ ddZ dS)Cifar100zCIFAR-100 dataset.rc Cs$t|jddgdgdddgddgd S) Nz8https://www.cs.toronto.edu/~kriz/cifar-100-binary.tar.gzz train.binztest.binzcifar-100-binary/zcoarse_label_names.txtzfine_label_names.txt coarse_labelrr)r1r$r&r&r'r3szCifar100._cifar_infoc CsNtjj|dtjtjtjjtdtjjddtjjdddddt d S) Na`This dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each. There are 500 training images and 100 testing images per class. The 100 classes in the CIFAR-100 are grouped into 20 superclasses. Each image comes with a "fine" label (the class to which it belongs) and a "coarse" label (the superclass to which it belongs).r dr )rrrrlrrrrr$r&r&r'r(s   zCifar100._infoN) rdrerfrgrrrhrirjr3r(r&r&r&r'rks    rkc@seZdZdZdS)r2a Contains the information necessary to generate a CIFAR dataset. Attributes: name (str): name of dataset. url (str): data URL. prefix (str): path prefix within the downloaded and extracted file to look for `train_files` and `test_files`. train_files (list): name of training files within `prefix`. test_files (list): name of test files within `prefix`. label_files (list): names of the label files in the data. label_keys (list): names of the label keys in the data. N)rdrerfrgr&r&r&r'r2s r2 _CifarInfo)r*r+r.r,r-r/r0rZccstjj|d }|}Wdn1swYd}t|d}||kr]tj|tj||d |f}||7}tj|tjd|d dt t f d}|d7}||fV||ks)dSdS) z!Yields (labels, np_image) tuples.rbNrrZ)dtypecountoffseti r)rZr) rFrGrHrIrJr\np frombufferuint8reshape_CIFAR_IMAGE_SIZE transpose)r8 labels_numberr;datars max_offsetr`imgr&r&r'r[s(     r[)rZ)rg __future__rrr collectionsr7numpyrutensorflow.compat.v2compatv2rFtensorflow_datasets.public_api public_apirryr!r#rGeneratorBasedBuilderrrk namedtupler2r[r&r&r&r's       b'