o Ni('@sdZddlmZddlmZddlmZddlZddlZddlZddlZ ddl m m Z ddlmZdZdZgdZd d gZd Zd Zd ZGdddejjZGdddejjZddZddZddZddZ dS)z?BigEarthNet remote sensing dataset of Sentinel-2 image patches.)absolute_import)division)print_functionNa@article{Sumbul2019BigEarthNetAL, title={BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding}, author={Gencer Sumbul and Marcela Charfuelan and Beg{"u}m Demir and Volker Markl}, journal={CoRR}, year={2019}, volume={abs/1902.06148} }aThe BigEarthNet is a new large-scale Sentinel-2 benchmark archive, consisting of 590,326 Sentinel-2 image patches. The image patch size on the ground is 1.2 x 1.2 km with variable image size depending on the channel resolution. This is a multi-label dataset with 43 imbalanced labels. To construct the BigEarthNet, 125 Sentinel-2 tiles acquired between June 2017 and May 2018 over the 10 countries (Austria, Belgium, Finland, Ireland, Kosovo, Lithuania, Luxembourg, Portugal, Serbia, Switzerland) of Europe were initially selected. All the tiles were atmospherically corrected by the Sentinel-2 Level 2A product generation and formatting tool (sen2cor). Then, they were divided into 590,326 non-overlapping image patches. Each image patch was annotated by the multiple land-cover classes (i.e., multi-labels) that were provided from the CORINE Land Cover database of the year 2018 (CLC 2018). Bands and pixel resolution in meters: * B01: Coastal aerosol; 60m * B02: Blue; 10m * B03: Green; 10m * B04: Red; 10m * B05: Vegetation red edge; 20m * B06: Vegetation red edge; 20m * B07: Vegetation red edge; 20m * B08: NIR; 10m * B09: Water vapor; 60m * B11: SWIR; 20m * B12: SWIR; 20m * B8A: Narrow NIR; 20m License: Community Data License Agreement - Permissive, Version 1.0. URL: http://bigearth.net/ )+zAgro-forestry areasAirportsz,Annual crops associated with permanent cropsz Bare rockzBeaches, dunes, sandszBroad-leaved forestz Burnt areaszCoastal lagoonszComplex cultivation patternszConiferous forestzConstruction siteszContinuous urban fabriczDiscontinuous urban fabricz Dump sites Estuariesz!Fruit trees and berry plantationszGreen urban areaszIndustrial or commercial unitszInland marsheszIntertidal flatszVLand principally occupied by agriculture, with significant areas of natural vegetationzMineral extraction sitesz Mixed forestzMoors and heathlandzNatural grasslandzNon-irrigated arable landz Olive grovesPasturesPeatbogszPermanently irrigated landz Port areasz Rice fieldsz*Road and rail networks and associated landSalinesz Salt marsheszSclerophyllous vegetationz Sea and oceanzSparsely vegetated areaszSport and leisure facilitieszTransitional woodland/shrub Vineyardsz Water bodiesz Water coursesrgballz5http://bigearth.net/downloads/BigEarthNet-v1.0.tar.gzzBigEarthNet-v1.0g@@cs"eZdZdZdfdd ZZS)BigearthnetConfigzBuilderConfig for Bigearthnet.Nc sF|tvr tdttjdd}tt|jdd|i|||_dS)zConstructs a BigearthnetConfig. Args: selection: `str`, one of `_DATA_OPTIONS`. **kwargs: keyword arguments forwarded to super. zselection must be one of %sz1.0.0z6New split API (https://tensorflow.org/datasets/splits)versionN) _DATA_OPTIONS ValueErrortfdscoreVersionsuperr __init__ selection)selfrkwargsv100 __class__rh/home/ubuntu/.local/lib/python3.10/site-packages/tensorflow_datasets/image_classification/bigearthnet.pyros  zBigearthnetConfig.__init__)N)__name__ __module__ __qualname____doc__r __classcell__rrrrr lsr c@sDeZdZdZeddddeddddgZddZd d Zd d Zd S) Bigearthnetz?Bigearthnet remote sensing dataset of Sentinel-2 image patches.r zSentinel-2 RGB channels)rname descriptionr z13 Sentinel-2 channelscCstjtjtjtjtjtjdtjtjd}|jjdkrDtjtjjgddtj tjj t dtj|d}d}n|jjd krtjtjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tjj d d gtj d tj tjj t dtj|d}d}tjj|t||dtdS)N)lrxlryulxuly)acquisition_date coordinates projection tile_sourcer )xr.)shape)names)imagelabelsfilenamemetadata)r2r3r )r0dtyper.<)B01B02B03B04B05B06B07B08B09B11B12B8Ar3r4r5zhttp://bigearth.net)builderr%featuressupervised_keyshomepagecitation)rrF FeaturesDictTexttfint64builder_configrImageSequence ClassLabel_LABELSTensorfloat32r DatasetInfo _DESCRIPTION _CITATION)r metadata_dictrFrGrrr_infosZ   zBigearthnet._infocCs$|t}tjjtjjd|idgS)zReturns SplitGenerators. archive_path)r$ gen_kwargs)download _ZIP_FILErrSplitGeneratorSplitTRAIN)r dl_managerdl_pathrrr_split_generatorss zBigearthnet._split_generatorscCsTtjjj}|jj}|d||g?Bd|t|?Bd|j ?Bd| t |?BS)zGenerates examples as dicts. ArchivePath ReadArchive ReshuffleProcessExamples) rr lazy_imports apache_beamrNrCreateFlatMap _read_archive transformsrfMap_process_example)rpipelinerZbeamrrrr_build_pcollections  zBigearthnet._build_pcollectionN) rrr r!r BUILDER_CONFIGSrYrcrrrrrrr#s  D r#c csi}d}tjjj|D]Z\}}|d7}|tjj\}}}| dr+| |d<n-| drR|dd} |dksB|dkrQ| d vrQ| || <| d g | nt d ||d krg||d <|Vi}d}q dS)z-Yields non-processed examples out of archive.rz_labels_metadata.jsonr5z.tifir >r:r;r<bandszUnexpected file: %s r4N)rrr\ extractoriter_tar_streamsplitospathsependswithread setdefaultappendAssertionError) rZrexampleread_band_filesfpathfobj_ patch_namefnamebandrrrrls2       rlcCsl|}t|d|d<|dd|d<|dpgD] }t||||<q|dkr0t||d|fS)Nr5r3rvr r4)copyjsonloadspop _load_tif_create_rgb_image)rrrrrrros rocCsPtj|d|d|dgdd}|td}t|ddtj|d <d S) zCreates and rescales RGB image.r<r;r:)axisgo@rr2N)npstackr_OPTICAL_MAX_VALUEclipastypeuint8)dimgrrrrs& rcCs8tjjjt|}t|  |j  tj }|S)z2Loads TIF file and returns as float32 numpy array.)rrrh PIL_ImageopenioBytesIOrarraygetdatareshapesizerrT)datarrrrrsr)!r! __future__rrrrrr{numpyrtensorflow.compat.v2compatv2rLtensorflow_datasets.public_api public_apirrWrVrRrr] _ZIP_SUBIDRrr BuilderConfigr BeamBasedBuilderr#rlrorrrrrrs.     $j