o bi/@snddlZddlZddlmZGdddZeZedddeDZe dd Z e e_ d d Z e e_ dS) N)get_protocols_providerc@sfeZdZdZeddZeddZeddZedd Zed d Z ed e d e de fddZ dS)ProtocolsProviderz~Note that these must be preinstalled on all nodes in the Ray cluster; it is not sufficient to install them in the runtime_env.cChdS)N> gss3uvgcspipfileazurecondahttpsclsrrU/home/ubuntu/.local/lib/python3.10/site-packages/ray/_private/runtime_env/protocol.py get_protocolszProtocolsProvider.get_protocolscCr)N>rrr r r rrrrrget_remote_protocols%rz&ProtocolsProvider.get_remote_protocolscCsLz ddl}ddlm}Wntytd|jwd|di}||fS)zSet up S3 protocol handling. Returns: tuple: (open_file function, transport_params) Raises: ImportError: If required dependencies are not installed. rNopenzBYou must `pip install smart_open[s3]` to fetch URIs in s3 bucket. clientr)boto3 smart_openr ImportError_MISSING_DEPENDENCIES_WARNINGr)rr open_filetransport_paramsrrr_handle_s3_protocol)s  z%ProtocolsProvider._handle_s3_protocolcCs@zddlm}ddlm}W|dfStytd|jw)zSet up Google Cloud Storage protocol handling. Returns: tuple: (open_file function, transport_params) Raises: ImportError: If required dependencies are not installed. r)storagerzTYou must `pip install smart_open[gcs]` to fetch URIs in Google Cloud Storage bucket.N) google.cloudrrrrr)rrrrrr_handle_gs_protocol?s  z%ProtocolsProvider._handle_gs_protocolcCszddlm}ddlm}ddlm}Wnty"td|jwt d}|s.t dd|d }d |||d i}||fS) aSet up Azure blob storage protocol handling. Returns: tuple: (open_file function, transport_params) Raises: ImportError: If required dependencies are not installed. ValueError: If required environment variables are not set. r)DefaultAzureCredential)BlobServiceClientrzpYou must `pip install azure-storage-blob azure-identity smart_open[azure]` to fetch URIs in Azure Blob Storage. AZURE_STORAGE_ACCOUNTz`Azure Blob Storage authentication requires AZURE_STORAGE_ACCOUNT environment variable to be set.zhttps://z.blob.core.windows.net/r) account_url credential) azure.identityr"azure.storage.blobr#rrrrosgetenv ValueError)rr"r#razure_storage_account_namer%rrrr_handle_azure_protocolUs,     z(ProtocolsProvider._handle_azure_protocolprotocol source_uri dest_filec Cs8||vsJd}d}|dkr |tdd}dddd}n>|dkr+|\}}n3|dkr6|\}}n(|d krA|\}}nzd d lm}Wnty]td |d |j w||d|d-}||d}| | Wdn1s}wYWddSWddS1swYdS)axDownload file from remote URI to destination file. Args: protocol: The protocol to use for downloading (e.g., 's3', 'https'). source_uri: The source URI to download from. dest_file: The destination file path to save to. Raises: ImportError: If required dependencies for the protocol are not installed. Nr zfile://)rcSs t||SNr)urimoderrrrrs z8ProtocolsProvider.download_remote_uri..open_filerrr rrz+You must `pip install smart_open` to fetch z URIs. rbwb) rlenrr!r-rrrupperrwriteread)rr.r/r0tprfinfoutrrrdownload_remote_uri}s<   "z%ProtocolsProvider.download_remote_uriN) __name__ __module__ __qualname__r classmethodrrrr!r-strr=rrrrrs     'rProtocolcCsi|]}||qSrr7.0r.rrr srGcsfddtDS)Ncsg|]}|qSrrDrErrr s z%_remote_protocols..)_protocols_providerrrrrr_remote_protocolss rJcCst|j||Sr1)rIr=value)selfr/r0rrr_download_remote_urisrM)enumr)%ray._private.runtime_env.default_implrrrIEnumrrCrArJremote_protocolsrMr=rrrrs #