o oi@sdZddlmZdZdZeddedDZddlZdd l m Z m Z m Z m Z mZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZdd lmZddl Z!ddl"Z#ddl$Z$ddl%Z&ddl'mZ(ddl)Z*ddl+Z*e,e*j-d se*j-e*j-_.gd Z/ej0re(j1Z2e(j3Z4e&j5eefZ6e&j7eZ8ne9d Z2e9dZ4e9dZ6e9dZ8dZ:e9dZ;ejfZ?GdddZ@e@ZAdddZBdddZCdd!d"ZDdd%d&ZE ddd,d-ZFdd0d1ZGdd4d5ZHdd7d8ZIdd:d;ZJdd=d>ZKddAdBZLddCdDZMddFdGZNddIdJZOddLdMZPe&jQfddOdPZRddQdRZS Sde&jQdTddUdVZT W Xde&jQdTddZd[ZUe&jVgd\gd]gd^ge&jQdTZWe&jXYeWZZe&jVgd_e&jQdTZ[ddadbZ\ddddeZ]e&jVgdfgdggdhge&jQdT^Z_e&jXYe_Z`e&jVgdie&jQdTZaddjdkZbddmdnZcdddrdsZdddtduZeddxdyZfdd{d|Zgdd~dZhejidddZjejidddZkGdddZlddddddZm dd ddZnddddd ddZoddd ddZp d d ddZqddddddddZrdddZsdddddZt dddddddddddddd dddZudddZvdddZwddd„ZxGddĄdejyZzdddƄZ{GddȄdȃZ|Gddʄde|Z}Gdd̄de|Z~Gdd΄de(jeZdddЄZddd҄ZddԜdddքZddd؄Zdddddٜddd݄Zddddd߄Z ddddddddddddd dddZdS(a+ `mediapy`: Read/write/show images and videos in an IPython/Jupyter notebook. [**[GitHub source]**](https://github.com/google/mediapy)   [**[API docs]**](https://google.github.io/mediapy/)   [**[PyPI package]**](https://pypi.org/project/mediapy/)   [**[Colab example]**](https://colab.research.google.com/github/google/mediapy/blob/main/mediapy_examples.ipynb) See the [example notebook](https://github.com/google/mediapy/blob/main/mediapy_examples.ipynb), or better yet, [**open it in Colab**](https://colab.research.google.com/github/google/mediapy/blob/main/mediapy_examples.ipynb). ## Image examples Display an image (2D or 3D `numpy` array): ```python checkerboard = np.kron([[0, 1] * 16, [1, 0] * 16] * 16, np.ones((4, 4))) show_image(checkerboard) ``` Read and display an image (either local or from the Web): ```python IMAGE = 'https://github.com/hhoppe/data/raw/main/image.png' show_image(read_image(IMAGE)) ``` Read and display an image from a local file: ```python !wget -q -O /tmp/burano.png {IMAGE} show_image(read_image('/tmp/burano.png')) ``` Show titled images side-by-side: ```python images = { 'original': checkerboard, 'darkened': checkerboard * 0.7, 'random': np.random.rand(32, 32, 3), } show_images(images, vmin=0.0, vmax=1.0, border=True, height=64) ``` ## Video examples Display a video (an iterable of images, e.g., a 3D or 4D array): ```python video = moving_circle((100, 100), num_images=10) show_video(video, fps=10) ``` Show the video frames side-by-side: ```python show_images(video, columns=6, border=True, height=64) ``` Show the frames with their indices: ```python show_images({f'{i}': image for i, image in enumerate(video)}, width=32) ``` Read and display a video (either local or from the Web): ```python VIDEO = 'https://github.com/hhoppe/data/raw/main/video.mp4' show_video(read_video(VIDEO)) ``` Create and display a looping two-frame GIF video: ```python image1 = resize_image(np.random.rand(10, 10, 3), (50, 50)) show_video([image1, image1 * 0.8], fps=2, codec='gif') ``` Darken a video frame-by-frame: ```python output_path = '/tmp/out.mp4' with VideoReader(VIDEO) as r: darken_image = lambda image: to_float01(image) * 0.5 with VideoWriter(output_path, shape=r.shape, fps=r.fps, bps=r.bps) as w: for image in r: w.add_image(darken_image(image)) ``` ) annotationsgooglez1.1.6ccs|]}t|VqdSN)int).0numrD/home/ubuntu/.local/lib/python3.10/site-packages/mediapy/__init__.py gr .N)CallableIterableIteratorMappingSequence)Any Resampling) show_image show_images show_video show_videos read_image write_image read_video write_video VideoReader VideoWriter VideoMetadatacompress_imagedecompress_imagecompress_videodecompress_videohtml_from_compressed_imagehtml_from_compressed_video resize_image resize_videoto_rgbto_type to_float01to_uint8set_output_heightset_max_output_height color_ramp moving_circleset_show_save_dir set_ffmpegvideo_is_available _ArrayLike _DTypeLike_NDArray_DTypei-1_Tc@s&eZdZUdZded<dZded<dS)_Configffmpeg_Pathffmpeg_name_or_pathN _Path | None show_save_dir)__name__ __module__ __qualname__r:__annotations__r<rrrr r7s r7pathr9argsrkwargsreturncOst|g|Ri|S)z9Opens the file; this is a hook for the built-in `open()`.)open)rArBrCrrr _opensrFboolcCs~dS)zEReturns True if the path is in the filesystem accessible by `ffmpeg`.TrrArrr _path_is_localsrI str | NonecCsttj}r t|SdS)z;Returns a path to the ffmpeg program, or None if not found.N)shutilwhich_configr:str)filenamerrr _search_for_ffmpeg_pathsrPrNNonecOs(ittjdd|}t|i|dS)z'Prints arguments to stderr immediately.T)fileflushN)dictsysstderrprint)rBrCrrr _print_errsrXiterable Iterable[_T]n int | NoneIterator[tuple[_T, ...]]cCs"d dd}tt||t|d S) zGReturns elements collected as tuples of length at most `n` if not None.r[rrYrZrDtuple[_T, ...]cSstt||Sr)tuple itertoolsislice)r[rYrrr takez_chunked..takerN)r[rrYrZrDr^)iter functoolspartial)rYr[rbrrr _chunkeds rgiteratortuple[_T, Iterable[_T]]cCst|\}}t|}||fS)aGiven an iterator, returns first element and re-initialized iterator. >>> first_image, images = _peek_first(moving_circle()) Args: iterator: An input iterator or iterable. Returns: A tuple (first_element, iterator_reinitialized) containing: first_element: The first element of the input. iterator_reinitialized: A clone of the original iterator/iterable. )r`teenext)rhpeekeriterator_reinitializedfirstrrr _peek_firstsroshapetuple[int, int]cCsBt|dkrtd|dtdd|Dstd|ddS)zEChecks that `shape` is of the form (height, width) with two integers.zShape z$ is not of the form (height, width).css|] }t|tjVqdSr) isinstancenumbersIntegralrirrr r z"_check_2d_shape..z contains non-integers.N)len ValueErrorallrprrr _check_2d_shapes r}str | Sequence[str]cCsTtj|t|ttjtjddd}t|jddd|jr(t d|j d|jdd S) aGExecutes command, printing output from stdout and stderr. Args: args: Command to execute, which can be either a string or a sequence of word strings, as in `subprocess.run()`. If `args` is a string, the shell is invoked to interpret it. Raises: RuntimeError: If the command's exit code is nonzero. FT)shellstdoutrVcheckuniversal_newlines)endrSz Command 'z' failed with code r N) subprocessrunrsrNPIPESTDOUTrWr returncode RuntimeErrorrB)rBprocrrr _runs rtextcCstjtj|dS)z/In a Jupyter notebook, display the HTML `text`.N)IPythondisplayHTML)rrrr _display_htmlsr name_or_pathcCs |t_dS)aoSpecifies the name or path for the `ffmpeg` external program. The `ffmpeg` program is required for compressing and decompressing video. (It is used in `read_video`, `write_video`, `show_video`, `show_videos`, etc.) Args: name_or_path: Either a filename within a directory of `os.environ['PATH']` or a filepath. The default setting is 'ffmpeg'. N)rMr:)rrrr r0"s r0 num_pixelsrc C@ztd}d|d}||WdSttfyYdSw)z@Overrides the height of the current output cell, if using Colab.google.colab.outputz%google.colab.output.setIframeHeight("zpx")N importlib import_moduleeval_jsModuleNotFoundErrorAttributeErrorroutputsrrr r+0s  r+c Cr)zCSets the maximum height of the current output cell, if using Colab.rz9google.colab.output.setIframeHeight(0, true, {maxHeight: z})Nrrrrr r,;s r,dtypecCs2t|}t|jtjtjfstd|d|S)z"Returns validated media data type.Type z0 is not a valid media data type (uint or float).)npr issubclasstypeunsignedintegerfloatingrzrrrr _as_valid_media_typeLs  rxcCs<t|}|jtkr|tjttjj}t|j|S)zGConverts to ndarray (if not already), and checks validity of data type.) rasarrayrrGastypeuint8iinfomaxr)rarrr _as_valid_media_arrayVs   rarrayc Cst|}t|}~|jtkrt|j|jtkr3||}t|tjr1||t |j }|S|j|kr<|}|St|tjrt|jtjrSt |jj }n t |dd}d}t |j }|t tj j krtj ||tjd}||d|}|S|t tjj kr|tj||d|}|S|tj||d}t|}|t|k}||}|||<|jdkr|n|d}|St|tjsJ||}t|jtjr||t |jj }|S)aGReturns media array converted to specified type. A "media array" is one in which the dtype is either a floating-point type (np.float32 or np.float64) or an unsigned integer type. The array values are assumed to lie in the range [0.0, 1.0] for floating-point values, and in the full range for unsigned integers, e.g. [0, 255] for np.uint8. Conversion between integers and floats maps uint(0) to 0.0 and uint(MAX) to 1.0. The input array may also be of type bool, whereby True maps to uint(MAX) or 1.0. The values are scaled and clamped as appropriate during type conversions. Args: array: Input array-like object (floating-point, unsigned int, or bool). dtype: Desired output type (floating-point or unsigned int). Returns: Array `a` if it is already of the specified dtype, else a converted array. ?r?r)rrrrGrr issubdtyperrrrclipuint16rfloat32uint32float64 atleast_1dndimr) rrrresultsrc_maxdst_maxscaledstvalues_too_largerrr r(_sL           r(rcCsPt|}t|}t|tjstd|dt|jtjr#|St||S)aIf array has unsigned integers, rescales them to the range [0.0, 1.0]. Scaling is such that uint(0) maps to 0.0 and uint(MAX) maps to 1.0. See `to_type`. Args: a: Input array. dtype: Desired floating-point type if rescaling occurs. Returns: A new array of dtype values in the range [0.0, 1.0] if the input array `a` contains unsigned integers; otherwise, array `a` is returned unchanged. rz is not floating-point.)rrrrrrzr()rrrrr r)s   r)cCs t|tjS)z7Returns array converted to uint8 values; see `to_type`.)r(rr)rrrr r*s r*@rrcCsHt|t|}tt|ddd|}tj|dddd}t||S)a/Returns an image of a red-green color gradient. This is useful for quick experimentation and testing. See also `moving_circle` to generate a sample video. Args: shape: 2D spatial dimensions (height, width) of generated image. dtype: Type (uint or floating) of resulting pixel values. rrrrraxis)r}rrmoveaxisindicesinsertr()rpryximagerrr r-s   r-r num_imagescs@ttd fdd tfddtDS) aReturns a video of a circle moving in front of a color ramp. This is useful for quick experimentation and testing. See also `color_ramp` to generate a sample image. >>> show_video(moving_circle((480, 640), 60), fps=60) Args: shape: 2D spatial dimensions (height, width) of generated video. num_images: Number of video frames. dtype: Type (uint or floating) of resulting pixel values. image_indexrrDr4cstd}ttdd}ddd|df}tdd}tj||ddd |k}d }ttjrFt|gd}|||<|S) zReturns a video frame image.rrrg333333?rg?rrr)rrr) r-rrrminsumrrr()rrrcenterradius_squaredinsidewhite_circle_color)rrrprr generate_images  z%moving_circle..generate_imagecg|]}|qSrrrv)rrr z!moving_circle..N)rrrDr4)r}rrrrange)rprrr)rrrrpr r.s  r.)gA`"?gxÅ¿g>?)gbX9?gx|ҿgb!z)gv/?g}?gE)rrrrgbcCs4t|}|jddkrtd|jd|ttS)zReturns the RGB image/video mapped to YUV [0,1] color space. Note that the "YUV" color space used by video compressors is actually YCbCr! Args: rgb: Input image in sRGB space. rThe last dimension in is not 3.)r)rprz_YUV_FROM_RGB_MATRIX_YUV_CHROMA_OFFSETrrrr yuv_from_rgbs ryuvcCs4t|}|jddkrtd|jd|ttS)zReturns the YCbCr image/video mapped to RGB [0,1] color space.rrrrr)r)rprzr_RGB_FROM_YCBCR_MATRIX)rrrr rgb_from_ycbcr9srrmodePIL.Image.ImagecCs4t|}|jdvrtd|jdtjj||dS)zGReturns a PIL image given a numpy matrix (either uint8 or float [0,1]).rrr Image shape  is neither 2D nor 3D.r)rrrzrpPILImage fromarray)rrrrr _pil_imageDs rcst|}|jdvrtd|jdtt|jtjp#|jtj ko(|jdk}|jtj ko:|jdko:|jddv}|s?|rVtj t |j dddt jjjd |jd S|jdkrfttt||jStfd d t|dd DS)a+Resizes image to specified spatial dimensions using a Lanczos filter. Args: image: Array-like 2D or 3D object, where dtype is uint or floating-point. shape: 2D spatial dimensions (height, width) of output image. Returns: A resampled image whose spatial dimensions match `shape`. rrrrrr)rNr)resamplercg|]}t|qSrr%)rchannelr|rr roz resize_image..r)rrrzrpr}rrrrrrrresizerrrLANCZOSr(r%r)dstackr)rrpsupported_single_channelsupported_multichannelrr|r r%Ls, " r%videoIterable[_NDArray]cs ttfdd|DS)zResizes `video` to specified spatial dimensions using a Lanczos filter. Args: video: Iterable of images. shape: 2D spatial dimensions (height, width) of output video. Returns: A resampled video whose spatial dimensions match `shape`. crrrrrr|rr rrz resize_video..)r}rr)rrprr|r r&vs r& path_or_urlcCst|to |dS)N)zhttp://zhttps://zfile://)rsrN startswith)r rrr _is_urlsr bytescCst|r(t|ts Jtj| }|}Wd|S1s!wY|St|d }|}Wd|S1s=wY|S)zCReturns the contents of the file specified by either a path or URL.Nrb)r rsrNurllibrequesturlopenreadrF)r responsedatafrrr read_contentss    r Iterator[str]ccst|s t|swYWddSWddS1sVwYdS)zContext to write a temporary local file and subsequently copy it remotely. Args: path: File, which may be remote. Yields: The name of a local file which may be subsequently copied remotely. rRwbrN) rIrNrrrrrrFwrite read_bytes)rArrrrrrr _write_via_local_files    "r#c@s.eZdZdZdddZddd Zdd d ZdS)r/aSave all titled output from `show_*()` calls into files. If the specified `directory` is not None, all titled images and videos displayed by `show_image`, `show_images`, `show_video`, and `show_videos` are also saved as files within the directory. It can be used either to set the state or as a context manager: >>> set_show_save_dir('/tmp') >>> show_image(color_ramp(), title='image1') # Creates /tmp/image1.png. >>> show_video(moving_circle(), title='video2') # Creates /tmp/video2.mp4. >>> set_show_save_dir(None) >>> with set_show_save_dir('/tmp'): ... show_image(color_ramp(), title='image1') # Creates /tmp/image1.png. ... show_video(moving_circle(), title='video2') # Creates /tmp/video2.mp4. directoryr;cCstj|_|t_dSr)rMr<_old_show_save_dir)selfr$rrr __init__s zset_show_save_dir.__init__rDrQcCsdSrrr&rrr __enter__szset_show_save_dir.__enter___rcGs |jt_dSr)r%rMr<r&r*rrr __exit__ zset_show_save_dir.__exit__N)r$r;rDrQr*rrDrQ)r=r>r?__doc__r'r)r,rrrr r/s   r/T)apply_exif_transposerr1cCst|}t|||S)aReturns an image read from a file path or URL. Decoding is performed using `PIL`, which supports `uint8` images with 1, 3, or 4 channels and `uint16` images with a single channel. Args: path_or_url: Path of input file. apply_exif_transpose: If True, rotate image according to EXIF orientation. dtype: Data type of the returned array. If None, `np.uint8` or `np.uint16` is inferred automatically. )rr )r r1rrrrr rs rpngfmtcKsjt|}t|jtjrt|}t|d}t|j|fd|i|WddS1s.wYdS)aYWrites an image to a file. Encoding is performed using `PIL`, which supports `uint8` images with 1, 3, or 4 channels and `uint16` images with a single channel. File format is explicitly provided by `fmt` and not inferred by `path`. Args: path: Path of output file. image: Array-like object. If its type is float, it is converted to np.uint8 using `to_uint8` (thus clamping to the input to the range [0.0, 1.0]). Otherwise it must be np.uint8 or np.uint16. fmt: Desired compression encoding, e.g. 'png'. **kwargs: Additional parameters for `PIL.Image.save()`. r formatN) rrrrrr*rFrsave)rArr3rCrrrr rs  "rgrayvminvmaxcmapr8 float | Noner9r:&str | Callable[[_ArrayLike], _NDArray]cCst|}~|durttt||tjn|}|dur,ttt||tj n|}|d|||tt j }t |t rWt tdrOtj|}n tjj|}n|}||}|jddkrwtt|ddkrw|ddd f}|S) a7Maps scalar values to RGB using value bounds and a color map. Args: array: Scalar values, with arbitrary shape. vmin: Explicit min value for remapping; if None, it is obtained as the minimum finite value of `array`. vmax: Explicit max value for remapping; if None, it is obtained as the maximum finite value of `array`. cmap: A `pyplot` color map or callable, to map from 1D value to 3D or 4D color. Returns: A new array in which each element is affinely mapped from [vmin, vmax] to [0.0, 1.0] and then color-mapped. Nfloat colormapsrr).rr.r)rraminwhereisfiniteinfamaxrfinfor=epsrsrNhasattr matplotlibr>pyplotcmget_cmaprpr{r))rr8r9r:rrgb_from_scalarrrr r's(*"   $r')r3cKsVt|}t}t|j|fd|i||WdS1s$wYdS)aReturns a buffer containing a compressed image. Args: image: Array in a format supported by `PIL`, e.g. np.uint8 or np.uint16. fmt: Desired compression encoding, e.g. 'png'. **kwargs: Options for `PIL.save()`, e.g. `optimize=True` for greater compression. r4N)rioBytesIOrr5getvalue)rr3rCrrrr r>s  $rrcCsNtjt|}|rtj|}|dur |jdkrtj ntj }tj ||dS)aReturns an image from a compressed data buffer. Decoding is performed using `PIL`, which supports `uint8` images with 1, 3, or 4 channels and `uint16` images with a single channel. Args: data: Buffer containing compressed image. dtype: Data type of the returned array. If None, `np.uint8` or `np.uint16` is inferred automatically. apply_exif_transpose: If True, rotate image according to EXIF orientation. NIr) rrrErLrMImageOpsexif_transposerrrrr)rrr1 pil_imagerrr r Os  r F)titleborder pixelatedr3widthheightrSrT bool | strrUc Cst|d}t|tr|d}n|rd}nd}|rdnd}d|d|d |d |d |d |d } |r@d|d| d} | S)aReturns an HTML string with an image tag containing encoded data. Args: data: Compressed image bytes. width: Width of HTML image in pixels. height: Height of HTML image in pixels. title: Optional text shown centered above image. border: If `bool`, whether to place a black boundary around the image, or if `str`, the boundary CSS style. pixelated: If True, sets the CSS style to 'image-rendering: pixelated;'. fmt: Compression encoding. utf-8; border:1px solid black; rrUautoz

base64 b64encodedecodersrN) rrVrWrSrTrUr3b64 s_pixelatedrrrr r#es0   r#cCst|dks J||r|r||fS|r&|s&|t||d|ddfS|r:|s:t||d|dd|fS|dddS)zBReturns (width, height) given optional parameters and image shape.rrrrrNr)ryr)rVrWrprrr _get_width_heights  rh)rScKstt|g|gfi|S)aDisplays an image in the notebook and optionally saves it to a file. See `show_images`. >>> show_image(np.random.rand(100, 100)) >>> show_image(np.random.randint(0, 256, size=(80, 80, 3), dtype='uint8')) >>> show_image(np.random.rand(10, 10) - 0.5, cmap='bwr', height=100) >>> show_image(read_image('/tmp/image.png')) >>> url = 'https://github.com/hhoppe/data/raw/main/image.png' >>> show_image(read_image(url)) Args: image: 2D array-like, or 3D array-like with 1, 3, or 4 channels. title: Optional text shown centered above the image. **kwargs: See `show_images`. Returns: html string if `return_html` is `True`. )rrr)rrSrCrrr rsrrr) rVrW downsamplecolumnsr8r9r:rTylabel html_classrU return_htmlimages/Iterable[_ArrayLike] | Mapping[str, _ArrayLike]titlesIterable[str | None] | Nonerirjrkrl bool | Nonermc  st|tr|dur tdt|t|n*t||dur*dgtnt|ttkrEtdtdtdd fd d fd d Dd fdd |rjfdd Ddd D t D]/\}}|rtj rt tj |d}t |dd }| |Wdn1swYqvd f dd }|}t|tdkrdd Ddd D |}t|tdks| r|St|dS)aDisplays a row of images in the IPython/Jupyter notebook. If a directory has been specified using `set_show_save_dir`, also saves each titled image to a file in that directory based on its title. >>> image1, image2 = np.random.rand(64, 64, 3), color_ramp((64, 64)) >>> show_images([image1, image2]) >>> show_images({'random image': image1, 'color ramp': image2}, height=128) >>> show_images([image1, image2] * 5, columns=4, border=True) Args: images: Iterable of images, or dictionary of `{title: image}`. Each image must be either a 2D array or a 3D array with 1, 3, or 4 channels. titles: Optional strings shown above the corresponding images. width: Optional, overrides displayed width (in pixels). height: Optional, overrides displayed height (in pixels). downsample: If True, each image whose width or height is greater than the specified `width` or `height` is resampled to the display resolution. This improves antialiasing and reduces the size of the notebook. columns: Optional, maximum number of images per row. vmin: For single-channel image, explicit min value for display. vmax: For single-channel image, explicit max value for display. cmap: For single-channel image, `pyplot` color map or callable to map 1D to 3D color. border: If `bool`, whether to place a black boundary around the image, or if `str`, the boundary CSS style. ylabel: Text (rotated by 90 degrees) shown on the left of each row. html_class: CSS class name used in definition of HTML element. pixelated: If True, sets the CSS style to 'image-rendering: pixelated;'; if False, sets 'image-rendering: auto'; if None, uses pixelated rendering only on images for which `width` or `height` introduces magnification. return_html: If `True` return the raw HTML `str` instead of displaying. Returns: html string if `return_html` is `True`. Nz3Cannot have images dictionary and titles parameter.z2Number of images does not match number of titles ( vs ).rr2rDr4cst|}|jdks|jdkr|jddvstd|jd|jdkr5|jddkr5|dddddf}|jdkrBt|d}|S) Nrrr)rrrzImage with shape z? is neither a 2D array nor a 3D array with 1, 3, or 4 channels.rrr7)rrrprzr')r)r:r9r8rr ensure_mapped_to_rgbs"  z)show_images..ensure_mapped_to_rgbcrrrr)rurr rrzshow_images..csH|jdd}t|\}}||dks||dkr"t|||f}|S)Nrrrr)rprhr%)rrpwh)rWrVrr maybe_downsample s z%show_images..maybe_downsamplecrrrr)rxrr rrcSg|]}tt|qSrrr*rrrr rz.pngr rrNc sg}tD]5\}}}t |jdd\}}||jdkp'||jdk}dur.n|}|t|||||dqg}t|D]-} ddfdd| D} rfd } d | d d | } |d d| dqEd|S)Nrrrr)rSrTrUrc3|] }|dVqdSzNrretdrr r )zCshow_images..html_from_compressed_images..3writing-mode:vertical-lr; transform:rotate(180deg); 
)ziprhrpappendr#rgjoin) html_stringsrrSpng_datarvrw magnified pixelated2 table_stringsrow_html_stringsrstyle) rTrjrWrl list_images list_titlesrU png_datasrVrkrr html_from_compressed_imagess.  z0show_images..html_from_compressed_imagesrcSs$g|]}|ddddddfqS)Nrrrrrrr r5s$cSryrrzrrrr r6r{)rr2rDr4)rr4rDr4rDrN)rsrrzlistkeysvaluesryrrMr<rrrFr!_IPYTHON_HTML_SIZE_LIMITr)rnrprVrWrirjr8r9r:rTrkrlrUrmrSrrArrrr)rTr:rjrurWrlrrrxrUrr9r8rVrkr rsL 5   codeccCs|dkrdSdS)Ngif.gifz.mp4rrrrr _filename_suffix_from_codecArcrcCs t}|stdtjd|S)Nz Program 'zB' is not found; perhaps install ffmpeg using 'apt install ffmpeg'.)rPrrMr:rHrrr _get_ffmpeg_pathEs  rcCs tduS)zKReturns True if the program `ffmpeg` is found. See also `set_ffmpeg`. N)rPrrrr r1Os r1c@s2eZdZUdZded<ded<ded<ded <d S) raRepresents the data stored in a video container header. Attributes: num_images: Number of frames that is expected from the video stream. This is estimated from the framerate and the duration stored in the video header, so it might be inexact. shape: The dimensions (height, width) of each video frame. fps: The framerate in frames per second. bps: The estimated bitrate of the video stream in bits per second, retrieved from the video header. rrrqrpr=fpsr\bpsN)r=r>r?r0r@rrrr rWs   rc Cst|std|dtddt|dddddd d g }tj|tjd d }| \}}Wd n1s9wYd }}}}| dD]l} t d| } r^t | dd}t d| } rlt | d}d| vrd| vrt d| } std| t | dt | d}} t d| } std| t| d} t d| } rt | d}qK|std||std||dvr| |}} | |f}t||| |S)z?Returns attributes of video stored in the specified local file.z Video file 'z' is not found.-nostdin-iz-acodeccopy-vcodec-fnull-rY)rVencodingN z, bitrate: *([0-9.]+) kb/srizframe= *([0-9]+) rz Stream #0:z: Video:z, ([0-9]+)x([0-9]+)z)Unable to parse video dimensions in line rrz, ([0-9.]+) fpsz(Unable to parse video framerate in line z\s*rotate\s*:\s*(\d+)z Unable to find frames in video: zUnable to parse video header: )Zi)rris_filerrrNrPopenr communicatesplitresearchrgroupfindallr= fullmatchr)rAcommandrr*errrrrVrotationlinematchmatchesrWrrprrr _get_video_metadatajsX   rc@seZdZdZd ddZd S) _VideoIOz/Base class for `VideoReader` and `VideoWriter`.rr5 image_formatrNrDcCsFdddtj}tjddddtjd|d |d |di|j|S) z8Returns ffmpeg pix_fmt given data type and image format.lebe)littlebigrgb24yuv444pr6rrr6rgb48 yuv444p16gray16)rU byteorderrrrr)r&rrnative_endian_suffixrrr _get_pix_fmts   z_VideoIO._get_pix_fmtN)rr5rrNrDrN)r=r>r?r0rrrrr rsrc@seZdZUdZded<ded<ded<ded <d ed <d ed <ded<ded<d ed<dejdd*ddZd+ddZd,dd Z d-d"d#Z d.d%d&Z d/d'd(Z d)S)0ra7Context to read a compressed video as an iterable over its images. >>> with VideoReader('/tmp/river.mp4') as reader: ... print(f'Video has {reader.num_images} images with shape={reader.shape},' ... f' at {reader.fps} frames/sec and {reader.bps} bits/sec.') ... for image in reader: ... print(image.shape) >>> with VideoReader('/tmp/river.mp4') as reader: ... video = np.array(tuple(reader)) >>> url = 'https://github.com/hhoppe/data/raw/main/video.mp4' >>> with VideoReader(url) as reader: ... show_video(reader) Attributes: path_or_url: Location of input video. output_format: Format of output images (default 'rgb'). If 'rgb', each image has shape=(height, width, 3) with R, G, B values. If 'yuv', each image has shape=(height, width, 3) with Y, U, V values. If 'gray', each image has shape=(height, width). dtype: Data type for output images. The default is `np.uint8`. Use of `np.uint16` allows reading 10-bit or 12-bit data without precision loss. metadata: Object storing the information retrieved from the video header. Its attributes are copied as attributes in this class. num_images: Number of frames that is expected from the video stream. This is estimated from the framerate and the duration stored in the video header, so it might be inexact. shape: The dimensions (height, width) of each video frame. fps: The framerate in frames per second. bps: The estimated bitrate of the video stream in bits per second, retrieved from the video header. r9r rN output_formatr5rrmetadatarrrqrpr=rr\r_num_bytes_per_imager)rrr3cCsj|dvr td|d||_||_t||_|jjtjtjfvr*td|dd|_d|_ d|_ dS)N>rrr6zOutput format  is not rgb, yuv, or gray.r is not np.uint8 or np.uint16.) rzr rrrrrrr_popen_proc)r&r rrrrr r's   zVideoReader.__init__rD 'VideoReader'cCst}z`t|j|_|j}t||_|j\|_|_|_|_ | |j |j }dddd|j }|j j }t|j|||_|dddd|dd d d d |d ddg}tj|tjtjd|_|j|_W|Styr|dddw)Nrrr-vpanicrrrrawvideor image2pipe-pix_fmtz-vsyncvfrr)rrV)rrr r)rrrrprrrrritemsizemathprodrrrrrr Exceptionr,)r& ffmpeg_pathtmp_namepix_fmt num_channelsbytes_per_channelrrrr r)sH     zVideoReader.__enter__r*rrQcG |dSrcloser+rrr r,r-zVideoReader.__exit___NDArray | NonecCs|jsJd|jjs J|jj|j}|s|dSt||jks&Jtj||jd}|j dkr@|j g|j dR}|S|j dkrVt |j dg|j Rdd}|S|j d krc|j |j }|St ) zReads a video image frame (or None if at end of file). Returns: A numpy array in the format specified by `output_format`, i.e., a 3D array with 3 color channels, except for format 'gray' which is 2D. z.Error: reading from an already closed context.Nrrrrrrrr6)rrrrrryr frombufferrrreshaperprAssertionError)r&rrrrr rs$    zVideoReader.readIterator[_NDArray]ccs |}|dur dS|Vqr)r)r&rrrr __iter__4szVideoReader.__iter__cCsF|jr|jdddd|_d|_|jr!|jdddd|_dSdS)zCTerminates video reader. (Called automatically at end of context.)N)rr,rrr(rrr r;s zVideoReader.closeN)r r9rrNrr3)rDrr/)rDr)rDrr.) r=r>r?r0r@rrr'r)r,rrrrrrr rs& "  *  rc @s\eZdZdZddddddddejdd d-ddZd.d!d"Zd/d%d&Zd0d)d*Z d1d+d,Z dS)2ra Context to write a compressed video. >>> shape = (480, 640) >>> with VideoWriter('/tmp/v.mp4', shape, fps=60) as writer: ... for image in moving_circle(shape, num_images=60): ... writer.add_image(image) >>> show_video(read_video('/tmp/v.mp4')) Bitrate control may be specified using at most one of: `bps`, `qp`, or `crf`. If none are specified, `qp` is set to a default value. See https://slhck.info/video/2017/03/01/rate-control.html If codec is 'gif', the args `bps`, `qp`, `crf`, and `encoded_format` are ignored. Attributes: path: Output video. Its suffix (e.g. '.mp4') determines the video container format. The suffix must be '.gif' if the codec is 'gif'. shape: 2D spatial dimensions (height, width) of video image frames. The dimensions must be even if 'encoded_format' has subsampled chroma (e.g., 'yuv420p' or 'yuv420p10le'). codec: Compression algorithm as defined by "ffmpeg -codecs" (e.g., 'h264', 'hevc', 'vp9', or 'gif'). metadata: Optional VideoMetadata object whose `fps` and `bps` attributes are used if not specified as explicit parameters. fps: Frames-per-second framerate (default is 60.0 except 25.0 for 'gif'). bps: Requested average bits-per-second bitrate (default None). qp: Quantization parameter for video compression quality (default None). crf: Constant rate factor for video compression quality (default None). ffmpeg_args: Additional arguments for `ffmpeg` command, e.g. '-g 30' to introduce I-frames, or '-bf 0' to omit B-frames. input_format: Format of input images (default 'rgb'). If 'rgb', each image has shape=(height, width, 3) or (height, width). If 'yuv', each image has shape=(height, width, 3) with Y, U, V values. If 'gray', each image has shape=(height, width). dtype: Expected data type for input images (any float input images are converted to `dtype`). The default is `np.uint8`. Use of `np.uint16` is necessary when encoding >8 bits/channel. encoded_format: Pixel format as defined by `ffmpeg -pix_fmts`, e.g., 'yuv420p' (2x2-subsampled chroma), 'yuv444p' (full-res chroma), 'yuv420p10le' (10-bit per channel), etc. The default (None) selects 'yuv420p' if all shape dimensions are even, else 'yuv444p'. h264Nrr) rrrrqpcrf ffmpeg_args input_formatrencoded_formatrAr9rprqrrNrVideoMetadata | Nonerr;rr\rrrr~rrr3rrJrDrQc Cst||dur |r |j}|dur|dkrdnd}|dkr%td|d|dur.|r.|j}|dur6t|nd}|durH|dkrHtd|d|dur]t|trU|dkr]td |d td d |||fD} | d kr{td|d|d|dt| trt | nt | } | dvrtd| dt | } | j t jt jfvrtd| dt||_||_tdd |D}| dur|rdnd} |s| drtd| d|d||_||_||_||_||_| |_| |_| |_ | |_| dkr | s t|jdkrdnd}|durd |gng|durd!|gng|dur,d d"d#|gng|_|jdkrX|jjd$krFtd%|jd&d'|_g|_d(}d)|d*dg|j|_d|_ d|_!d|_"dS)+Nrg9@gN@rzFrame-per-second value z is invalid.rzBitrate value zQuantization parameter z is not a positive integer.css|]}|duVqdSrr)rrrrr r r z'VideoWriter.__init__..rz-Must specify at most one of bps, qp, or crf (z, rt>rrr6z Input format rrrcss|] }|ddkVqdS)rrrNr)rdimrrr r rxyuv420pr)yuv42yuvj42zWith encoded_format z., video dimensions must be even, but shape is r iz-vbz-qp0z-crfrzFile 'z' does not have a .gif suffix.pal8z1split[s0][s1];[s0]palettegen[p];[s1][p]paletteusez-vfr)#r}rrzrrrsrrNshlexrrrrrrrrrrArpr{r rrrrrrrr _bitrate_argsrr#rr)r&rArprrrrrrrrrrnum_rate_specificationsall_dimensions_are_even video_filterrrr r'us         zVideoWriter.__init__ 'VideoWriter'cCst}||j|j}zMt|j|_|j}|j\}}|ddddddd|d|d|d |jd d d d|j d|j g|j |j d |g}t j|t jt jd|_|j|_W|Styg|dddw)Nrerrorrrrrz-srz-rrrz-anz-y)stdinrV)rrrrr#rAr)rprrrrrrrrrrrr,)r&r input_pix_fmtrrWrVrrrr r)sV      zVideoWriter.__enter__r*rcGrrrr+rrr r,r-zVideoWriter.__exit__rr4cCs|jsJdt|jjtjtjfrt||j}|j|jkr+td|jd|jd|j dkr?|j dkr>td|j dn'|j dkrQ|j d krQt |||f}|j d kr]|j dd ksftd|j d |j d d|j krtd|j d dd |j d|j dkrt |dd}|}|jjsJ|jj|t|kr|j|jjsJ|jj}td|jd|d S)a"Writes a video frame. Args: image: Array whose dtype and first two dimensions must match the `dtype` and `shape` specified in `VideoWriter` initialization. If `input_format` is 'gray', the image must be 2D. For the 'rgb' input_format, the image may be either 2D (interpreted as grayscale) or 3D with three (R, G, B) channels. For the 'yuv' input_format, the image must be 3D with three (Y, U, V) channels. Raises: RuntimeError: If there is an error writing to the output file. z,Error: writing to an already closed context.z Image type z != r r6rrzImage dimensions z are not 2D.rrz are invalid.Nz- do not match those of the initialized video rrError writing '': )rrrrrrbool_r(rzrrrprrtobytesrr!rywaitrVrrerrA)r&rrrrrr add_images:        zVideoWriter.add_imagecCs|jr<|jr|jjr|jjsJ|jj|jr.|jj}td|j d||j dddd|_d|_|j rL|j dddd|_ dSdS)zFFinishes writing the video. (Called automatically at end of context.)rrN) rrrrVrr rrerrAr,r#)r&rrrr r*s   zVideoWriter.close)rAr9rprqrrNrrrr;rr\rr\rr;rr~rrNrr3rrJrDrQ)rDrr/)rr4rDrQr.) r=r>r?r0rrr'r)r,r rrrrr rGs"2 Y / *rc@s2eZdZUdZded< ddd d ZdddZdS) _VideoArrayzEWrapper to add a VideoMetadata `metadata` attribute to a numpy array.rrNclstyping.Type['_VideoArray'] input_arrayr4rD '_VideoArray'cCst||}||_|Sr)rrviewr)rrrobjrrr __new__@sz_VideoArray.__new__rrrQcCs|durdSt|dd|_dS)Nr)getattrr)r&rrrr __array_finalize__Isz_VideoArray.__array_finalize__r)rrrr4rrrDr)rrrDrQ)r=r>r?r0r@rrrrrr r ;s   r cKsLt|fi|}ttt||jdWdS1swYdS)a$Returns an array containing all images read from a compressed video file. >>> video = read_video('/tmp/river.mp4') >>> print(f'The framerate is {video.metadata.fps} frames/s.') >>> show_video(video) >>> url = 'https://github.com/hhoppe/data/raw/main/video.mp4' >>> show_video(read_video(url)) Args: path_or_url: Input video file. **kwargs: Additional parameters for `VideoReader`. Returns: A 4D `numpy` array with dimensions (frame, height, width, channel), or a 3D array if `output_format` is specified as 'gray'. The returned array has an attribute `metadata` containing `VideoMetadata` information. This enables `show_video` to retrieve the framerate in `metadata.fps`. Note that the metadata attribute is lost in most subsequent `numpy` operations. )rN)rr rrr_r)r rCreaderrrr rOs$rcKst|\}}|jdd}|j}|tkrttj}n t|tjr(ttj}dt |ddi|}t |f||d|}|D]}| |q@WddS1sSwYdS)a~Writes images to a compressed video file. >>> video = moving_circle((480, 640), num_images=60) >>> write_video('/tmp/v.mp4', video, fps=60, qp=18) >>> show_video(read_video('/tmp/v.mp4')) Args: path: Output video file. images: Iterable over video frames, e.g. a 4D array or a list of 2D or 3D arrays. **kwargs: Additional parameters for `VideoWriter`. Nrrr)rpr) rorprrGrrrrrrrr )rArnrC first_imagerprwriterrrrr rhs   "rrrcKsft|}t!}t|d|}t||fd|i||WdS1s,wYdS)aLReturns a buffer containing a compressed video. The video container is 'mp4' except when `codec` is 'gif'. >>> video = read_video('/tmp/river.mp4') >>> data = compress_video(video, bps=10_000_000) >>> print(len(data)) >>> data = compress_video(moving_circle((100, 100), num_images=10), fps=10) Args: images: Iterable over video frames. codec: Compression algorithm as defined by `ffmpeg -codecs` (e.g., 'h264', 'hevc', 'vp9', or 'gif'). **kwargs: Additional parameters for `VideoWriter`. Returns: A bytes buffer containing the compressed video. rRrN)rrrrrrr")rnrrCrrrrrr r!s  $r!cKsTt}t|d}||t|fi|WdS1s#wYdS)z8Returns video images from an MP4-compressed data buffer.zfile.mp4N)rrrrrr)rrCrrrrr r"s  $r")rSrTloopautoplayrrc Cst|d}t|tr|d}n|rd}nd}d|d|d|d|r(d nd|r.d nd }d |d |d } |rFd|d| d} | S)aReturns an HTML string with a video tag containing H264-encoded data. Args: data: MP4-compressed video bytes. width: Width of HTML video in pixels. height: Height of HTML video in pixels. title: Optional text shown centered above the video. border: If `bool`, whether to place a black boundary around the image, or if `str`, the boundary CSS style. loop: If True, the playback repeats forever. autoplay: If True, video playback starts without having to click. rYrZr[rzcontrols width="r]r^zobject-fit:cover;"z loopz autoplay mutedzr_r`rarb) rrVrWrSrTrrrfoptionsrrrr r$s0    r$cKst|g|gfi|S)aUDisplays a video in the IPython notebook and optionally saves it to a file. See `show_videos`. >>> video = read_video('https://github.com/hhoppe/data/raw/main/video.mp4') >>> show_video(video, title='River video') >>> show_video(moving_circle((80, 80), num_images=10), fps=5, border=True) >>> show_video(read_video('/tmp/river.mp4')) Args: images: Iterable of video frames (e.g., a 4D array or a list of 2D or 3D arrays). title: Optional text shown centered above the video. **kwargs: See `show_videos`. Returns: html string if `return_html` is `True`. )r)rnrSrCrrr rsrr) rVrWrirjrrrrrkrlrmvideos?Iterable[Iterable[_NDArray]] | Mapping[str, Iterable[_NDArray]]rrrc  st|tr|dur tdt|}t|}n*t|}|dur)dgt|nt|}t|t|krDtdt|dt|d| dvrPtd| dg}t||D]\}}t|d d}t |\}}t |||j dd \|r|j d ks|j d krfd d|D}|d }t |||||| d}|rt jrt| }tt j||}t|dd }||Wdn1swY| dkr|j d kp݈|j d k}t|f|d|d| }n t|fd|i| }||qWg}t||D]/}ddfdd|D}| r(d}d|d| d|}|d| d|dqd|}| r@|St|dS) aDisplays a row of videos in the IPython notebook. Creates HTML with `