qi&ddlmZmZmZddlZddlmZmZm Z m Z m Z ddl m Z mZe r ddlmZddlmZer dd lmZmZmZmZe j0eZeed Gd d e Zy))AnyUnionoverloadN)add_end_docstringsis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_image)*MODEL_FOR_IMAGE_SEGMENTATION_MAPPING_NAMES-MODEL_FOR_INSTANCE_SEGMENTATION_MAPPING_NAMES-MODEL_FOR_SEMANTIC_SEGMENTATION_MAPPING_NAMES.MODEL_FOR_UNIVERSAL_SEGMENTATION_MAPPING_NAMEST)has_image_processorc ,eZdZdZdZdZdZdZfdZdZ e de e dfd e d eee e ffd Ze dee edzd e d eeee e ffd Zde e dee edfd e d eee e feeee e fzffd ZddZdZ ddZxZS)ImageSegmentationPipelinea Image segmentation pipeline using any `AutoModelForXXXSegmentation`. This pipeline predicts masks of objects and their classes. Example: ```python >>> from transformers import pipeline >>> segmenter = pipeline(model="facebook/detr-resnet-50-panoptic") >>> segments = segmenter("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") >>> len(segments) 2 >>> segments[0]["label"] 'bird' >>> segments[1]["label"] 'bird' >>> type(segments[0]["mask"]) # This is a black and white mask showing where is the bird on the original image. >>> segments[0]["mask"].size (768, 512) ``` This image segmentation pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"image-segmentation"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=image-segmentation). FTNct||i|t|dtj}|j t |j t|j t|j|y)Nvision) super__init__r rcopyupdaterrrcheck_model_type)selfargskwargsmapping __class__s [/opt/pipecat/venv/lib/python3.12/site-packages/transformers/pipelines/image_segmentation.pyrz"ImageSegmentationPipeline.__init__Dsb $)&)$)<AACDEDEEF g&c i}i}d|vr|d|d<|d|d<d|vr|d|d<d|vr|d|d<d|vr|d|d<d|vr|d|d<|i|fS)Nsubtask thresholdmask_thresholdoverlap_mask_area_thresholdtimeout)rr!preprocess_kwargspostprocess_kwargss r$_sanitize_parametersz.ImageSegmentationPipeline._sanitize_parametersNs  ,29,= y )+1)+< i ( & .4[.A { + v %39:J3K / 0 (F 2@FGd@e < =  +1)+< i ( "&888r%inputsz Image.Imager!returnc yNr,rr0r!s r$__call__z"ImageSegmentationPipeline.__call___sber%c yr3r,r4s r$r5z"ImageSegmentationPipeline.__call__bsnqr%c hd|vr|jd}| tdt| |fi|S)a Perform segmentation (detect masks & classes) in the image(s) passed as inputs. Args: inputs (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`): The pipeline handles three types of images: - A string containing an HTTP(S) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the same format: all as HTTP(S) links, all as local paths, or all as PIL images. subtask (`str`, *optional*): Segmentation task to be performed, choose [`semantic`, `instance` and `panoptic`] depending on model capabilities. If not set, the pipeline will attempt tp resolve in the following order: `panoptic`, `instance`, `semantic`. threshold (`float`, *optional*, defaults to 0.9): Probability threshold to filter out predicted masks. mask_threshold (`float`, *optional*, defaults to 0.5): Threshold to use when turning the predicted masks into binary values. overlap_mask_area_threshold (`float`, *optional*, defaults to 0.5): Mask overlap threshold to eliminate small, disconnected segments. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: If the input is a single image, will return a list of dictionaries, if the input is a list of several images, will return a list of list of dictionaries corresponding to each image. The dictionaries contain the mask, label and score (where applicable) of each detected object and contains the following keys: - **label** (`str`) -- The class label identified by the model. - **mask** (`PIL.Image`) -- A binary mask of the detected object as a Pil Image of shape (width, height) of the original image. Returns a mask filled with zeros if no object is found. - **score** (*optional* `float`) -- Optionally, when the model is capable of estimating a confidence of the "object" described by the label and the mask. imageszICannot call the image-classification pipeline without an inputs argument!)pop ValueErrorrr5)rr0r!r#s r$r5z"ImageSegmentationPipeline.__call__esBX v ZZ)F >hi iw1&11r%ct||}|j|jfg}|jjj j dk(rx|i}nd|gi}|jd |gdd|}|j|j}|j|dd|jjjdd|d<n/|j|gd}|j|j}||d <|S) N)r+OneFormerConfig task_inputspt)r8return_tensors max_length)paddingr@r? input_ids target_sizer,) rheightwidthmodelconfigr#__name__image_processortodtype tokenizer task_seq_len)rimager'r+rCr!r0s r$ preprocessz$ImageSegmentationPipeline.preprocesss5'2 ekk23 ::   & & / /3D D''3)T))X%XQWXFYYtzz*F$(NN}%$::,,99# %3%  %F= !))%)NFYYtzz*F +} r%cV|jd}|jdi|}||d<|S)NrCr,)r9rF)r model_inputsrC model_outputss r$_forwardz"ImageSegmentationPipeline._forwards5"&&}5 " 2\2 '2 m$r%c<d}|dvr-t|jdr|jj}n0|dvr,t|jdr|jj}|||||||dd}g}|d} |d D]} | | d k(d z} t j | j jtjd } |jjj| d} | d} |j| | | d|S|dvrt|jdr|jj||dd}g}|j } tj| }|D]v} | | k(d z} t j | jtjd } |jjj| } |jd| | dx|St!d|dt#|j)N>Npanoptic"post_process_panoptic_segmentation>Ninstance"post_process_instance_segmentationrC)r(r)r* target_sizesr segmentation segments_infoidL)modelabel_idscore)ralabelmask>Nsemantic"post_process_semantic_segmentation)rYzSubtask z is not supported for model )hasattrrIrVrXr fromarraynumpyastypenpuint8rFrGid2labelappendreuniquer:type)rrRr'r(r)r*fnoutputs annotationrZsegmentrcrbralabelss r$ postprocessz%ImageSegmentationPipeline.postprocesss4 ( (WT5I5IKo-p%%HHB * *wt7K7KMq/r%%HHB >#-,G*=9   GJ">2L"?3 R$ 5<tzz|':':288'D3O ))227:3FG(!!EE4"PQ  R.!* *wt7K7KMq/r**MMM-,HNGJ"==?LYY|,F Q$-4t{{288'<3G ))2259!!D5$"OP  Qxy0LTRVR\R\M]L^_` `r%)NN)Ng??rv)rH __module__ __qualname____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerrr/rrstrrlistdictr5rOrSru __classcell__)r#s@r$rrs!FO #O'9"euS-%78eCeDQUVY[^V^Q_L`ee qtCy4 +>>q#qRVW[\`adfiai\jWkRlqq02CS 4 ;NNO02[^02 d38n T$sCx.%9 : :02d,kn,r%r)typingrrrrhrjutilsrrr r r baser rPILr image_utilsrmodels.auto.modeling_autorrrr get_loggerrHloggerrr,r%r$rss''kk4(   H %,FGDDHDr%