qiddlmZddlmZmZmZddlmZmZm Z m Z m Z ddl m Z mZe r ddlmZddlmZer dd lZdd lmZe j,eZeed Gd de Zy ))UserDict)AnyUnionoverload)add_end_docstringsis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_imageN)6MODEL_FOR_ZERO_SHOT_IMAGE_CLASSIFICATION_MAPPING_NAMEST)has_image_processorc PeZdZdZdZdZdZdZfdZe de e dfde e de d e ee e ffd Ze de e e dzde e de d e e ee e ffd Zde e e e de dfde e de d e ee e fe e ee e fzffd Zdd Z ddZdZdZxZS)#ZeroShotImageClassificationPipelineaL Zero shot image classification pipeline using `CLIPModel`. This pipeline predicts the class of an image when you provide an image and a set of `candidate_labels`. Example: ```python >>> from transformers import pipeline >>> classifier = pipeline(model="google/siglip-so400m-patch14-384") >>> classifier( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["animals", "humans", "landscape"], ... ) [{'score': 0.965, 'label': 'animals'}, {'score': 0.03, 'label': 'humans'}, {'score': 0.005, 'label': 'landscape'}] >>> classifier( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["black and white", "photorealist", "painting"], ... ) [{'score': 0.996, 'label': 'black and white'}, {'score': 0.003, 'label': 'photorealist'}, {'score': 0.0, 'label': 'painting'}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image classification pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"zero-shot-image-classification"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=zero-shot-image-classification). FTc ft|di|t|d|jty)Nvision)super__init__r check_model_typer)selfkwargs __class__s g/opt/pipecat/venv/lib/python3.12/site-packages/transformers/pipelines/zero_shot_image_classification.pyrz,ZeroShotImageClassificationPipeline.__init__Cs, "6"$) TUimagez Image.Imagecandidate_labelsrreturnc yNrrr!r"rs r__call__z,ZeroShotImageClassificationPipeline.__call__Is #r c yr%rr&s rr'z,ZeroShotImageClassificationPipeline.__call__Ns&)r c ld|vr|jd}| tdt| |fd|i|S)a Assign labels to the image(s) passed as inputs. Args: image (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`): The pipeline handles three types of images: - A string containing a http link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly candidate_labels (`list[str]`): The candidate labels for this image. They will be formatted using *hypothesis_template*. hypothesis_template (`str`, *optional*, defaults to `"This is a photo of {}"`): The format used in conjunction with *candidate_labels* to attempt the image classification by replacing the placeholder with the candidate_labels. Pass "{}" if *candidate_labels* are already formatted. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list of dictionaries containing one entry per proposed label. Each dictionary contains the following keys: - **label** (`str`) -- One of the suggested *candidate_labels*. - **score** (`float`) -- The score attributed by the model to that label. It is a value between 0 and 1, computed as the `softmax` of `logits_per_image`. imageszSCannot call the zero-shot-image-classification pipeline without an images argument!r")pop ValueErrorrr')rr!r"rrs rr'z,ZeroShotImageClassificationPipeline.__call__SsHJ v JJx(E =rs swS8HSFSSr c Xi}d|vr|d|d<d|vr|d|d<d|vr|d|d<|iifS)Nr"timeouthypothesis_templater)rtokenizer_kwargsrpreprocess_paramss r_sanitize_parametersz8ZeroShotImageClassificationPipeline._sanitize_parameters~sa  '4:;M4N 0 1  +1)+< i ( F *7=>S7T 3 4 "b((r c|i}t||}|j|gd}|j|j}||d<|Dcgc]}|j |}}ddi} d|j j jvr| jdd d | j||j|fd di| } | g|d <|Scc}w) N)r.pt)r*return_tensorsr"paddingTsiglip max_length@)r6r8 truncationr5 text_inputs) rimage_processortodtypeformatmodelconfig model_typeupdate tokenizer) rr!r"r/r.r0inputsx sequencestokenizer_default_kwargsr;s r preprocessz.ZeroShotImageClassificationPipeline.preprocesss  #! 5'2%%eWT%J4::&%5!"?"&&}5 k!nh /%a.K&a.+K$**;{;l;!1.. r c|jd}|dd}d|jjjvrHt j |j d}|j}t|tsH|g}nD|jdj d}|j}t|ts|g}tt||dDcgc] \}}||d  }}}|Scc}}w) Nr"rKrr7)dimc|d S)Nrr)rFs rzAZeroShotImageClassificationPipeline.postprocess..s_`ab_c^cr )key)scorelabel) r+r@rArBtorchsigmoidsqueezetolistrLlistsoftmaxsortedzip) rrPr"rKprobsscoresrXcandidate_labelresults r postprocessz/ZeroShotImageClassificationPipeline.postprocesss(,,-?@x(+ tzz((33 3MM&)11"5E\\^Ffd+ NNrN*2226E\\^Ffd+ +1V=M1NTc*d &o 6    s#C7r%)NzThis is a photo of {}.NN)__name__ __module__ __qualname____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerrrrstrr^rdictr'r2rIrQrf __classcell__)rs@rrrs`@O #OV #3 -.#BFs)#WZ# d38n ##)#Ym!44)HLS )]`) d4S>" #)))TS$s)]D4GGH)Ts))T )T d38n T$sCx.%9 : : )TV )4 ."r r) collectionsrtypingrrrutilsrr r r r baserrPILr image_utilsrrZmodels.auto.modeling_autor get_loggerrgloggerrrr rr{su ''5(b   H %,FGf(fHfr