qi=_dZddlZddlmZmZmZddlmZm Z m Z m Z m Z ddl mZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZmZmZdd l m!Z!ejDe#Z$e!rddl%Z%Gd d ed Z&GddeZ'dgZ(y)z!Image processor class for Nougat.N)BaseImageProcessor BatchFeature get_size_dict)get_resize_output_image_sizepadresizeto_channel_dimension_format to_pil_image) IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDChannelDimension ImageInputPILImageResamplingget_image_sizeinfer_channel_dimension_formatis_scaled_imagemake_flat_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) ImagesKwargs) TensorTypefilter_out_non_signature_kwargslogging)is_vision_availablec0eZdZUdZeed<eed<eed<y)NougatImageProcessorKwargsa do_crop_margin (`bool`, *optional*, defaults to `True`): Whether to crop the image margins. do_thumbnail (`bool`, *optional*, defaults to `True`): Whether to resize the image using thumbnail method. do_align_long_axis (`bool`, *optional*, defaults to `False`): Whether to align the long axis of the image with the long axis of `size` by rotating by 90 degrees. do_crop_margin do_thumbnaildo_align_long_axisN)__name__ __module__ __qualname____doc__bool__annotations__d/opt/pipecat/venv/lib/python3.12/site-packages/transformers/models/nougat/image_processing_nougat.pyrr4sr)rF)totalc$|eZdZdZdgZeZdddejddddddddf de de d e e e fdzd ed e d e d e de de e zde de ee zdzde ee zdzddffd Zdej"fdZdZ d#dej"de dedzde ezdzdej"f dZ d$dej"d e e e fde ezdzde ezdzdej"f dZ d$dej"d e e e fde ezdzde ezdzdej"f dZej0ddfdej"d e e e fd ede ezdzde ezdzdej"f dZej0ddfdej"d e e e fd ede ezdzde ezdzdej"f dZedddddddddddddej8dfd ede dzde dzd e e e fdzd edzd e dzd e dzd e dzde dzde e zdzde dzde ee zdzde ee zdzd!e ezdzdedzde ezdzdej@j@f"d"Z!xZ"S)%NougatImageProcessora Constructs a Nougat image processor. Args: do_crop_margin (`bool`, *optional*, defaults to `True`): Whether to crop the image margins. do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions to the specified `size`. Can be overridden by `do_resize` in the `preprocess` method. size (`dict[str, int]` *optional*, defaults to `{"height": 896, "width": 672}`): Size of the image after resizing. Can be overridden by `size` in the `preprocess` method. resample (`PILImageResampling`, *optional*, defaults to `Resampling.BILINEAR`): Resampling filter to use if resizing the image. Can be overridden by `resample` in the `preprocess` method. do_thumbnail (`bool`, *optional*, defaults to `True`): Whether to resize the image using thumbnail method. do_align_long_axis (`bool`, *optional*, defaults to `False`): Whether to align the long axis of the image with the long axis of `size` by rotating by 90 degrees. do_pad (`bool`, *optional*, defaults to `True`): Whether to pad the images to the largest image size in the batch. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image by the specified scale `rescale_factor`. Can be overridden by the `do_rescale` parameter in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `1/255`): Scale factor to use if rescaling the image. Can be overridden by the `rescale_factor` parameter in the `preprocess` method. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by `do_normalize` in the `preprocess` method. image_mean (`float` or `list[float]`, *optional*, defaults to `IMAGENET_DEFAULT_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `list[float]`, *optional*, defaults to `IMAGENET_DEFAULT_STD`): Image standard deviation. pixel_valuesTNFgp?r do_resizesizeresampler r!do_pad do_rescalerescale_factor do_normalize image_mean image_stdreturnc  t|di| ||nddd}t|}||_||_||_||_||_||_||_ ||_ | |_ | |_ | | nt|_| | |_yt|_y)Nii)heightwidthr()super__init__rrr/r0r1r r!r2r3r4r5r r6r r7)selfrr/r0r1r r!r2r3r4r5r6r7kwargs __class__s r*r=zNougatImageProcessor.__init__is "6"'tc-JT","   ("4 $,((2(>*DY&/&;AUr)imagectjtj|}|ddddgf}|jddd}|S)zGThis is a reimplementation of a findNonZero function equivalent to cv2.Nr)np column_stacknonzeroreshape)r>rAnon_zero_indicesidxvecs r*python_find_non_zeroz)NougatImageProcessor.python_find_non_zerosC??2::e+<=!!aV),Aq) r)ctj|djt}tj|djt}|d|d}}|d|z dz}|d|z dz}||||fS)zHThis is a reimplementation of a BoundingRect function equivalent to cv2.rrC)axisrrC)rFminastypeintmax)r> coordinates min_values max_valuesx_miny_minr;r:s r*python_bounding_rectz)NougatImageProcessor.python_bounding_rects~VVKf5<rArZr[r\datamax_valmin_valgraycoordsrWrXr;r:s r* crop_marginz NougatImageProcessor.crop_margins,  $ >u E U6GHxx c*+22288<((*((* g HHUOE/7HJZJ_J_`E*,E;@QR  L  Lw7W#45;n$**40&*&?&?&G#ueV E5%%-HI&&rxx0+E3DFVF[F[\S^Ri '{rAr0r[r\ input_height input_width output_height output_widthrot_axess r*align_long_axisz$NougatImageProcessor.align_long_axiss.%35FW$X! k&*8nd7m|  $ >u E   0 5 5 5H "2"8"8 8H89J8KLM M = ([<-G = ([<-GHHUAH5E  "/{VghE r)c|d|d}}t||\}}||z } ||z } | dz} | dz} | | z } | | z }| | f| |ff}t||||S)a Pad the image to the specified size at the top, bottom, left and right. Args: image (`np.ndarray`): The image to be padded. size (`dict[str, int]`): The size `{"height": h, "width": w}` to pad the image to. data_format (`str` or `ChannelDimension`, *optional*): The data format of the output image. If unset, the same format as the input image is used. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. r:r;rmrE)r[r\)rr)r>rAr0r[r\rwrxrurv delta_width delta_heightpad_toppad_left pad_bottom pad_rightpaddings r* pad_imagezNougatImageProcessor.pad_images('+8nd7m| $25FW$X! k"[0 $|3 !#!#!G+ (* Z(8Y*?@5'{N_``r)c t||\}}|d|d} } t|| } t|| } | |k(r| |k(r|S||kDrt|| z|z } n||kDrt|| z|z } t|f| | f|d||d|S)as Resize the image to make a thumbnail. The image is resized so that no dimension is larger than any corresponding dimension of the specified size. Args: image (`np.ndarray`): The image to be resized. size (`dict[str, int]`): The size `{"height": h, "width": w}` to resize the image to. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BICUBIC`): The resampling filter to use. data_format (`Optional[Union[str, ChannelDimension]]`, *optional*): The data format of the output image. If unset, the same format as the input image is used. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. rmr:r;g@)r0r1 reducing_gapr[r\)rrPrRr ) r>rAr0r1r[r\r?rurvrwrxr:r;s r* thumbnailzNougatImageProcessor.thumbnail#s2%35FW$X! k&*8nd7m| \=1K. \ !e{&:L + % f,|;rAr0r1r[r\r? shortest_edge output_size resized_images r*r zNougatImageProcessor.resizeVsh0T"DNDM: 2  Rc   #/    r)imagesreturn_tensorsc "||n |j}||n |j}||n |j}||n |j}||n |j}||n |j }||n |j }| | n |j} | | n |j} | | n |j} | | n |j} | | n |j} t|}t|s tdt| | | | | ||||Dcgc] }t!|}}| r#t#|drt$j'd|t)|d}|r |Dcgc]}|j+||}}|r!|Dcgc]}|j-|||}}|r"|Dcgc]}|j/||||}}|r!|Dcgc]}|j1|||}}|r!|Dcgc]}|j3|||}}| r!|Dcgc]}|j5|| | }}| r"|Dcgc]}|j7|| | | }}|Dcgc]}t9||| }}d |i}t;|| Scc}wcc}wcc}wcc}wcc}wcc}wcc}wcc}wcc}w)a+ Preprocess an image or batch of images. Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. do_crop_margin (`bool`, *optional*, defaults to `self.do_crop_margin`): Whether to crop the image margins. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`dict[str, int]`, *optional*, defaults to `self.size`): Size of the image after resizing. Shortest edge of the image is resized to min(size["height"], size["width"]) with the longest edge resized to keep the input aspect ratio. resample (`int`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`. Only has an effect if `do_resize` is set to `True`. do_thumbnail (`bool`, *optional*, defaults to `self.do_thumbnail`): Whether to resize the image using thumbnail method. do_align_long_axis (`bool`, *optional*, defaults to `self.do_align_long_axis`): Whether to align the long axis of the image with the long axis of `size` by rotating by 90 degrees. do_pad (`bool`, *optional*, defaults to `self.do_pad`): Whether to pad the images to the largest image size in the batch. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image by the specified scale `rescale_factor`. rescale_factor (`int` or `float`, *optional*, defaults to `self.rescale_factor`): Scale factor to use if rescaling the image. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `list[float]`, *optional*, defaults to `self.image_mean`): Image mean to use for normalization. image_std (`float` or `list[float]`, *optional*, defaults to `self.image_std`): Image standard deviation to use for normalization. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: defaults to the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. zSInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, or torch.Tensor)r3r4r5r6r7r/r0r1rzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.r^)r0r\)rAr0r1r\)rAr0r\)rAscaler\)rAmeanstdr\rpr.)rf tensor_type)rr/r0r1r r!r2r3r4r5r6r7rrrsrrrlogger warning_oncerrkrzr rrrescale normalizer r)r>rrr/r0r1r r!r2r3r4r5r6r7rr[r\rArfs r* preprocesszNougatImageProcessor.preprocess}sJ,:+E4K^K^!*!6IDNN 'tTYY'38'3'?|TEVEV 3E3Q/W[WnWn!-4;;#-#9Zt +9+E4K^K^'3'?|TEVEV #-#9Zt !*!6IDNN )&1F#rs s%!)%! 6<vay I  `fgW\d&&u@Q&RgFg ouvfkd**5tO`*avFv $ %dXYj kF ouvfkdnn5tO`navFv ouvfkdnn5tO`navFv $ 5Rc dF $U^opF ou ej '{N_ `  '>BB_=hw ww   s69I$I)*I. I31I8I=7JJ<J )NN)NN)#r"r#r$r%model_input_namesr valid_kwargsrBILINEARr&dictstrrRfloatlistr=rFndarrayrLrYrrkrzrBICUBICrr rrrrrPILImager __classcell__)r@s@r*r-r-Cs D((-L $&*'9'B'B!#(&-!1504 V V V38nt# V % V  V! V V Ve  V VDK'$. V4;&- V  VD"**+"/3;? 2zz22&, 2 !11D8 2  2r6:;? -zz-38n-++d2 - !11D8 -  -f6:;? !azz!a38n!a++d2 !a !11D8 !a  !aP(:'A'A59;? 0 zz0 38n0 % 0 ++d2 0 !11D8 0  0 n(:'A'A59;? %zz%38n%% % ++d2 % !11D8 % %N%&'+!%&*.2$(*.""&-1$(150426/?/E/E;?#PCPCt PC$; PC 38nt# PC %t+ PCTkPC!4KPCt PC4KPCe d*PCTkPCDK'$.PC4;&-PCj(4/PC &,!PC"!11D8#PC$ %PC'PCr)r-))r%numpyrFimage_processing_utilsrrrimage_transformsrrr r r image_utilsr r rrrrrrrrrrprocessing_utilsrutilsrrrutils.import_utilsr get_loggerr"rrrr-__all__r(r)r*rs(UU    -II5   H % U KC-KC\ " "r)