qi= d dl mZ ddlmZmZ G d de Z G d de Z G d d e Z G d
d e Zg d Z y
) )PreTrainedConfig )CONFIG_MAPPING
AutoConfigc L e Zd ZdZdZdZdeiZ d fd Z xZ S )EdgeTamVisionConfiga
This is the configuration class to store the configuration of a [`EdgeTamVisionModel`]. It is used to instantiate a SAM
vision encoder according to the specified arguments, defining the model architecture. Instantiating a configuration
defaults will yield a similar configuration to that of SAM 2.1 Hiera-tiny
[facebook/EdgeTAM](https://huggingface.co/facebook/EdgeTAM) architecture.
Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the
documentation from [`PreTrainedConfig`] for more information.
Args:
backbone_config (`Union[dict, "PreTrainedConfig"]`, *optional*, defaults to `timm/repvit_m1.dist_in1k`):
Configuration for the vision backbone. This is used to instantiate the backbone using
`AutoModel.from_config`.
backbone_channel_list (`List[int]`, *optional*, defaults to `[384, 192, 96, 48]`):
The list of channel dimensions for the backbone.
backbone_feature_sizes (`List[List[int]]`, *optional*, defaults to `[[256, 256], [128, 128], [64, 64]]`):
The spatial sizes of the feature maps from the backbone.
fpn_hidden_size (`int`, *optional*, defaults to 256):
The hidden dimension of the FPN.
fpn_kernel_size (`int`, *optional*, defaults to 1):
The kernel size for the convolutions in the neck.
fpn_stride (`int`, *optional*, defaults to 1):
The stride for the convolutions in the neck.
fpn_padding (`int`, *optional*, defaults to 0):
The padding for the convolutions in the neck.
fpn_top_down_levels (`List[int]`, *optional*, defaults to `[2, 3]`):
The levels for the top-down FPN connections.
num_feature_levels (`int`, *optional*, defaults to 3):
The number of feature levels from the FPN to use.
hidden_act (`str`, *optional*, defaults to `"gelu"`):
The non-linear activation function in the neck.
layer_norm_eps (`float`, *optional*, defaults to 1e-06):
The epsilon for the layer normalization.
initializer_range (`float`, *optional*, defaults to 0.02):
The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
vision_configedgetam_vision_modelbackbone_configc
|g dn|}| ddgddgddggn|}|ddgn|}t |t r'|j dd |d< t |d di |}n | t j
d dd
g d d
}|| _ || _ || _ || _ || _
|| _ || _ || _
| | _ |
| _ | | _ | | _ t% | L di |
y )N)i ` 0 @ r r
model_type timm_wrapperztimm/repvit_m1.dist_in1kT) r r )in_chans
features_only out_indices)
model_args )
isinstancedictgetr r from_pretrainedr backbone_channel_listbackbone_feature_sizesfpn_hidden_sizefpn_kernel_size
fpn_stride fpn_paddingfpn_top_down_levelsnum_feature_levels
hidden_actlayer_norm_epsinitializer_rangesuper__init__)selfr r r! r" r# r$ r% r& r' r( r) r* kwargs __class__s c/opt/pipecat/venv/lib/python3.12/site-packages/transformers/models/edgetam/configuration_edgetam.pyr, z EdgeTamVisionConfig.__init__E s 7L6S 2Yn 2H2Pc3Z#sb"X .Vl )<(Cq!fI\ ot ,,;,?,? n,]OL ) ,_\-J K ^o ^O
$ ( 8 8*()DQ]^ O
/ &;"&<# . . $ "4 $ ,!2
"6" ) NNNr r r r Nr geluư>{Gz?)
__name__
__module__ __qualname____doc__base_config_keyr r sub_configsr,
__classcell__r/ s @r0 r r sQ $L &O'J:K " # .# .#r1 r c 8 e Zd ZdZdZ d fd Z xZS )EdgeTamPromptEncoderConfigaB
This is the configuration class to store the configuration of a [`EdgeTamPromptEncoder`]. The [`EdgeTamPromptEncoder`]
module is used to encode the input 2D points and bounding boxes.
Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the
documentation from [`PreTrainedConfig`] for more information.
Args:
hidden_size (`int`, *optional*, defaults to 256):
Dimensionality of the hidden states.
image_size (`int`, *optional*, defaults to 1024):
The expected output resolution of the image.
patch_size (`int`, *optional*, defaults to 16):
The size (resolution) of each patch.
mask_input_channels (`int`, *optional*, defaults to 16):
The number of channels to be fed to the `MaskDecoder` module.
num_point_embeddings (`int`, *optional*, defaults to 4):
The number of point embeddings to be used.
hidden_act (`str`, *optional*, defaults to `"gelu"`):
The non-linear activation function in the encoder and pooler.
layer_norm_eps (`float`, *optional*, defaults to 1e-06):
The epsilon used by the layer normalization layers.
scale (`float`, *optional*, defaults to 1):
The scale factor for the prompt encoder.
prompt_encoder_configc t
| di | || _ || _ || _ || _ || _ || _ || _ || _ y Nr )
r+ r, hidden_size
image_size
patch_sizemask_input_channelsnum_point_embeddingsr( r) scale) r- rB rC rD rE rF r( r) rG r. r/ s r0 r, z#EdgeTamPromptEncoderConfig.__init__ sQ "6"&$$#6 $8!$ ,
r1 )r i rH r2 r3 r r5 r6 r7 r8 r9 r, r; r< s @r0 r> r> v s3 4 .O r1 r> c @ e Zd ZdZdZ d fd Z xZS )EdgeTamMaskDecoderConfiga
This is the configuration class to store the configuration of a [`EdgeTamMaskDecoder`]. It is used to instantiate a EDGETAM
memory encoder according to the specified arguments, defining the model architecture.
Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the
documentation from [`PreTrainedConfig`] for more information.
Args:
hidden_size (`int`, *optional*, defaults to 256):
Dimensionality of the hidden states.
hidden_act (`str`, *optional*, defaults to `"gelu"`):
The non-linear activation function in the EDGETAM mask decoder.
mlp_dim (`int`, *optional*, defaults to 2048):
The dimension of the MLP in the two-way transformer.
num_hidden_layers (`int`, *optional*, defaults to 2):
The number of hidden layers in the two-way transformer.
num_attention_heads (`int`, *optional*, defaults to 8):
The number of attention heads in the two-way transformer.
attention_downsample_rate (`int`, *optional*, defaults to 2):
The downsample rate for the attention layers.
num_multimask_outputs (`int`, *optional*, defaults to 3):
The number of multimask outputs.
iou_head_depth (`int`, *optional*, defaults to 3):
The depth of the IoU head.
iou_head_hidden_dim (`int`, *optional*, defaults to 256):
The hidden dimension of the IoU head.
dynamic_multimask_via_stability (`bool`, *optional*, defaults to `True`):
Whether to use dynamic multimask via stability.
dynamic_multimask_stability_delta (`float`, *optional*, defaults to 0.05):
The stability delta for the dynamic multimask.
dynamic_multimask_stability_thresh (`float`, *optional*, defaults to 0.98):
The stability threshold for the dynamic multimask.
mask_decoder_configc
t | di |
|| _ || _ || _ || _ | | _ |
| _ | | _ | | _ || _
|| _ || _ || _ || _
y rA )r+ r, rB num_multimask_outputsr( iou_head_depthiou_head_hidden_dim dynamic_multimask_via_stability!dynamic_multimask_stability_delta"dynamic_multimask_stability_threshnum_hidden_layersnum_attention_headsmlp_dimattention_downsample_rate)r- rB r( rW rU rV rX rO rP rQ rR rS rT r. r/ s r0 r, z!EdgeTamMaskDecoderConfig.__init__ s} "6"&%:"$ ,#6 /N,1R.2T/ "3  )B&r1 ) r r2 i r r r r r Tg?g\(\?rJ r< s @r0 rL rL sB !F ,O "# (,*.+/ C Cr1 rL c < e Zd ZdZdZeeedZ d fd Z xZ
S )
EdgeTamConfiga
[`EdgeTamConfig`] is the configuration class to store the configuration of a [`EdgeTamModel`]. It is used to instantiate a
EDGETAM model according to the specified arguments, defining the memory attention, memory encoder, and image encoder
configs. Instantiating a configuration defaults will yield a similar configuration to that of the SAM 2.1 Hiera-tiny
[facebook/edgetam.1-hiera-tiny](https://huggingface.co/facebook/edgetam.1-hiera-tiny) architecture.
Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the
documentation from [`PreTrainedConfig`] for more information.
EdgeTAM checkpoints with `model_type="edgetam_video"` are compatible with `EdgeTamModel` since the video variant
weights are a superset of the image-only model weights. You may see a warning about model type mismatch when
loading such checkpoints, which can be safely ignored in this case.
Args:
vision_config (Union[`dict`, `EdgeTamVisionConfig`], *optional*):
Dictionary of configuration options used to initialize [`EdgeTamVisionConfig`].
prompt_encoder_config (Union[`dict`, `EdgeTamPromptEncoderConfig`], *optional*):
Dictionary of configuration options used to initialize [`EdgeTamPromptEncoderConfig`].
mask_decoder_config (Union[`dict`, `EdgeTamMaskDecoderConfig`], *optional*):
Dictionary of configuration options used to initialize [`EdgeTamMaskDecoderConfig`].
initializer_range (`float`, *optional*, defaults to 0.02):
Standard deviation for parameter initialization.
Example:
```python
>>> from transformers import (
... EdgeTamVisionConfig,
... EdgeTamPromptEncoderConfig,
... EdgeTamMaskDecoderConfig,
... EdgeTamModel,
... )
>>> # Initializing a EdgeTamConfig with `"facebook/edgetam.1_hiera_tiny"` style configuration
>>> configuration = EdgeTamConfig()
>>> # Initializing a EdgeTamModel (with random weights) from the `"facebook/edgetam.1_hiera_tiny"` style configuration
>>> model = EdgeTamModel(configuration)
>>> # Accessing the model configuration
>>> configuration = model.config
>>> # We can also initialize a EdgeTamConfig from a EdgeTamVisionConfig, EdgeTamPromptEncoderConfig, and EdgeTamMaskDecoderConfig
>>> # Initializing EDGETAM vision encoder, memory attention, and memory encoder configurations
>>> vision_config = EdgeTamVisionConfig()
>>> prompt_encoder_config = EdgeTamPromptEncoderConfig()
>>> mask_decoder_config = EdgeTamMaskDecoderConfig()
>>> config = EdgeTamConfig(vision_config, prompt_encoder_config, mask_decoder_config)
```
edgetam)r r? rM c ||ni }||ni }||ni }t |t r&|j dd |d< t |d di |}t |t r|j }t |t r|j }|| _ t di || _ t
di || _ || _
t | 0 di | y )Nr r
r )
r r r r r> to_dictrL r r? rM r* r+ r, )r- r r? rM r* r. r/ s r0 r, zEdgeTamConfig.__init__3 s *7)B
9N9Z 5`b 5H5T 1Z\ mT **7*;*;LJ`*aM, ' *= +F G X- XM +-G H$9$A$A$C ! )+C D"5"="="? *%?%XBW%X"#;#R>Q#R !2
"6"r1 )NNNr4 ) r5 r6 r7 r8 r r r> rL r: r, r; r< s @r0 r[ r[ s8 6p J#!; 7K " # #r1 r[ )r[ r r> rL N)
configuration_utilsr autor r r r> rL r[ __all__r r1 r0 rb sU ( 4 -[#* [#|1!1 1hFC / FCRY#$ Y#x mr1