towhee.models.swin_transformer.swin_transformer_blockΒΆ

Classes

SwinTransformerBlock

Swin Transformer Block. :param dim: Number of input channels. :type dim: int :param input_resolution: Input resulotion. :type input_resolution: tuple[int] :param num_heads: Number of attention heads. :type num_heads: int :param window_size: Window size. :type window_size: int :param shift_size: Shift size for SW-MSA. :type shift_size: int :param mlp_ratio: Ratio of mlp hidden dim to embedding dim. :type mlp_ratio: float :param qkv_bias: If True, add a learnable bias to query, key, value. Default: True :type qkv_bias: bool, optional :param drop: Dropout rate. Default: 0.0 :type drop: float, optional :param attn_drop: Attention dropout rate. Default: 0.0 :type attn_drop: float, optional :param drop_path: Stochastic depth rate. Default: 0.0 :type drop_path: float, optional :param act_layer: Activation layer. Default: nn.GELU :type act_layer: nn.Module, optional :param norm_layer: Normalization layer. Default: nn.LayerNorm pretrained_window_size (int): Window size in pre-training. :type norm_layer: nn.Module, optional.