towhee.models.perceiver.create_self_attention_block.create_self_attention_block¶
- towhee.models.perceiver.create_self_attention_block.create_self_attention_block(num_layers: int, num_channels: int, num_heads: int, dropout: float, activation_checkpoint: bool = False)[source]¶
Self attention block for Perceiver https://arxiv.org/pdf/2103.03206.pdf. :param num_layers: Number of layers. :type num_layers: int :param num_channels: Number of channels. :type num_channels: int :param num_heads: Number of parallel attention heads. :type num_heads: int :param dropout: Dropout probability. :type dropout: nn.Module :param activation_checkpoint: Use activation checkpointing. :type activation_checkpoint: bool
