towhee.models.layers.cross_attention¶

Functions

`apply_rotary_pos_emb`
`rotate_half`

Classes

`CrossAttention`	cross attention - using multi-query + one-headed key / values as in PaLM w/ optional parallel feedforward.
`LayerNorm`
`Residual`
`RotaryEmbedding`