Firehawk52-backup/llama
28
0
Fork 0
mirror of https://github.com/meta-llama/llama.git synced 2026-01-15 08:22:55 -03:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update transformer mask comment

Browse Source 
Update names for consistency with code

Co-authored-by: ruanslv <ruanslv@gmail.com>
This commit is contained in:
Alex
2023-11-13 13:41:06 -06:00
committed by GitHub
parent e9077bd241
commit 6b3154bfbb
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
llama/model.py
View File

@@ -481,8 +481,8 @@ class Transformer(nn.Module):
# When performing key-value caching, we compute the attention scores
# only for the new sequence. Thus, the matrix of scores is of size
# (seq_len, total_len), and the only masked entries are (i, j) for
# j > cached_len + i, since row i corresponds to token cached_len + i.
# (seqlen, cache_len + seqlen), and the only masked entries are (i, j) for
# j > cache_len + i, since row i corresponds to token cache_len + i.
mask = torch.hstack([
torch.zeros((seqlen, start_pos), device=tokens.device),
mask