Skip to content

Commit 6a312dd

Browse files
authored
【cherry-pick】add hunyuanvideo readme & fix bug in attention_processor.py (#1232) (#1274)
1 parent 38eaa73 commit 6a312dd

File tree

3 files changed

+28
-15
lines changed

3 files changed

+28
-15
lines changed
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# HunyuanVideo视频生成
2+
3+
HunyuanVideo是由腾讯开发的13B参数量的开源视频生成模型,能够生成高质量,高动态的视频。本仓库提供了HunyuanVideo的paddle实现,目前仅支持推理。
4+
5+
## 快速开始
6+
### 环境准备
7+
若曾使用PaddlePaddle主页build_paddle_env.sh脚本安装PaddlePaddle,请根据本身cuda版本手动更新版本[Installation](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html)
8+
9+
更新diffusers:在ppdiffusers目录下运行以下命令:
10+
```shell
11+
python install -e .
12+
```
13+
14+
## 推理示例
15+
16+
### 硬件要求
17+
* 硬件要求:至少要求50G以上显存
18+
19+
### 文本到视频
20+
21+
运行如下脚本,生成视频
22+
```shell
23+
python ppdiffusers/examples/inference/text_to_video_generation-hunyuan_video.py
24+
```

ppdiffusers/examples/inference/text_to_video_generation-hunyuan_video.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,10 @@
2929
transformer = transformer,
3030
text_encoder = text_encoder,
3131
tokenizer = tokenizer,
32-
paddle_dtype=paddle.float16)
32+
paddle_dtype=paddle.float16,
33+
map_location="cpu")
34+
pipe.vae.enable_tiling()
35+
pipe.vae.enable_slicing()
3336
prompt = 'A cat walks on the grass, realistic.'
3437
output = pipe(
3538
prompt=prompt,

ppdiffusers/ppdiffusers/models/transformer_hunyuan_video.py

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -139,9 +139,6 @@ def __call__(
139139
if attention_mask.dtype == paddle.bool:
140140
L, S = query.shape[-2], key.shape[-2]
141141
attention_mask_tmp = paddle.zeros([1,1,L, S], dtype=query.dtype)
142-
# attention_mask_tmp = paddle.zeros_like(attention_mask).to(query.dtype)
143-
# attention_mask_tmp[attention_mask==True] = 0
144-
# attention_mask_tmp[attention_mask==False] = float("-inf")
145142
attention_mask_tmp = attention_mask_tmp.masked_fill(attention_mask.logical_not(), float("-inf"))
146143
attention_mask = attention_mask_tmp
147144

@@ -153,17 +150,6 @@ def __call__(
153150
dropout_p=0.0,
154151
is_causal=False,
155152
).transpose([0,2,1,3])
156-
# hidden_states = query
157-
# import pdb;pdb.set_trace()
158-
159-
# hidden_states = scaled_dot_product_attention_paddle(
160-
# query=query,
161-
# key=key,
162-
# value=value,
163-
# attn_mask=attention_mask,
164-
# dropout_p=0.0,
165-
# is_causal=False,
166-
# )
167153

168154
hidden_states = hidden_states.transpose(
169155
perm=dim2perm(hidden_states.ndim, 1, 2)

0 commit comments

Comments
 (0)