即梦AI肢体与动态
#肢体与动态
中文翻译
修复 Seedance 2.0 中舞蹈准确性的深度图技巧
给模型一段有人跳舞的视频,它会试图解读一切——人物、服装、灯光、房间,以及其中某处的动作。给它一张深度图,就只剩下动作可供解读了。
大多数试图将舞蹈迁移到角色上的创作者会直接引用源素材,然后纳闷为什么编舞会偏移。问题不在于模型——而在于你给了它十个变量,而你只想要一个。
以下是工作流程
1. 先在 GPT Image 2 中锁定角色参考——面部、体型、服装,这样身份就能独立于所施加的任何动作而保持稳定
2. 将源舞蹈素材转换为深度图,而不是使用原始视频——这会完全剥离原表演者的外貌、服装和环境
3. 将深度图作为动作参考输入,将角色设定图作为身份参考输入——两个独立输入各司其职,而不是一个输入试图兼顾两者
5. 让深度图只承载空间运动——模型接收身体位置和动量,没有关于谁在移动或他们长什么样的竞争信息
6. 始终保持角色和动作输入相互隔离——一旦你将外貌数据混入动作参考,模型就会开始在两个身份之间进行权衡
为什么这有效
• 原始素材一次性把一切传给模型——表演者、服装、房间、灯光——编舞要与所有这些争夺注意力
• 深度图是纯空间信息,因此唯一需要迁移的就是运动
• 将身份与动作分离意味着角色可以保持锁定,同时舞蹈保持准确——通常你是在两者之间做取舍
• 准确性的提升不是模型变得更好了,而是模型需要做的决定更少了
使用场景:
⁃ 将舞蹈和编舞迁移到原创角色上
⁃ 无需动作捕捉的动作捕捉式工作流程
⁃ 任何需要让特定动作完整保留的序列
⁃ 基于现有表演素材构建的角色展示内容
角色设定图回答的是谁在跳舞。深度图回答的是怎么跳——而将这两个问题分开,就是整个技巧的关键。
原帖
THE DEPTH MAP TRICK THAT FIXED DANCE ACCURACY IN SEEDANCE 2.0
Feed the model a video of someone dancing and it tries to interpret everything- the person, the clothes, the lighting, the room, and somewhere in there, the movement. Feed it a depth map and there's nothing left to interpret but the motion.
Most creators trying to transfer a dance to a character reference the source footage directly, then wonder why the choreography drifts. The problem isn't the model - it's that you handed it ten variables when you only wanted one.
Here's the workflow
1. Lock the character reference in GPT Image 2 first -face, build, costume, so identity holds independently of whatever motion gets applied to it
2. Convert the source dance footage into a depth map instead of using the raw video -this strips out the original performer's appearance, clothing, and environment entirely
3. Feed the depth map as the motion reference and the character sheet as the identity reference- two separate inputs doing two separate jobs, not one input trying to do both
5. Let the depth map carry only spatial movement -the model receives body position and momentum with no competing information about who's moving or what they look like
6. Keep the character and motion inputs isolated throughout - the moment you mix appearance data into the motion reference, the model starts negotiating between two identities
Why this works
• Raw footage passes the model everything at once- performer, wardrobe, room, lighting -and the choreography competes with all of it for attention
• A depth map is pure spatial information, so the only thing left to transfer is movement
• Separating identity from motion means the character can stay locked while the dance stays accurate - normally you're trading one for the other
• The accuracy gain isn't the model getting better, it's the model getting fewer decisions to make
Use cases:
⁃ Dance and choreography transfer onto original characters
⁃ Motion capture-style workflows without motion capture
⁃ Any sequence where a specific movement needs to survive intact
⁃ Character showcase content built on existing performance footage
The character sheet answers who's dancing. The depth map answers how - and keeping those two questions separate is the whole trick.
