Open-Sora v1.1.0

📍 Open-Sora 1.1 released

• 🌠 Model weights are available here. It is trained on 0s~15s, 144p to 720p, various aspect ratios videos. See our report 1.1 for more discussions.
• 🔧 Data processing pipeline v1.1 is released. An automatic processing pipeline from raw videos to (text, video clip) pairs is provided, including scene cutting, filtering(aesthetic, optical flow, OCR, etc.), captioning managing. With this tool, you can easily build your video dataset.
• ✅ Improved ST-DiT architecture includes rope positional encoding, qk norm, longer text length, etc.
• ✅ Support training with any resolution, aspect ratio, and duration (including images).
• ✅ Support image and video conditioning and video editing, and thus support animating images, connecting videos, etc.

Visit the Open-Sora Gallery to view more samples.

Source hpcaitech/Open-Sora · v1.1.0 · 2024-05-07 · upstream release

Mirrored verbatim for indexing. All rights belong to the upstream project.