Video-to-music generation
Exploring how video content can guide music generation, with an interest in extending the approach to longer videos.
Overview
Exploring how video content can guide music generation, with an interest in extending the approach to longer videos.
Approach
Reproducing a DiffRhythm-based baseline and investigating model improvements and evaluation for video-conditioned music generation.
My contribution
Leading the project, including baseline reproduction, experimental iteration, and the design of a long-video scoring framework.
Outcome
Ongoing experiments. Quantitative results and audiovisual examples will accompany a future update.