This week, AI startups, apart from OpenAI, have been dedicated to their product roadmaps amidst the OpenAI chaos dominating the news.
Stability AI introduced Stable Video Diffusion, an AI model that generates videos by animating existing images, following its existing Stable Diffusion text-to-image model. Stable Video Diffusion is one of the few available video-generating models in open source and commercially.
However, it is currently in a “research preview” phase. Usage requires agreeing to terms outlining intended and non-intended applications.
Considering the history of similar AI research previews, concerns arise about potential misuse if the model circulates the dark web. Without a built-in content filter, there are worries about potential abuse.
Stable Video Diffusion consists of two models: SVD and SVD-XT. SVD transforms images into 576×1024 videos in 14 frames, while SVD-XT offers 24 frames, both capable of generating videos at a speed of three to 30 frames per second.
According to a whitepaper released with Stable Video Diffusion, SVD and SVD-XT were trained on a large dataset of videos and “fine-tuned” on a smaller set. The source of the training data is unclear, potentially leading to legal and ethical challenges around usage rights.
Image Credits: Stability AI
The models produce high-quality four-second clips, comparable to outputs from other companies.
However, there are limitations such as the inability to generate videos without motion, slow camera pans, text control, or render text legibly. Stability mentions the models’ adaptability for use cases like generating 360-degree views of objects.
Stability seeks to expand Stable Video Diffusion by building on and extending SVD and SVD-XT, including a “text-to-video” tool for web use with potential applications in various fields.
Recently, Stability AI secured $25 million through a convertible note, bringing its total raised to over $125 million. However, there are concerns around the company’s financial stability and unresolved issues, including the departure of key executives.
Image Credits: Stability AI
Amidst financial struggles and key personnel departures, the future of Stability AI and its video-generating model remains uncertain.
Source link
“`