Adobe is working on video generator AI

Like many of today's AI video production tools, Adobe's model builds footage from scratch and leverages three new features in Premiere Pro: object and detail addition, object removal, and production extension.
Add Object allows users to select a section of a video clip and enter a prompt to insert objects into that section.
Object Removal removes a portion of objects and images from clips, such as boom microphones or coffee cups in the background of a photo. In the case of video expansion, it adds a few frames to the beginning or end of a clip.
Video enhancement is not about creating full scenes, but adding frames to sync with the soundtrack.

To combat the deepfakes that inevitably pop up around AI-powered tools, Adobe says it's bringing Content Credentials, which is metadata to identify AI-generated media, to Premiere. Premier shows not only what content was generated with AI, but also which AI model was used to generate it.

Adobe also pays photographers and artists on its media platform, Adobe Stock, up to $120 to submit short video clips to teach its video production model. The company pays contributors an annual bonus based on the volume of content in the warehouse and how it is used.

This marks an approach in stark contrast to that of generative AI video rivals like OpenAI, which are said to have scraped publicly available web data — including YouTube videos — to train their models. YouTube CEO Neil Mohan stated that using YouTube videos to teach OpenAI's text-to-video generator would be a violation of the platform's terms of service.

Firefly's image generation models have so far been widely derided as weak and incomplete compared to Midjourney, OpenAI's DALL-E 3, and other competing tools, as well as lacking a time frame in video release.

Adobe says it's working with OpenAI on ways to bring Sora into Premiere's workflow. Given the Hollywood AI startup's recent advances, the OpenAI tie-in seems logical. Adobe will be ready to cooperate with others in the future.