There's a brand new tool that is going to change cinematography and VFX forever, and a new tool that they are calling Sora for Music is here, and the results are absolutely Incredible.

This is your AI film news of the week.

We're going to kick things off by talking about a brand new tool that kind of flew under the radar, but I really do believe it has the power to transform the way we create Films.

The tool is called Fizz Avatar, and basically what happens is, the tool allows you to record a scene with multiple cameras at multiple angles.

Then you as the editor are able to go in and change the camera angle in post, you can change the close of your subjects, and you even have the ability to relight your scene.

Essentially this converts a normal production into an animation workflow.

This has huge implications, because pretty soon there will be simply multiple cameras on set recording an environment, and your actors will basically act out the scene as if It's similar to a play.

Then having all of that coverage, you can go into post and reframe your scene.

Is this tool screen ready in the current state?

No, it is not, but pretty soon as this technology progresses, the line between animation and live action and gaming is going to become increasingly blurred.

We also came across an incredible AI music tool this last week that blows all other AI music tools out of the water.

It's called Udio, and it is making a splash.

The tool really speaks for itself.

I want to share a few examples that I came across.

This first example is called Carolina Oh, let's give it a quick listen.

The results from this tool are absolutely amazing.

That sounds like an incredibly well-crafted song from a legitimate real-world artist.

I want to show you another example.

It can do really incredible covers as well.

It did a House of the Rising Sun cover in the style of gospel music.

Let's take a quick listen.

And finally, I have to show you the best song that's come from Udio.

It is Dune, the Broadway musical.

I really think that they're onto something here.

Maybe we'll have a Dune musical in the near future.

Let's take a quick listen.

Oh, it's so good.

Well, Timothy Chalamet, hopefully after they get done recording the new Dune movie, we can do the musical version because I think that would be absolutely fantastic.

As you can see, Udio's potential is absolutely mind-blowing and the effects that this type of technology will have on the music industry cannot be overstated, but there's Actually quite a few features that are specifically really interesting for filmmakers.

Let me show you how to use this tool.

Basically, you can go over to the Udio website.

Udio is currently in beta.

There are some bugs with the website, so you may run into some issues, but they give you a ton of generations for free.

You can click the link below this video to find the tool.

And to create a song, all you have to do is go to the prompt box at the top and type in a prompt.

For our example, we'll say a cinematic soundtrack that blends hope with melancholy.

And they do allow you to click on certain tags here if you want to add more tags.

The cool thing is on their explore page, you can actually click into any song and see the tags that they use, which is really helpful.

It's very similar to the Midjourney explore page, which is really, really helpful.

Again, you do have the ability to add in your own custom lyrics, which is really, really awesome.

I think the O'Carolina song that we looked at had custom lyrics.

You actually have the ability to set tags for if you want the song to have a verse, a pre-chorus, a chorus, and you can identify which parts of the songs can incorporate Certain lyrics, which makes it really helpful compared to Suno.

And of course, they have an instrumental button, which is really helpful if you're using music that will go into a film.

And all you have to do is go ahead and click create.

The songs take a few minutes to create, Suno at this point is much faster than Udio, but I highly recommend going to the Udio website and clicking around and listening to Some of the staff picks and trending songs from their website.

And on that note, that brings us to our game of the week.

This week's game is super simple.

I'm going to play you two songs.

I want you to tell me which one was created in Udio and which one was created in Suno.

We will randomly select one of the winning answers from the comments of this video.

The winner will receive some free merch from the Curious Refuge merch store.

Okay, so let's listen to song number one.

Alright, cheers to you, Irvin.

That was song number one.

Let's listen to song number two.

Well, cheers to you, Irvin.

Happy birthday.

I also came across a new white paper this last week that really does outline the future of animation.

Basically, the idea is that you can take a Gaussian splat and animate the scene through text prompts.

In their example, they have a scene that they scanned, and basically through typing in a prompt, they are able to animate a vase and some flowers in that vase in a 3D world.

And this technology can do even more.

There's another example on the website where they have a Lego toy, and they basically say they want to turn it into a pile of sand.

You type in the prompt, and it basically inherits the physics and turns into sand.

They also have the ability to outpaint and remove objects from a 3D world with just text.

They have this mind-blowing example on the website where they basically ask the tool to remove a chair, but the ball still inherits all of the physics that would be present In a ball.

The chair is removed, and the ball starts bouncing. And so that is going to have huge implications for the future of gaming and also for set design.

Pretty soon, similar to Minority Report, which pops up again and again, you will be able to look around an augmented reality experience and change certain components, and The physics will actually be realistic.

Is this tool screen-ready at this point?

I would say no, but it's very easy to see that with the current rate of progression with artificial intelligence, tools like this will more than likely be hitting the market In the next one to two years.

Have you ever tried to create a time-lapse using AI tools?

You probably have been really disappointed.

AI is not really good at doing time-lapse.

For example, earlier I typed in a time-lapse of flowers growing inside of Runway, and we basically just have flowers waving in the breeze.

It looks amazing, but it's not really a time-lapse.

And there's even this example from Marques Brownlee, who basically asked Sora to create a time-lapse of a 3D printer, and again, it doesn't really look like a time-lapse.

It just looks like live-action footage with the physics being all wrong.

In light of all these problems, some researchers put together a tool called Magic Time-L Time-Lapse, and it basically allows you to create really interesting time-lapses From a simple prompt.

There are some really interesting examples over on their website.

For example, we have bean sprouts growing and maturing from seeds.

Looks pretty realistic.

We have dough that swells and browns in the oven.

Looks pretty delicious.

We also have a bud transforming into a flower, and it even inherits some anime styles.

For example, we have an ice cube melting and a time-lapse of a delicate pink plum blossom.

I didn't know a plum blossom looked like that, but there you go.

The reason why these results look so good is the researchers trained their data on time-lapse A lot of times, these video generation models are trained on a ton of different Data, but if you want specific video results, sometimes it's better to train a custom model so that the results are more in line with what you're looking for.

Some researchers at Google came out with a new white paper that looks like it could change the way that we do video interpolation forever.

Don't get scared by the word interpolation.

All it really means is figuring out what happens between certain key frames.

It's where we get the term key frame, and interpolation is basically the process of going from one frame to the other.

Interpolation is most prominent typically when you want to slow down footage.

Let's say you want to take footage that was shot in 24 frames per second, and you want to change it to 120 frames per second.

Well, in that case, you want to use a tool like Topaz Video, which handles interpolation between frames.

But let's say you only have a few images.

How would you create a moving video if you only have three images to work with?

Well, that's where this tool really shines.

It's called Video Interpolation with Diffusion Models, and basically all you have to do is upload a few images, and it will create video that interpolates between those Images.

There are a ton of really interesting examples on their website.

We have this video of a camel walking, and again, look at the reference images.

It's only a few reference images, and it completely interprets between those images.

We have this video of a woman walking, this video of a horse galloping, this video of a motorcycle, and this guy playing tennis.

As you can see, this is in a simple research phase at this point, but pretty soon, you'll be able to upload a bunch of images, and video will interpret between those images.

It's going to dramatically change the way that we recreate scenes and moments in time.

And of course, this looks like it could have huge implications for documentary filmmaking.

And speaking of Google, they also came out this last week with the ability for their enterprise customers to create text to live image.

Basically they're saying you can type in text and get a short AI video in return.

The generations are really limited at this point.

They're only 24 frames per second at 360p for 4 seconds, but it really does show that Google is really trying to be in competition with other AI video tools like Runway.

If you've ever played around with Korea's live AI image generator tool, then you know it is super fun to play with.

Well, they actually came out with a new stylization feature this last week that is awesome.

Let me show you how to use it.

To use the new feature, just go over to the Korea website and go to real time generation.

You want to make sure you're in the HD mode here, and we'll go ahead and clear our canvas.

And I'm going to quickly change the background, we'll just change it to, let's say a light gray here, and we'll type in our prompt, a 3D model of a dog.

It's time to play around with the canvas.

It's super fun.

We'll just select kind of some, like a darker brown color, just making a dog here.

And it's so forgiving, it's hilariously forgiving.

We'll just like make a rough drawing of a dog, click away, and look at that, we now have a dog.

This is a short haired dog.

Let's say that I really wanted a fuzzy dog, and I was really having trouble dialing it in.

Well, with this new stylized feature, you actually can customize your real time generations really easy.

I have this image of this kind of fuzzy creature, this is just a Midjourney image, and all you have to do is drag and drop the image into that image icon, and we'll drop it In there and we can dial it in.

We'll just dial it in about like that, maybe to about, we'll say, eight.

And if we compare and contrast them to our original image, this is our original on the left, this is our real time generation on the right.

The cool thing is we can now go in and completely change this drawing, and it will still inherit the style.

For example, if we delete that generation and go in and adjust our brush size, let's say that I want our dog to basically be a close up.

We'll just design like this, like big round, like this, snout here, about like that, and click away.

And oh no, it looks like a cursed Furby, we'll fix this, no worries at all.

And oh, it's really cute, it's like an interesting Pokemon character.

We'll make sure we add the nose here.

We have our dog, he's looking in the right direction, I think we should give him some regular eyes here, so we'll just like quickly draw in some eyes, about like that, and Oh no, he's cursed.

We'll give him some pupils here, here we go, about like that, oh no, it's so funny.

Oh my gosh, actually you know what, I love this, I think he looks great.

Obviously you have the ability to go in, you can finesse and fine tune anyway you want, but actually I think we may have found just a new mascot for Curious Refuge.

Pika Labs is also hosting a 72 hour film competition in collaboration with ElevenLabs.

The contest is going to run from April 12th to April 14th, and the winner will actually get $2,000.

They have $500 prizes for runners up when it comes to visuals, sound, narrative, and experimentation, and they said that the top winner gets a very special prize.

They weren't specific, so I can only guess that it is a diamond encrusted Pika character, and it turns out that a Pika is actually an animal and not just Pikachu.

And another cool thing, Dave Clark, the instructor of our AI advertising course, is actually going to be one of the judges for this competition alongside Danny Boyle, who Directed 28 Days Later and Slumdog Millionaire.

You'll find a link below this video to check out that competition.

And speaking of competition, our NAB party is right around the corner.

On Monday, April 15th at 8pm, we are hosting an epic party at the HyperX Arena in Las Vegas.

This event is being hosted in collaboration with a peer group from the TV Academy, and we also have incredible sponsorships from Puget Systems, Fabric, and Davant.

The event will start out as a party and transition into the world's first creative AI eSports tournament.

We have 16 competitors who will showcase their skills and compete for an Apple Vision Pro.

We'll also have AI generated cocktails at the event, along with an AI photo booth.

It's going to be really fun, and I have a quick little bonus, anybody that is watching this video who wants to attend the event.

If you use the code WEBSHOW50 at checkout, you'll get 50% off your ticket.

You'll find a link below this video, we can't wait to see you at the event.

Our research team also came out with the ability to add style transfers to your footage this last week, and the quality is much better than some of the other video style Transfer tools that we've had up to this point.

It's basically like runway gen 1, but the style is much more consistent.

For example, the other day I was working on the countdown timers for our AI creative competition, and I wanted to change this countdown timer into a different style.

However, when I ran the countdown timer through runway, it basically gave me this, which is completely impossible to read, and every now and then a frog pops up, which is delightful, but not really helpful.

The tool is free to use, it is limited at this point, but of course, like so many other tools, it is only going to get better from here.

All you have to do is upload a video clip, it is limited right now to 2 seconds, so it's really just an experimental tool.

And after you upload your video, it is then time to edit the image.

You can type in whatever prompt you want.

In this example, we have a woman in the style of a Vincent Bango painting.

After you've edited the video to your liking, you can go down to the video editing stage, type in your prompt, and click run video editing.

Here's a quick example of our video in action.

You can see that it did a really good job of inheriting the style and the composition from the reference footage.

I think this tool has really big implications for the future of visual effects as well.

For example, we have this video clip of this couple, and let's say that we actually want it to be snowing.

Well, all you have to do is add in a snow overlay, or you could paint in the snow that you want to see in the scene, and then you just go to the video editing stage, type in Your prompt, and click run video editing.

And here's a quick example of that footage.

You can see that it's now snowing in the scene.

As you can see, this is laying a blueprint for the future of visual effects.

Pretty soon you'll be able to drag and drop assets or images into your scene, and AI will bring them to life and composite them all together.

I also came across a really interesting workflow example from Akshat Sharma this last week, where basically he took a Gaussian splat scanned scene and brought it into Unreal Engine.

This has huge implications because you can basically take that scan and create a 3D world, not only for video games, but also for virtual production screens and eventually Live action footage.

Akshat Sharma's team this last week also came out with the ability to stylize a 3D scanned world.

The results are really incredible.

Basically whenever you take a scan, you are able to type in a prompt and change the scene that the scan lives in.

There are some really amazing examples over on their website.

We have a few examples of changing a scene from a daylight scene to a sunset scene.

We also have the ability to change it from summertime to a scene where it is snowing and so many more things.

In music news, the team at Spotify came out with AI generated playlists.

They're only available to people in Australia and the UK at this time, but it does look really interesting.

The thing that makes these playlists really unique is the fact that you can use natural language and emotion to create the playlist.

Instead of typing in, I want some studying music for focus, you can be really specific with your life and your emotions and it will give you a playlist for that.

For example, you can say I want sad music for painting drying flowers or tracks for horse riding into a sunset.

And the cool thing is it's a multimodal experience.

As you are using the AI generated playlist, you can basically say I want more pop music or I want it to be less upbeat and the playlist will adjust as if you're actually Talking to a DJ.

We also came across a really interesting experiment using Google Gemini 1.5 Pro this last week from Bilawul and it really blew our minds.

Basically Bilawul uploaded a video file that he was going to upload to his YouTube.

He then asked it to create the video chapters for his video and it did it automatically.

If you share a lot of videos online, whether it's on social media or YouTube or Vimeo, this type of technology is an absolute game changer.

You can simply upload the video and have AI create the descriptions and chapters for your videos.

We also came across an interesting post this last week from Nick St. Pierre where basically he was reflecting on the fact that Midjourney has been around for about two years Now.

And he basically compared and contrasted day one of Midjourney versus today.

We wanted to do the exact same thing.

First up we have a young Japanese woman smiling.

Midjourney 1 looks absolutely cursed.

Midjourney version 6 looks basically photorealistic.

We have a glamour street medium format photography shot.

Midjourney version 1 looks terrible.

Midjourney 6 incredible.

We have this beautiful shot of a landscape.

Look at the reflections in the water.

It looks super realistic.

We have this cheeseburger and I think the Midjourney version 1 tried to create an onion based cheeseburger, which does not sound very good, but Midjourney 6 looks super Realistic.

We have a woman's necklace as a sunflower.

A woman standing at the beach alone in the style of Studio Ghibli.

And finally, I had to do a double take on this one.

We have a RABBIT, a porcupine, two cats, and a wizard having a tea party in a 90s animated TV series style aesthetic.

Midjourney version 1 created this beautiful abstract image that is kind of cool, but not anything like the prompt.

And Midjourney 6, while it didn't include all of the characters, the quality of the overall image had me doing a double take because it looks really amazing.

And that brings us to our AI Films of the Week.

We're going to kick things off with this Nike spec ad created by Adam Castlebrand.

The spec ad feels like a real ad.

You can really see how Adam has video editing skills to help bring everything together.

The pacing is really nice, and it's one of the better examples of an AI commercial that I've seen up to this point.

Next up, we have Warriors by Lay Atta.

It's basically a short film that has humans against robots, and he does an amazing job with world building.

Some of the VFX shots inside of this film are really mind-blowing, and it features a lot of interesting movement.

The sound design is really on point, and it is one of the better AI films that we've seen over the last few weeks.

And our final film of the week comes from Dave Clark.

He created a spec trailer for a Gundam movie where he basically created this epic Hollywood trailer.

He actually was kind enough to break down how he put this trailer together in our recent office hours at Curious Refuge.

If you're a Curious Refuge student, be sure to log on and you can see how Dave put together this project.

Thank you so much for watching this week's episode of AI Film News.

Of course, if you want to get AI Film News delivered directly to your inbox, you can subscribe to our newsletter by visiting CuriousRefuge.com, and if you enjoyed this Video, be sure to like and subscribe.

And of course, if you can make it out to our party at NAB, we would love to say hello.

Thank you so much for watching this week's episode of AI Film News.

We'll see you next week.