I know the video's not very exciting yet, but bare with me!
It shows possibilities. Possibilities I don't think are very far off.
I've been wanting to use AI to create game assets for quite some time, but image generators have so far disappointed me with their attempts at sprite sheets.
But what if...what if you could make animations from AI video instead?
The State of AI Video
There are currently two main branches of AI video.
The first is the text-to-video or image-to-video generators such as Runway ML, Pika, and Final Frame. These aren't quite there yet in quality and are still bad with motion.
The second is video-to-video like AnimateDiff. This type handles motion much better and can change the style or people within a video. However, they also have a higher learning curve and require a video with the movement you want.
Imagine if you could take either of these, feed them an image, and get coherent walking (or more interesting) videos, easy peasy.
I don't think the first type of video makers are too far off, and I think they'll do it with greater ease than the second type.
So I couldn't help but wonder, once the technology is there, would we be able to take frames of these animations and use them as sprites for games?
A Substitute for Testing
I tried, honest, but I couldn't get a video of good enough quality from the AI video makers, so I decided to substitute the video with one of my own creation from Blender.
I'm a beginner in Blender, so I started off with something easy: a marble. I created the marble, created the animation, then rendered every other frame of that animation.
In the future I hope to be able to take a character concept from Dall-E 3 or Midjourney, ask an AI video generator for different motions then take frames from that video, but this will have to do for our workflow test.
Creating the Sprite Sheet
I've been unhappy with the background remover tools I've used in the past, so at first, I thought I might have to turn the frames into vectors to get a clean look.
Fortunately, before I committed too much time to vectorization, I asked X/Twitter for advice. Based on the recommendations of @botzero_net and @AIandDesign, I gave ClipDrop a chance.
I'm so happy I did. Look at how clean these marbles look!
Once I ran each frame through ClipDrop's Background Remover, I used Affinity Designer to combine them into one single sprite sheet. The top row is for horizontal movement, while the bottom row is for vertical.
Learning Godot
I had decided a long time ago that if I were ever to try to make a game, I'd use Godot. The primary reasons are its simplicity, small file size, and truly free price compared to other "free" engines like Unreal and Unity. I highly recommend looking into Godot if you're just starting out.
I followed a YouTube Video by Chris' Tutorials in order to get my marble moving and grooving as a sprite in a 2D setting. You can see the final result in the video at the top of the article.
Conclusion
In my opinion, this workflow worked very well. Outside of the hitch of not quite being able to get an AI video for it. Yet.
I'd be the first to admit that AI video is not my expertise. For all of you with more experience in this area: do you think you could currently create a video good enough for sprite animation?
If anyone manages to accomplish this, please let me know! I can just feel this coming around the corner and can't wait to see it happen!
コメント