Experimenting With Gigapixel AI for Video

I recently stumbled upon Gigapixel AI for Video, a new cloud-based upscaling service from the makers of Gigapixel AI, the software that I have used for my upscaling projects Remako Mod, DS9 Enhanced, and Half-Life: ReSrced. I decided to experiment a bit with it, and here are the results.

So what does it do?

If you know a little bit about AI upscaling, then you know that it's slow. This is somewhat doable for a graphics mod like Remako, which had about 5,000 images that needed upscaling. But if you want to video, then you need to upscale every frame. Upscaling an hour-long video means upscaling more than 100,000 images. That could take days, even on a high-end computer.

With Gigapixel for Video you upload a video, after which multiple ultra-high-end PCs do the upscaling for you. This is that 'power of the cloud' thing that everyone has been talking about in recent years.

How does it look?



When I learned about Gigapixel for Video, I knew exactly what I wanted to try out. This AI uspcaling method would be ideal for those late 90s/early 00s shows that weren't fully shot on film nor fully digital. Those would cost millions to remaster by hand, so there's a big chance that those won't be remastered any time soon. Perhaps AI upscaling can do the trick?

The service is paid, but luckily there is a free proof upscale feature that anyone can use. It is limited to 60 seconds of footage and only shows a comparison of the original and the upscaled version. For this experiment, it was good enough.

So I threw in a few TV shows from the late 90s/early 00s. The results you can see in the video above. In the rest of the article, I want to analyze the output and delve into the possibilities of this technology, as well as the pitfalls.

One sidenote: the resolution of the video is 4K. However, Gigapixel Video only goes up to 4x the resolution. This puts the output resolution of these videos between 2K and 4K. Since it was closer to 4K, I enlarged them in my video editing program to 4K.

Star Trek: Deep Space Nine

I have done DS9 manually with Gigapixel AI, so I have a good base of comparison. Using Gigapixel Video resulted in something that looks very good, but slightly less detailed compared to doing it manually. However, considering that I did this in about ten minutes, versus half a day doing the same video manually, I am impressed by the results.

This shot really shows off the increase of detail in Rom's skin. Click to enlarge.

Babylon 5

One of the most requested shows that I have been asked to upscale after I did DS9, this upscale shows that image quality is everything. The resolution is similar to DS9 but still, there are some weird artifacts that pop up with people moving in the mid-far distance. This is less of an issue with DS9. I think this is because the DVDRip I used had a bit more visual noise than the DS9 video file.

You look like you had too much plastic surgery, lady!

Star Trek: Voyager

Voyager is in almost the same situation as Deep Space Nine, as to why it isn't likely to get a real remaster. It would look great in HD, but it's not popular enough to justify the millions of dollars it would cost to redo the show, which was shot on a mix of film and video. Can AI upscaling come to the rescue?

Not with this source file. I have noticed that there aren't a lot of good DVDRips online of Voyager. Many are interlaced or relatively low quality. This is what we have here. The visual noise, like with Babylon 5, creates a weird effect when people move in the mid-far distance.

Are you okay Chakotay? Click to enlarge.
Some shots show promise though, such as this shot of a Species 8472 alien.

Honestly, this now looks like it was rerendered in 4K. Click to enlarge.
If I had a source file as good as the one I have for DS9, I could get much better results.

Buffy the Vampire Slayer

I haven't watched this in a while, but I read that the existing home video releases of this vampire show haven't been received very well by fans.

Lesson: interlaced videos don't work well with upscaling. Click to enlarge.
The results with this video are mixed. The source video has a lot of interlacing, which makes action-heavy sections of the video look almost the same as the original resolution. Static shots show more improvement though.

Click to enlarge.

Earth: Final Conflict

This show isn't as well known as these other ones, but it's a favorite from my childhood. This upscale attempt is the worst of them all. I think this is because the resolution of the source video is so low (576x320). Coupled with visual noise, this results in a lot of graphical artifacts and weird effects.

Click to enlarge.
There's one shot that comes out very nice though.

Click to enlarge.

What are the applications for this?

This was a quick dive into the Gigapixel Video service. Here are some lessons I learned from the above experiments:
  • The resolution can't be too low (640x480 or more).
  • You need good source files to make it look good. DVDRips will most often have a lot of visual noise due to compression (to shrink the file and save download time).
  • The video can't be interlaced, or else it won't look much better at all.
What could be the practical applications of this? How it is now, it's not yet the solution that can turn DVD-quality movies in Blu-Ray releases. However, this might be a good way to offer a higher-res video for streaming services like Netflix. If CBS puts their best looking source file for a DS9 episode into Gigapixel Video, I imagine the result would be convincing enough. Any imperfections would be covered up by the low bit-rate that the streaming services. It's not DS9 2.0, but DS9 1.5 would be an improvement too.

Let's take the example of Deep Space Nine and see how long it would take to upscale the entire 176-episode series, as well as how much it would cost.

When I did my experiments, these 1 min videos took no more than 10 mins. One 45-min episode would thus take about 7.5 hours. You can upscale multiple videos at a time, but even if this wasn't the case and you had to upscale each video in sequence, this would only take 55 days to do all 176 episodes.

In terms of time, it seems more than doable. What about the cost? This is still pretty cheap. I did the calculations and this is what it ended up as:
  • One 45-min episode would cost $50
  • A 26-episode season would cost $1,300
  • The entire 176-episode series would cost $8,800
This is quite doable for any production company. I imagine that it is a matter of time until a production company will use a service like this to upscale their old shows to a higher resolution.

What do you think Gigapixel for Video can be used for? Leave your comment below.


Comments

Zekker said…
I uploaded a 1.5 minute video for upscaling a few days ago. Still not done. Topaz does not give out any estimates for the time to completion. Is it hours, days, weeks? I think all users would like to know. What was your experience with turn around time?
CaptRobau said…
Five minutes tops. Try reuploading.
Jens Jansen said…
Just in case you didn't notice, they do have a standalone beta for download and 30days free trial. I'm currently trying it on a full episode of SG-1. But I think 4k as a goal is really stretching it. ;)
Input is a 720x576i @25Hz PAL DVD
I think doing a 4x stretch and then downscaling it to 720p is already a great advantage over the original DVD, but I don't have a 4k TV yet ;)
CaptRobau said…
Hi Jens, I noticed. Tried it out too. Not as fast as the cloud-based variant but at least you can do whole episodes.
Zim said…
I tried to upscale my lovely lovely show The Golden Girls but at the end it became to a mess :(