Tutorial: Upscaling Video With Topaz AI Gigapixel

In this tutorial, I will explain how to upscale videos from a low resolution to a higher resolution such as 1080p or 4K using neural networks.

This is an improved and streamlined version of the tutorial I posted previously. Instead of using multiple programs to do all of the steps, this tutorial makes do with only two programs: AI Gigapixel by Topaz Labs and FFmpeg.

This tutorial is written for Windows users. Both programs will work for Mac though.

Step 1. Download and install the programs

AI Gigapixel

AI Gigapixel

This program by Topaz Labs is a paid program, but it is one of the best methods for upscaling photos and video. It uses a neural network which analyzes thousands of photos to learn how details generally get lost when downscaling. The algorithm "learns" how to fill in the details in new images based on what is learned from the photos it was trained on. This machine learning allows Gigapixel to add new detail to these images.

There is a free 30-day trial available for AI Gigapixel, so you can try it out for yourself. After the trial has ended it normally costs $99.99 for a lifetime license. As an official affiliate, I can give you a 15% discount. This means it's only $84.99. To get the discount, enter the coupon code captrobau15 during checkout.

You can download the free trial or purchase AI Gigapixel here.

Installation is easy, just follow the installer's instructions.

FFmpeg

While Gigapixel helps you upscale your video, you will need FFmpeg to extract the frames and audio from the original and put the upscaled video back together. This is a command-line program, but I will guide you through all the commands below.

You can download FFmpeg for free here.

Extract the zip file. You will use the ffmpeg.exe later.

Step 2. Prepare your workspace

To make the upscaling process more efficient, I suggest you make an orderly workspace somewhere on your computer. I'm going to make a folder on my desktop called 'upscale-projects'. Make another folder in 'upscale-projects' for your first project. In my example, I will call it 'example', and I will be upscaling an episode of Star Trek: Deep Space Nine.

Within your project folder, make three folders:
  • frames-original
  • frames-upscaled
  • video
In the next couple of steps, we will fill those folders with audio, images, and videos.

Keep in mind that extracting an entire episode or movie requires quite a bit of free space on your hard drive. One 45 minute episode needs 29 GB for the original frames. If you upscale to 1080p that will add another 72 GB. If you upscale to 4K you will need around 180 GB for an episode's worth of frames. And finally, you will need about 1 GB or more for the exported episode. So make sure you have enough room on your PC.

Step 3. Export The Frames Of The Source Video

You will use FFmpeg to extract the individual frames of the source video. This is the tutorial I used,  which tells you about more options.

How to export the source video:
  1. Get your source video and put it into example/video
  2. Put ffmpeg.exe in the folder you also put your source video into.
  3. Open up command prompt (cmd)
  4. In cmd, navigate to the work folder. Do this by typing in: cd C:\Users\UserName\Desktop\upscale-projects\example\video. The part after cd depends on where your work folder is of course.
  5. Now you will give FFmpeg the command to export the frames. Type in:
ffmpeg -i example.mkv example%04d.png -hide_banner
 What does this all mean?
  • example.mkv is the source file in my case. Change it to the name and extension of your source video file.
  • example%04d tells FFmpeg how to name the frames (example0001.png, example0002, etc.). Adding text before the frame number helps to keep multiple video's worths of extracted frames organized.
  • .png is the extension of the exported file. PNG is bigger than JPG, but JPG creates extra artifacts that the upscaling program can't handle very well. So the lossless PNG format is preferred.
  • -hide_banner hides FFmpeg compilation information. Personal preference.
When you press enter on the filled in command, FFmpeg will get to work. This will take a while, depending on the length of the video and the original resolution.

Step 3. Upscale The Frames To 1080p/4K

You will use AI Gigapixel to upscale the frames from the SD original. There's a lot of things you can with Gigapixel, but in this tutorial, I will focus on the settings that are most suitable for most SD footage.

Once you've installed the program, open it up. Drag one of the exported frames to the middle of the program's window. You will then be shown something that looks like the screenshot below.

Click to enlarge
Below I will explain what all the settings mean:
  • You can base the upscale's resolution by scale, width or height. I chose height, because...
  • I want to make the upscaled image have the height of 4K footage, so I input 2160 pixels. Full HD has a height of 1080 pixels.
  • Suppress Noise and Remove Blur can improve the quality of the end result. This really depends on the project. Just try out everything to see what gives the best result for your project. I chose Low and Low, as it gave just a bit of smoothing/noise removal without going overboard.
  • I save to a Custom Folder and point it to upscale-projects\example\frames-upscaled. Ignore Prefix and Suffix.
  • For video it's best to output in JPG format. It is smaller and results in a fast upscale process. Since the original is PNG, you will need to choose Convert File Format: Yes. The few artifacts created by the JPG format at Maximum Quality won't be noticed when you later convert the frames to a video.
  • Keep Color Profile to Yes, as it's faster.
In the preview window, you can see the result of the upcoming upscale. Change the settings until you are happy with the results.

Once you have a working set of settings, you can start upscaling footage. Select a few hundred frames when first getting your footing with this whole workflow and drag and drop them in the window. Upscaling an entire episode can take a long time, so start small.

How many frames you can upscale in one go depends on your PC's specs. It can be 1000s, 10000s, etc. You'll just have to find out for yourself. How fast the entire upscaling process also depends on your PC. It could take a day/multiple days to convert an entire episode/movie depending on the length and the upscaled resolution (4K takes a lot longer than 1080p).

Step 4. Exporting The Audio Of The Video

While the frames are being upscaled in the background, you can go ahead and do other things. For example, ripping the audio from the original video file using FFmpeg. This is the tutorial I used for this part.

Perform the following steps:
  1. Open up command prompt (cmd)
  2. In cmd, navigate to the work folder. Do this by typing in: cd C:\Users\UserName\Desktop\upscale-projects\example\video.
  3. Now you will give FFmpeg the command to export the audio. Type in:
ffmpeg -i original-video.mkv -vn -acodec copy original-audio.wav
What does this mean?
  1. example.mkv is the source file in my case. Change it to the name and extension of your source video file.
  2. vn means no video.
  3. acodec copy tells FFmpeg to use the same audio stream that's already in there.
  4. original-audio.wav is the name of the exported file. I chose .wav as a file format, because it is the most generally supported lossless codec.
When you press enter on the filled in command, FFmpeg will get to work. It should take at most a few minutes.

Step 5. Combining The Frames And Audio Into A Video

Once AI Gigapixel is done upscaling the frames, you can combine these frames with the audio into a video. I used this tutorial for this section.
  1. Move the exported audio file to example/frames-upscaled
  2. Copy ffmpeg.exe into the example/frames-upscaled folder
  3. Open up command prompt (cmd)
  4. In cmd, navigate to the work folder. Do this by typing in: cd C:\Users\UserName\Desktop\upscale-projects\example\frames-upscaled. The part after cd depends on where your work folder is of course.
  5. Now you will give FFmpeg the command to combine the frames and the audio. Type in:
ffmpeg -r 29.970628 -f image2 -s 1440x1080 -i example%04d.jpg -i original-audio.wav -vcodec libx264 -crf 20 -pix_fmt yuv420p -acodec copy upscaled_video.mp4
What do these commands mean?

  1.  -r is the frame rate of the exported file. 29.970628 is a common fps for older American shows, but it could be 24, 30, or 60. It depends on the original video's frame rate.
  2. -s is the resolution. This should be the same as the upscaled images, in my case 1440x1080.
  3. example%04d.jpg is the way the frames you upscaled are named. Change example to the text you chose in step 3.
  4. original-audio.wav is the audio file you want to combine with the frames into a video.
  5. vcodec libx264 is the codec you're using. Libx264 is a widely supported codec, so a safe bet.
  6. -crf is the quality of the video. The lower number the higher the quality but the larger the file size, with a range of 15-25 a good trade-off between file size and quality.
  7. -pix_fmt determines the pixel format. I chose yuv420p because it is one of the most compatible pixel formats.
  8. -acodec copy tells FFmpeg to copy the audio from the input stream (the .wav file) to the output stream (the video)
  9. upscaled_video.mp4 is the name of the output video file.
Press enter to start generating the video file. This can take several minutes to complete, depending on the length of the video and the video quality (crf) chosen.

Once it's done, your upscaled video is ready to be watched, shared and enjoyed!

Interested in AI Gigapixel?

Did this tutorial interest you in Topaz's AI Gigapixel? There is a free 30-day trial available for AI Gigapixel, so try it out for yourself.

Normally costs $99.99 to purchase for a lifetime license. As an official affiliate, I can give you a 15% discount

This means it's only $84.99. To get the discount, enter the coupon code captrobau15 during checkout.

You can download the free trial or purchase AI Gigapixel here.























Comments

  1. I'm using MKVToolNix to take the audio track from the original file and put it into the upconverted file, however, the audio is out of sync in the new HD file. Any advice for correcting that?

    ReplyDelete
    Replies
    1. I keep getting an error saying "could find no file with path 'example%04d.jpg' and index in the range 0-4 example%04d.jpg: no such file or directory"

      Delete
    2. I had the same problem before I changed the video name to a single word without any accents (from sequĂȘncia to sequencia (portuguese)) so maybe it's the same for you

      Delete
  2. Hi could you pls give us a new update to FF7? :)

    ReplyDelete
  3. Thanks for the tutorial. I'm curious to know what's the specs of the computer you are using for Gigapixel. I know it's a gpu heavy application, but I have a gtx 1070 and the process is taking forever. 1.5 hours for a quarter of a 2min video lol. Guess I'm in needs of an upgrade if I decide to get serious into this.

    ReplyDelete
    Replies
    1. It is pretty slow on mine too. I have a RX470 GPU.

      Delete
    2. In this case I've noticed it's pretty GPU CLOCK intensive, as the gigapixel is taking the whole GPU clock available on my 1070ti but only about 2ish GB from its memory

      In this case nvidia tends to fare better as it customarily has better clock speeds, so I'd look for an upgrade that took that into account.

      My task manager also curiously (at least for me and I'm not specially tech savvy) accounts the work gigapixel is doing as "3D" so maybe the research into which would be the best GPU to upgrade to could take info from reccomendations for GPUs that render 3D well

      Delete
  4. I did this a few months back for a 20 min episode in SD. I used different tools to extract the frames and recompile them back to video, but same process (I used virtualdub2). extract frames, upscale, recompile.

    There were over 30,000 frames, and on my 2080ti it took nearly 40 hours. On import, any batch larger than a few thousand frames took a long time to load in to ai gigapixel, but eventually loads all the frames.

    If you want to see the result, check out here https://imgur.com/a/4Jxy3bb
    The whole episode is also up on tpb

    ReplyDelete
  5. i try to download gigapixel but dont work, it send a link for another program, isnt the program from the photos :(

    ReplyDelete
    Replies
    1. If you go to https://topazlabs.com/gigapixel-ai/ and click on Download Trial it should give you the right program.

      Delete
    2. It has had a slight graphical overhaul since I made this tutorial so that might be confusing.

      Delete
  6. Hi,

    I tryed Megapixel AI on an Anime scene and it seems not that good with drawed content.
    Much Artefact in the middle of the scene.
    With wifu2x i got an better result.

    I tryed also dubble scaling. First wifu2x and second Gigapixel AI.
    http://www.framecompare.com/image-compare/screenshotcomparison/PY6PNNNX

    ReplyDelete
    Replies
    1. Thanks. I think both are quite an improvement over the original. Keep up the good work.

      Delete
  7. Would anybody be interested in sharing specific setting or in this case number values used to upscale footage, specifically with the Topaz AI Software? I do own the entire software package but just can't get the results CaptRobau gets out of his material ... Any help would be greatly appreciated ...

    ReplyDelete
    Replies
    1. For DS9 Enhanced and my Remako Mod, I used an older version of Gigapixel (3.0.4). The various settings changed a lot over that time, so it can't be easily translated to the settings of new versions.

      Delete
  8. I'd like to mention there's a couple of new options available for upscaling videos now, that automate and speed up a lot of this process: Video2X https://github.com/k4yt3x/video2x and Dandere2x https://github.com/aka-katto/dandere2x

    The quality might not be quite as good as Topaz AI Gigapixel but it's totally free and automates the entire process (as long as you're comfortable using the command line on Windows).

    ReplyDelete
    Replies
    1. That looks pretty good for anime upscales, seeing as it uses Waifu2x. I'll look into it.

      Delete

Post a Comment