Not only face changing, but also neural network can let you see 4K movies 30 years ago

category:Internet
 Not only face changing, but also neural network can let you see 4K movies 30 years ago


u25b2 1895 movie train in picture from: wiki

Renovating old films with neural networks

Recently, a foreign YouTube released the 1895 documentary train in enhanced by neural network. The whole movie is only 45 seconds long. It was shot by Louis Lumiere and Auguste Lumiere in a coastal city of France.

The movie train in enhanced by neural network

It is said that when the train drove to the camera, a large number of spectators ran out of the theater in fear, showing peoples curiosity and fear of new technology at that time. Of course, these past events have become urban legends.

But it is also of far-reaching significance to renovate the film with the neural network of new technology.

In 1895, the train came into the station was made of 35mm film. Since the projector at that time was driven by hand, we can roughly think that its original frame rate was between 16 and 24 frames.

u25b2 original film of train entering the station shot in 1895

Due to the immature film technology at that time, we can see that the pictures and scenes are relatively fuzzy, and the train also has obvious drag shadow when it comes.

However, after the image resolution enhancement and frame insertion of neural network, the old film obtained 4K ~ 60fps picture quality. If it is not for the unique image jitter of black-and-white films and film films, the picture fluency and clarity can almost match the current smartphone.

What makes neural network have such effect on image enhancement and inserting frame?

As we know, the definition of digital video is generally determined by resolution and frame rate (regardless of the bit rate that affects image compression quality). The enhancement of neural network to video mainly focuses on these two parameters.

Resolution enhancement

The comparison between waifu2xsrcnn algorithm and traditional algorithm

Neural network has unique advantages in enhancing the resolution. Maybe you have heard of a software Waifu 2x before, which is often used by animation fans to enlarge animation illustrations. Of course, it can also be used for photo enlargement.

The core method of Waifu 2x is to train an end-to-end network through machine learning, use low-resolution image as input to get the corresponding high-resolution result image, and the final result has a good performance in the degree of image sawtooth and blur, and its training principle is similar to FCN model.

u25b2 the effect pictures of different algorithms on the increase of video resolution come from: download.co.jp

In effect, the srcnn (super-resolution convolution neural network) of Waifu 2x is better than the traditional bicubic interpolation algorithm.

Of course, Waifu 2x algorithm can only be used on static pictures. However, the methods are the same, and the NGU algorithm for amplifying video resolution in madvr is also similar.

Video interpolation

For video interpolation, neural network has its own application. Previously, NVIDIA released a neural network called superslomo, which can generate intermediate frames through joint modeling of motion interpretation and occlusion reasoning combined with optical flow algorithm.

This technology can slow down the original 30 frame video to 240 frames, and add the motion details of the picture.

Huawei mate30pros 7680 frame slow motion is also generated by inserting 1080p / 960fps video frames through neural network. It can be seen that the similar neural network interpolation algorithm has a very high use value. Write in the end: technology is a double-edged sword. You can see that the neural network processing of images (i.e. AI images) is not a very terrible technology. It is a double-edged sword. If you use it to change the face of a video and infringe the portrait right of others, it is a bad technology. But if we can use it for old movie retreading, cell phone super slow motion, and real-time video enhancement, then its a good technology. Perhaps the young tuber who renovated train in just wanted to use the movies legend to tell us, dont be afraid of new technology. Source: Aifan Er editor in charge: Liao ziyao, nbjs10040

Huawei mate30pros 7680 frame slow motion is also generated by inserting 1080p / 960fps video frames through neural network. It can be seen that the similar neural network interpolation algorithm has a very high use value.

In the end: technology is a double-edged sword

It can be seen that the neural network processing of images (i.e. AI images) is not a very terrible technology. It is a double-edged sword. If you use it to change the face of a video and violate the right of others portraits, it is a bad technology.

But if we can use it for old movie retreading, cell phone super slow motion, and real-time video enhancement, then its a good technology.

Perhaps the young tuber who renovated train in just wanted to use the movies legend to tell us, dont be afraid of new technology.