Everything you see, it may not always be true. From KOL’s videos to Hollywood movies, we will dismantle the filter layer by layer. As long as you know about them, you can be anyone or anything you want in videos. Despite the magical facade, editing apps, Deepfake and computer graphics (CG) are all about algorithms. 

I think we all are no strangers to Instagram, Snapchat and Snow. Being photo-editing apps, all of them offer video-editing functions. Active photo-editing is nothing like passive photo-editing. The major difference is the allowed time, especially for live-streamed videos. Algorithms have to be as simplified as possible in order to minimize time lag. The system screenshots the images and adjusts the parameter with reference to the default beauty standards. Regardless, both of these technologies are based on facial recognition and key point detection. 

image 2

The basic principles of these apps include face recognition and key point detection. To start with, a face has to be detected before it can be beautified, this has proved to be an essential feature to enable its function. The app will then identify the human face using information from a database and use unique facial features to map the face. It also creates parameters to establish relationships between the mapped features, which includes mapping the ratio between features, measuring facial curvature etc. Once the face has been identified, the app will start highlighting essential features, using the obtained data to track and position the face and its important features. Interestingly, even when the image has been rotated, magnified or distorted, the added effects can still be retained, enabling successful image editing.

While we’re on the subject of beauty software, bilateral filter is something that cannot be dismissed. Even when the beautifying function is on, thanks to the bilateral filter, we can still recognize ourselves in the photos. Simply put, bilateral filter can ensure each parameter can be individually modified and keep the rest unchanged at the same time, maintaining the boundary of the face, allowing the rest of the facial features to remain undistorted.

The whitening concealment feature is an example of bilateral filter being put to use. The system first measures the position of facial features  and records the findings using the aforementioned facial recognition function. It then uses the Gaussian Blur to render rough skin surfaces smooth. Ordinarily, the Gaussian Blur can blur an image and reduce background interference. When applied to the skin, it achieves a markedly smoother skin tone. In the process, bilateral filter is also applied, protecting features at the boundary , like parts of the skin and the space between the eyebrows, thus allowing the facial features to be undistorted while rendering rough skin smooth.

bilateral filter bff81ef3 dbef 483e b97e 82b86232cca resize 750
Screen Shot 2019 09 23 at 3.59.37 PM

Besides bilateral filter and the Gaussian Blur, most beauty apps also use skin color detection. By measuring the user’s average skin tone and calculating the difference between different area, algorithm can be used to instantaneously modify the image. Using wrinkle remover and eye bag remover as an example, the AI detects the difference in brightness between the face and the eyebags or wrinkles and accordingly increases brightness in those areas. 

Actually, the logic behind all these features is shared between them. The aforementioned feature tracking technology is also the basis for slimming, hip enlarging and leg lengthening effects. The system can track specified features, and use them to mark the boundary of the face, allowing local alterations to the image, including slimming,  hip enlarging and leg lengthening to beautify the appearance of the user. 

Image filters, of course, are essential tools in a beauty app. Similarly, the filter is heavily based on facial recognition and feature mapping, subtly adjusting specific parameters to yield the best image, like the ‘doggy face filter’ which has been so popular in social media recently. Even filters that can change your gender and appearance are also ready-to-use features in the beauty camera app. Most filters do not alter appearance, rather it fine tunes the final image. These kinds of filters are primarily used for contrast and color adjustments, for example, in exposure, saturation and brightness etc, creating all kinds of sensory programs that correspondingly adjust parameters to predetermined levels.

image 1

Recently, mainland Internet company Tencent presented its newest findings in the ‘one button makeup remover’ software in the International Conference on Computer Vision in Italy. It stated that it also used AI to obtain the original unaltered image. Supposedly, even if the program does not have the image’s actual parameters, you can still obtain the original image. Tencent states that the technology is imaging but some practical details are yet to be elucidated.

Now you know KOLs actually have to rely on some very sophisticated technology. Next time, we’ll transition from phones to the silver screen and analyse the revolutionary CG technology.