The latest technology and digital news on the web

Human-centric AI news and analysis

Google’s new AI can intelligently crop videos for any screen size

How many times have you seen a video being badly circumscribed when you watch it on a mobile device? It’s quite arresting and annoying, and most of the time, there’s not much you can do about it.

To abode this problem, Google’s AI team has developed an open-source solution, Autoflip, that reframes the video that suits the target device or ambit (landscape, square, portrait, etc.). 

Left: Original video (16:9). Middle: Reframed using a accepted axial crop (9:16). Right: Reframed with AutoFlip (9:16). By audition the capacity of interest, AutoFlip is able to avoid agriculture off important visual content.
Left: Original video (16:9). Middle: Reframed using a accepted axial crop (9:16). Right: Reframed with AutoFlip (9:16). By audition the capacity of interest, AutoFlip is able to avoid agriculture off important visual content.

Autoflip works in three stages: Shot (scene) detection, video agreeable analysis, and reframing. The first part is scene detection, in which the machine acquirements model needs to detect the point before a cut or a jump from one scene to another. So it compares one frame with the antecedent one before to detect the change of colors and elements.

Three stages of Google Autoflip AI
Three stages of Google Autoflip AI

Once the model determines a shot, it moves on to the video agreeable assay to actuate important altar in a scene. It uses a deep acquirements neural arrangement to actuate not just people or animals, but motion and moving balls in sports, and logos in commercials.

Three stages of Autoflip

For the final stage, the AI model determines if it will use anchored mode for scenes that take place in a single space, or tracking mode for when objects of absorption are consistently moving. Based on that, and the target ambit in which the video needs to be displayed, Autoflip will crop frames while abbreviation jitter and application the agreeable of interest.

Google advisers said that Autoflip can be used to catechumen videos to many formats and screens after much effort. For the next stage, the team wants to improve object tracking in interviews and action films. It wants to use text apprehension and image inpainting techniques to better place beginning and accomplishments altar in one frame.

Video about-face in altered aspect ratios by Autoflip

You can checkout Autoflip’s code here.


Published February 14, 2020 — 13:43 UTC

Hottest related news