Named as one of CB Insights
100 Most Promising AI Startups 2023
In recent years, there are two major trends impacting the photography world: smartphone photography and remote work. As smartphone photography has become more popular, consumer expectations of high-quality images have increased. But small camera systems that lack heavy parts and large glass lenses have some optical limitations.
In addition, with more people working remotely, whether in a crowded coffee shop or at home with pets and kids, it's never been more important to focus on the important task at hand during conference calls and meetings. Background blur reduces interruption and distraction so that virtual meeting participants can focus in the same way as the human eye focuses on a subject.
Bokeh is defined as “the effect of a soft out-of-focus background that gets when shooting a subject, using a fast lens, at the widest aperture ( such as f/2.8 or wider)”.
Bokeh is the pleasing or aesthetic quality of out-of-focus blur in a photograph.
Background blur involves blurring the entire image except for the main subject, typically a person, using the same level of blur. On the other hand, bokeh also identifies the main subject but applies varying degrees of blur to the background based on its distance from the subject. The farther the background, the stronger the blur, replicating the focus of a DSLR camera.
There are two methods for achieving defocused blurry background / foreground: Natural, and Artificial.
The Natural Method:
This includes three main methods for achieving a shallow depth of field and therefore a blurry background, or foreground:
The Artificial Method:
This refers to applying post-processing blur using image editing applications like Adobe Photoshop. Many photo editing tools offer various blur effects.
Some smartphones, such as recent models of Google Pixel and Apple iPhone, can automatically apply digital blur while taking a portrait to simulate the shallow depth-of-field and bokeh of a lens. While photo-editing blur effects are useful for many purposes, it is challenging to replicate the appearance of a defocused background with aesthetically pleasing bokeh produced by a high-quality lens with a wide aperture. One reason for this difficulty is that the background blur generated by a lens occurs due to the mechanical workings of the lens obstructed by the subject.
Visionary.ai Bokeh is a semi-artificial, AI-driven, real-time bokeh effect that creates the most natural effect available on the market today. Visionary.ai Bokeh uses real-time face recognition to automatically detect the person of interest (POI) and create a very sharp and clear difference between the person of interest (POI) in a scene and the background.
A depth map is the key to creating the effect artificially. This maps out which objects are closer and which are further from the camera. However, producing a reliable depth map for a given scene is not a trivial task. As neural networks have proven to be highly effective for vision perception tasks, deep learning algorithms are commonly used for depth perception.
There are two main approaches: Monocular (single camera) and Stereo (two cameras).
Stereo depth estimation is considered to be the closest approach to the natural way to perceive depth, as our brain processes information from two eyes. The logic is simple: matching two images from two rectified cameras shows a large displacement to close objects, while far away objects show less displacement. Processing this information is key for both human and machine depth perception.
Fig 1: A simplified illustration of the parallax of an object against a distant background due to a perspective shift. When viewed from "Viewpoint A", the object appears to be in front of the blue square. When the viewpoint is changed to "Viewpoint B", the object appears to have moved in front of the red square.
Source: JustinWick at English Wikipedia, CC BY-SA 3.0 <http://creativecommons.org/licenses/by-sa/3.0/>, via Wikimedia Commons
Using real depth information in the learning process tends to result in better perception of small details in the image. Separating these details is challenging if using monocular information alone. The stereo approach therefore largely leads to higher accuracy in depth estimation.
Fig. 2 The top row shows stereo output, and the middle row shows monocular output. The stereo output provides more accurate maps, whereas the monocular output is less accurate and less sensitive to small details. For example, the wall has uniform depth, but the mono output perceives it as having varying depth. This is because the pictures hanging on the walls confuse the mono imaging, tricking it into perceiving different depths. The stereo output is able to calculate these more accurately.
In fact, the quality of Bokeh is subjective. In principle, however, a beautiful Bokeh should be smooth and without obvious geometric artifacts. The point of a blurred background is not to compete with the subject and other items in more or less good focus and not to distract from them. Bokeh helps create a beautiful background that adds a whole new dimension and depth to the image.
As video conferencing and high quality smartphone photography have become a key part of our lives, consumer expectations have increased, and we can expect to see continued improvements in Bokeh quality in the near future.