We designed FamilyAlbum to help families easily save, share, and remember the special moments in their children’s lives. To help users look back on and preserve the high number of photos and videos uploaded every day with more convenience, FamilyAlbum utilizes machine learning—a technology for data analysis driven by artificial intelligence—to provide useful AI-based features, all while protecting privacy.
In particular, we are focusing on the development of solutions for better selecting photos used in the generation of our popular 1s Movies, as well as for automatic recommendations for FamilyAlbum products such as prints, photobooks, New Year cards, photo gifts, and DVDs among others.
(Some mentioned services/products are exclusive to Japan)
1s Movies and how they’re made
1s Movies are compilation videos we send to parents every season.1 These compilation videos are automatically generated using photos and videos uploaded to FamilyAlbum. We have received testimonials from parents saying that, amid their busy schedules, these videos help them cherish their children even more, and seeing their children’s growth in 1s Movies moves them to tears every time.
Photos and videos used in 1s Movies are selected using proprietary logic based on machine learning in the following three stages.
(1) Facial/person recognition is performed on photos and videos using deep learning
First, we utilize deep learning to identify faces/people in photos.
(2) A rating is assigned to each photo/video
Next, each photo/video is given a rating based on the results of facial/person recognition, as well as visibility settings, Favorite status, comments, and more.
(3) Photos/videos are selected and combined into a balanced 1s Movie
The 1s Movie is then put together, prioritizing selecting photos and videos that are highly rated. To ensure that each 1s Movie has variety and balance, many other factors are also considered, such as keeping a good ratio of photos to videos and selecting photos and videos taken throughout the entire season.
1. Seasonal 1s Movies are distributed every three months to all users, but Premium and Premium Pro subscribers also receive Monthly 1s Movies and Yearly 1s Movies.
Facial recognition technology for stickers
The sticker feature creates custom stickers that can be used for one-tap reactions within the app. AI is used to detect, filter, and rate the best photos for stickers. The selection process is as follows.
(1) Filter out similar photos
(2) Detect faces in photos, determine pose and age, and filter out a child’s face when it is profile, overhead, or otherwise obscured, as well as any adult faces
(3) Assign ratings to the remaining faces based on “best shot detection” and multiple other factors
(4) The highest-rated photos are then used for stickers
The technology and processes for photo/face selection are incorporated into automatic recommendation functions for other products such as prints and photobooks.
Personal Pages
The Personal Pages feature automatically sorts photos and videos uploaded to FamilyAlbum by child.2 Besides allowing you to view your photos and videos by child, you can view them by categories such as all children and all family.
AI is also used in the creation of Personal Pages. We are able to automatically identify faces by getting the feature vectors for each face from photos and videos uploaded to FamilyAlbum (image 1), then use a machine learning technique called clustering to separate them into groups (image 2).
2. The Personal Pages feature requires a FamilyAlbum Premium or Premium Pro subscription.
This system is not perfect and still has difficulty differentiating between twins and other children who closely resemble each other, so we will continue working to improve its accuracy.
FamilyAlbum will continue to utilize technology to provide a service that allows users to easily share cherished photos and videos of their children with family, all while protecting privacy.
This system is not perfect and still has difficulty differentiating between twins and other children that closely resemble each other, so we will continue working to improve its accuracy. FamilyAlbum will continue to utilize technology to provide a service that allows users to easily share cherished photos and videos of their children with family.
