Volume estimation and image quality assessment with application in dietary assessment and evaluation

Chang Xu, Purdue University

Abstract

Measuring accurate dietary intake is considered to be an open research problem in the nutrition and health fields. Our team at Purdue University and the University of Hawaii Cancer Center have been developing an image analysis system to automatically estimate energy and nutrient intake from food images acquired by mobile devices for the past six years with support from the National Institutes of Health. This system known as the Technology Assisted Dietary Assessment System (TADA) has developed a mobile telephone food record (mpFR) application and deployed it on iOS and Android devices. The TADA system can automatically identify and quantify foods and beverages consumed based on analyzing meal images captured with a mobile device. After food items are segmented and identified, accurately reconstructing the volume of the food in the image is important for determining the nutrient content of the food. Once food portion size is estimated using volume and density information of the food items, the energy and nutrient information of the meal are obtained. In this thesis, we investigate the improvement of several aspects of the TADA system. We describe methods for food volume estimation, image quality assessment, and color correction. We propose a novel food portion size estimation method with the use of the 3D reconstruction and pose estimation methods based on a single image or multiple images. The single-view method estimates food volume by using prior information - segmentation and food labels generated from food identification methods in TADA. A 3D object model is reconstructed for each food item on the meal image using the prior shape information. Then, we determine the pose of the 3D model by projecting it onto the meal image. Subsequently, the food volume is estimated by matching the projection image of the transformed 3D model with the segment of the food item. We also implemented a multi-view shape recovery method using "Shape from Silhouettes" methods. We evaluated our single-view volume estimation models using food datasets from a 24 hour controlled eating occasion study and a free-living study with 56 food types. Apart from food volume estimation, we also investigate how to refine the volume estimate based on user adjustment from TADA/mpFR system. With the user feedback, corrected labels and hand segments, we obtained better food segmentation using active contours and consequentially improve the volume estimation. Food identification is a difficult problem since foods can dramatically vary in appearance. Such variations may arise not only from non-rigid deformations and intra-class variability in shape, texture, color and other visual properties, but also from changes in illumination and viewpoint. Therefore, it is very important to assist the user in requiring a good quality image by providing immediate feedback about the image quality. Low complexity image quality measures which are deployed on mpFR are also investigated. Furthermore, to address the color consistency problem, three color correction methods are proposed for illumination quality assessment.

Degree

Ph.D.

Advisors

Delp, Purdue University.

Subject Area

Computer Engineering|Nutrition

Off-Campus Purdue Users:
To access this dissertation, please log in to our
proxy server
.

Share

COinS