Navigating Weight Prediction with Diet Diary

Yinxuan Gui1, Bin Zhu2, Jingjing Chen*1, Chong Wah Ngo2, Yu-Gang Jiang1
1 Fudan University      2 Singapore Management University
ACM Multimedia 2024 (oral)
*Indicates Corresponding author
Paper Code dataset
Poster Dataset Distillation (PoDD)

DietDiary: We introduce a novel dataset, DietDiary, specifically for analyzing weight in relation to food intake. DietDiary encompasses diet diary of three meals over a period of time, accompanied by daily weight measurement. This example shows data records for two participants with different weight fluctuation trends in DietDiary. The records leading to weight gain are highlighted in red.

Abstract

Current research in food analysis primarily concentrates on tasks such as food recognition, recipe retrieval and nutrition estimation from a single image. Nevertheless, there is a significant gap in exploring the impact of food intake on physiological indicators (e.g., weight) over time. This paper addresses this gap by introducing the DietDiary dataset, which encompasses daily dietary diaries and corresponding weight measurements of real users. Furthermore, we propose a novel task of weight prediction with a dietary diary that aims to leverage historical food intake and weight to predict future weights. To tackle this task, we propose a model-agnostic time series forecasting framework. Specifically, we introduce a Unified Meal Representation Learning (UMRL) module to extract representations for each meal. Additionally, we design a diet-aware loss function to associate food intake with weight variations. By conducting experiments on the DietDiary dataset with two state-of- the-art time series forecasting models, NLinear and iTransformer, we demonstrate that our proposed framework achieves superior performance compared to the original models.

Less Than One Image-Per-Class

Framework Overview

We propose a novel framework that integrates food intake information for weight prediction. The “Unified Meal Representation Learning" (UMRL) module is proposed to map the historical food intake into a time series meal feature sequence. The features and historical weight sequence are combined and subsequently fed into an agnostic time series forecasting model to predict future weights.

Global Semantics

This figure shows the details of UMRL module. Various forms of intake information are considered, including images, ingredient annotations provided by users, and ingredient labels predicted from a pre-trained ingredient prediction model.

Global Semantics

Performance comparison based on NLinear and iTransformer

Performance comparison based on NLinear and iTransformer as time series forecasting model in terms of MSE and MAE. “image", “ing-users" and “ing-LMM" represent the food intake is food image, ingredient labels provided by users, ingredient labels predicted by FoodLMM respectively.

Results of 1 Image-Per-Class
Results of 1 Image-Per-Class

Our method consistently achieves superior performance over NLinear and iTransformer across all evaluated settings, with a significant margin of improvement.

Visualize

We qualitatively compare the weight prediction visualization among ground-truth weight (blue), NLinear model (green), and our framework based on NLinear (orange) using images as dietary information for different users. It is evident that both the trend and the exact predicted values of our framework are closer to the ground truth than those of the NLinear model.

BibTeX

@inproceedings{gui2024navigating,
        author = {Gui, Yinxuan and Zhu, Bin and Chen, Jingjing and Ngo, Chong Wah and Jiang, Yu-Gang},
        title = {Navigating Weight Prediction with Diet Diary},
        year = {2024},
        booktitle = {Proceedings of the 32nd ACM International Conference on Multimedia},
        pages = {127–136}
        }