The evolution of data processing technologies in the food industry (2/2)

This article continues from part one of this two-part article.

Product Stability and Shelf Life – Predictive Modelling

To ensure consumer safety, product quality and to preserve concentration levels of ingredients such as added nutrients, shelf life testing is performed. These are typically carried out via durability or challenge studies that span across weeks or months of testing. During these studies, the microbial counts and quality outcomes of interest are documented at defined time points and environmental conditions.

Using experimental data to predict the stability of different products, i.e. when developing new products with new formulations, is one area where predictive modelling can be applied. These models can help guide on the shelf life and stability of a new product variation or a change in the processing of a food product as a cost and time effective model. 

This work may consist of developing a statistical model from experimental data, for example preservatives, food additives, pH etc., and measuring the microbial stability of different products. The model will predict whether a new product formulation is stable or not and it will estimate the probability associated with this prediction.

Similarly predictive mathematical models can be built using product or ingredient parameters such as colour, texture, sensory characteristics etc. The aim is to determine the shelf life of this product based on the parameters given and experimental data which can take months or years of taking measurements for selected parameters. Decision and development times can be shortened using the insights from those mathematical models. In addition the critical parameters for predicting stability and shelf-life can be identified and separated from the non-critical parameters.

Tools that can be used to assess product shelf-life and safety from a microbial perspective are called predictive microbiological models. Predictive models have been developed for both spoilage and pathogenic organisms and there are growth, survival and heat inactivation models available for use. The models usually include variables such as temperature, pH, salt or equivalent water activity and initial contamination levels.

Primary models describe changes in microbial numbers or other microbial responses over time. The model may quantify colony forming units (CFUs) per ml, toxin formation, substrate levels (which are direct measures of the response), absorbance or impedance (which are indirect measures of the response). A mathematical equation or function describes the change in a response over time with a characteristic set of parameter values.

Secondary models describe the responses by the parameters of these primary models to changes in environmental conditions such as temperature, pH, or water activity.

Tertiary models are computer software routines that turn the primary and secondary models into “user-friendly” programs for model users in the forms of software applications and expert systems. These programs may calculate microbial responses to changing conditions, compare the effects of different conditions, or contrast the behavior of several microorganisms.

Once parameters have been entered into the system, a prediction can be produced. The prediction will usually be in the form of a growth curve, but parameters such as lag time, the time to reach a specified microbial level at a specified time are also predicted. 

Note that the above models can not always replace the actual experiments, but they can help steer formulations, new products being developed or process changes and greatly increase the likelihood of success from the experimental testing.

Modelling Industry Reformulation Efforts and Population Impacts

Food and Drink companies are constantly innovating their product ranges. Product reformulation and new product development plays a big role for meeting consumer needs for safe and healthy products. Mandatory or voluntary targets set by public health stakeholders for ensuring healthier nutrient profiles of foods are another important reason for new product development.

Within Food Drink Ireland, 15 member companies participated in a research project (3) with the aim to assess the impact new food and drink development and reformulation on nutrient intakes in Irish consumers using anonymised data and scientific modelling. The nutrients focussed in were sodium, total fat, saturated fat, total sugar and energy intakes. The shift in the consumption of products sold on the market from 2005 to 2017 was incorporated into this model, taking into account the shift in consumer preference via volume sales data, the change in product composition, the discontinuation of old products and introduction of new products of the participating companies’ product portfolios.

As a database on consumer behaviour the national Irish Nutrition surveys were used, including The National Teens’ Food Survey (2005 – 2006), National Children’s Food Survey (2003 – 2004), National Adult Nutrition Survey (2008 – 2010), National Preschool Nutrition Survey (2010 – 2011). 

These were combined with the collected nutritional data on the company products gathered for the years 2005 and 2017. For applicable food and drink products, original composition data from the surveys was replaced by the gathered company data, one given food being represented by multiple possible brands. Volume sales data for a given brand and food category were used to create weighted distributions of concentrations to represent the market.

To account for the rest of the market not represented within the data, an optimistic and conservative scenarios were created. The optimistic scenario assumes that other brands followed similar reformulation and consumer preference patterns whereas the conservative scenario assumes that all other products on the market remain unchanged over time. The latter is likely to be an underestimate because other companies and retail own brands are actively reformulating and the reality is likely to be somewhere in the middle.

Data on approximately 1800 food products and over 23,000 concentration data points were  collected in incorporated into the intake model. The overall findings of the project help to quantify the impact that industry efforts have had on consumer intakes over a 12 year period.

The biggest impact could be seen in total sugar intakes ranging from 3.2g/day of a reduction in children to 0.8g/day in adults. The second biggest impact were observed in the reduction of saturated fat intakes whereas other nutrient intake reductions being less impactful.

This was the second phase of this project, with the future aim to account for the change in package sizes of products and further monitoring of food reformulation and new product development. The launch of the report involved the industry stakeholders as well as the Irish Food Safety Authority demonstrating the importance of collaboration for improving public health related matters.


Overall, the use of data science is unlocking information that was not being used to its full potential before. It is providing a means to faster product development, supporting consumer health and product safety decisions, by replacing or complementing traditional food science methodologies. 

Overall, data science is having a very positive influence on  supporting more efficient and scientific decision making within the food and health sectors. However, some challenges still remain,  such as the access to the right expertise to ask the right questions, understanding and applying the correct methodologies, the availability and quality of the data used and handling the uncertainties that will inevitably arise. 

The application of scientific modelling, data science and new technologies are quickly maturing and with that comes the knowledge and the expertise required to  continue to grow this important toolkit that has become an integral part of many organisations in the food sector.

3 –$file/The+evolution+of+food+and+drink+in+Ireland+2005+-+2017+-+Reformulation+and+Innovation+-+Supporting+Irish+diets.pdf

You might also like

Get weekly industry insights from Creme Global

Download The Overview Now

Data Sharing on Creme Global Platform

Gain critical business intelligence
from shared, anonymized data.