Validity and Reproducibility of a Food Frequency Questionnaire for Pregnant Finnish Women

Abstract

There is increasing evidence that nutrients may be important in the development of some chronic diseases during fetal life (1). Observations suggesting that the destruction of the insulin-producing beta cells may begin before birth (2) led us to develop a food frequency questionnaire (FFQ) to be able to assess the diet of pregnant women in a prospective study on the development and determinants of type 1 diabetes (www.utu.fi/research/dipp). FFQs have been shown to be an appropriate method for assessing diet in a wide variety of epidemiologic settings, including studies among pregnant women (3–7). In comparison with short-term food records, the FFQ provides a better approximation of the habitual diet over a longer period (8). Although FFQs are not considered appropriate for estimating actual nutrient intake, they can be used for categorizing persons accurately according to intake and for identifying subjects at the extremes of intake. Among the feasible comparative methods available for validating an FFQ, food records are likely to have the smallest correlated errors (8) and have therefore been commonly used for this purpose.

The objective of this study was to assess the reproducibility and validity of a 181-item FFQ among women in the third trimester of pregnancy.

MATERIALS AND METHODS

Study design

The participants in the reproducibility study completed the FFQ twice at a 1-month interval and at the beginning and the end of the eighth month of pregnancy (figure 1). The participants in the validation study completed the FFQ 1 month before the start of the pregnancy leave (at the beginning of the eighth month of pregnancy) and 1 month after delivery. Food records were kept for 10 days (twice during 5 consecutive days at least 1 week apart) during the month preceding the pregnancy leave. The food record was chosen as the reference method since it is reliable for measuring the actual food consumption of an person and since its errors do not usually correlate with those of the food frequency method. Ethical approval for this study was obtained from the Ethical Committee of Oulu University Medical School and the Ethics Committee of the Public Health Care Department, City of Oulu, Finland.

FIGURE 1.

Open in new tabDownload slide

The design of the validation study among pregnant Finnish women, August 1995 to July 1996. FFQ, food frequency questionnaire; FR, food record.

Subjects

All of the pregnant women in their third trimester who were attending 13 maternity clinics in the city of Oulu between August 1995 and April 1996 were personally invited by their midwives to take part in the study. The women were alternately asked to participate in either the reproducibility study or the validation study.

Of the 123 pregnant women invited to take part in the reproducibility study, 111 (90 percent) completed the study. The reasons given for refusing to take part were moving to another area (n = 2) and complications during pregnancy (n = 1). Two women reported no reason. Among the 118 women who agreed to continue, 117 returned the first questionnaire and 113 returned the second. The results of two women were excluded from the analysis because of poorly completed questionnaires.

Altogether, 141 pregnant women were invited to participate in the validation study. Five women refused due to the strain of record keeping, and four refused because of complications in their pregnancies. Two women reported no specific reason. Of the 130 subjects who agreed to continue, 127 returned the first FFQ, 125 returned the first 5-day food record, 120 returned the second 5-day food record, and, finally, 116 returned the second FFQ. The reasons for giving up on the study included the burden of record keeping (n = 5) and premature birth (n = 2), while no reason was given by seven subjects. In the analysis, the results from three women were excluded because of incomplete food records. As a result, the final sample for the validation study comprised 113 women (80 percent of those invited). One recording day was removed for one participant and two for another woman due to gastroenteritis.

Sociodemographic data were collected by using a structured questionnaire. The participants in the validation study were similar to those in the reproducibility study with respect to age (mean, 29.6 years (5.1 standard deviation (SD))), body mass index at the beginning of pregnancy (mean, 22.7 (3.5 SD) for the validation study vs. 22.9 (3.2 SD) kg/m² for the reproducibility study), the proportion of women educated for at least 13 years (72 for the validation study vs. 75 percent for the reproducibility study), and number of previous pregnancies (median =1 for both groups). The distribution of all subjects by age and education was representative of pregnant Finnish women in general (9).

FFQ

Pietinen et al. (10, 11) developed a food use questionnaire of 276 food items for measuring the dietary intake of 29,000 healthy, middle-aged Finnish men participating in the Alpha-Tocopherol Beta-Carotene Cancer Prevention Study. That questionnaire was further modified into a semiquantitative FFQ comprising 110 food items for studying the diet of Finnish women participating in the Kuopio Breast Cancer Study (12). When further developing that questionnaire, we were able to use the information on the food consumption of women of childbearing age living in four different areas in Finland (North Karelia, Kuopio Province, Turku-Loimaa, and the capital area of Helsinki-Vantaa) collected by using food records in the Finrisk Study (13) 1 year before our field study. The questionnaire was specifically designed to reflect the Finnish food consumption habits. The “recipe” behind each food item was now weighted by the frequencies of consumption in the food group considered (e.g., the recipe for breakfast cereals consists of rice crisps, 33 percent; corn flakes, 23 percent; cereals with added sugar, 22 percent; and Weetabix (Weetabix Company, Kettering, Northants, United Kingdom), 22 percent). We used open-frequency categories in increasing order: less than once per month and how many times per month, week, or day. The serving sizes were based on commonly used portions identified during earlier Finnish dietary studies (14, 15), and for some foods (e.g. eggs and beverages), natural units were used.

The FFQ was designed to assess the entire diet over a period of 1 month. The main aim was to rank persons according to their food consumption and nutrient intake. The questionnaire comprised 181 food items and mixed dishes, grouped under subheadings: “milk products,” “potato, rice, and pasta as a side dish,” “cereals,” “fat on bread/spreads,” “fruits and berries,” “vegetables,” “salad dressings,” “warm main courses,” “fish and eggs,” “beverages,” and “desserts, sweets, and snacks.” There were additional empty lines for each food group to record foods not listed on the questionnaire. Moreover, questions were asked about where the meals were usually eaten (at home, in a cafeteria/restaurant) and the type of fat used in food preparation, baking, and salad dressings. Intake from dietary supplements was not included in this analysis, since this analysis focused on foods and nutrients obtained from the diet only.

We developed a software program linking the food frequency data with the nutrient software program developed at the National Public Health Institute, Helsinki, Finland (16). The program can accommodate the women's personal choices for fat (in cooking, baking, and salad dressings), which were asked about on the questionnaire. In the validation study, we took into account the women's personal choices for fat in cooking. We used butter in cooking in all recipes for those subjects who generally ate their meals at home and used butter in cooking. For other subjects, we determined the recipe fat to be butter, margarine, and oil in equal proportions, assuming that people who eat lunch at work do not usually know what type of fat is used in cooking.

Food records

Food consumption data were collected by means of two 5-day estimated food records. The first record was kept during weeks 29–32 of gestation, and the second was kept during weeks 33–36 of gestation. The time interval between the two records had to be at least 1 week. The food records covered 4 weekdays and 1 weekend day. The investigator (M. Karppinen) set the days to be recorded during the first meeting with the study subject, and these were mutually agreed upon. Each subject was then instructed to record everything she consumed during the recording periods. A booklet with 126 pictures of common food items and mixed dishes was used to facilitate the estimation of portion sizes (15). Immediately after each recording period, the records were checked at the maternity clinics by the investigator who also guided the record keeping by giving participants personal and written instructions in advance. Two university students who were majoring in human nutrition coded the dietary records.

Analysis of food consumption data

The food consumption data were analyzed by using a software program developed at the National Public Health Institute (16). The food composition database of the National Public Health Institute is continuously updated and is the most comprehensive one in Finland. Nutrient values in the food composition database are derived mainly from chemical analyses of Finnish foods. In addition, complementary data are obtained from the Finnish food industry and international food composition tables. The database currently includes about 1,600 individual food items and mixed dishes and more than 200 nutrients and other dietary factors. The system allows creation or modification of specific recipes, and the women's personal recipes were used whenever possible. However, standard recipes in the database are based on current Finnish cookbooks.

Statistical methods

The reproducibility study. Means and standard deviations for food consumption and nutrient intake were calculated for both of the questionnaires. Intraclass correlation was used to measure the reproducibility between the first and the second questionnaires. Intraclass correlation measures the fraction of total variation that is due to between-individual variability. A high value for the coefficient means a low within-person variation.

The degree of misclassification across categories between the FFQ and the dietary records was examined by dividing food consumption and nutrient intake into quintiles based on both methods. The proportions of correctly categorized subjects in the same or adjacent quintiles were calculated. Statistical analyses were performed using SAS procedures (18).

RESULTS

Reproducibility

The mean intake of all foods and nutrients, except for high-fat milk, buttermilk, and soft margarine, were generally higher when estimated using the second questionnaire than the first (table 1 and 2). The most conspicuous differences (>±25 percent) between the first and the second questionnaires among foods were for high-fat milk, soft drinks, vegetables, ice cream, and fish, and among nutrients, the largest difference was for ethanol. The intraclass correlation coefficients between questionnaires (table 1) ranged from 0.44 (ice cream) to 0.91 (coffee) for foods. The correlation coefficients were highest for the items consumed daily, such as coffee (0.91), low-fat milk (0.85), and butter (0.81) and lowest for rarely eaten foods such as ice cream (0.44), oils (0.54), and low-fat spreads (0.55) (table 1). The intraclass correlation coefficients for nutrients (table 2) ranged from 0.42 for ethanol to 0.72 for sucrose, riboflavin, and calcium. The average of all correlation coefficients for foods and nutrients was 0.65.

Validity

Food consumption was more likely to be overestimated than underestimated by the questionnaires; the estimates from the questionnaires were approximately 134 percent (first questionnaire, data not shown) and 129 percent (second questionnaire) of the values of the food records (table 3). The most marked overestimates (questionnaire ≥160 percent of food record) were for pork, soft margarine, fruit juices, inner organs, low-fat spreads, and vegetables. The most striking underestimate was for soft drinks. The mean nutrient intakes assessed using the FFQs were 138 (first questionnaire, data not shown) and 136 percent (second questionnaire) of the values, based on the means of 10-day food records (table 4). The most marked overestimates (questionnaire ≥160 percent of food record) were observed for thiamin and nitrate.

The unadjusted Pearson correlation coefficients for foods (table 3) ranged from 0.01 for high-fat milk to 0.97 for coffee. The correlations for foods were stronger than those for nutrients (mean, 0.48 vs. 0.38 (first questionnaire, data not shown) and 0.47 vs. 0.37 (second questionnaire)) and tended to be stronger for foods consumed with higher frequency. Adjustment for energy led to the most conspicuous changes for the intake of butter, sausages, and pork and the correction for attenuation for the consumption of eggs, potatoes, sausages, and berries. The adjusted and corrected correlation coefficients then ranged from 0.04 to 0.86.

The average correlation coefficient for nutrients was 0.37 (table 4). The energy-adjusted Pearson correlation coefficients for nutrients varied from 0.19 to 0.70. The most notable changes compared with the unadjusted values were observed for pantothenic acid, niacin, and chromium. When the effect of attenuation was taken into account, the coefficients ranged from 0.22 to 0.74. According to the improvement in the correlation coefficient, when attenuation was taken into account, the subjects had the greatest within-person variation in the intakes of total triglycerides, vitamin D, mercury, nitrite, copper, and vitamin B₁₂.

Classification

The subjects were divided into quintiles by food consumption (table 5) and nutrient intake (energy-adjusted estimates) (table 6) as measured by using FFQs and food records.

An average of 70 percent (52–94 percent) of the women were classified by both methods into the same or adjacent quintiles according to their food intake, and an average of 69 percent (58–81 percent) were classified as such according to nutrient intakes. For foods, only five of 30 proportions remained under 60 percent, and for the nutrients, only two of 47 proportions did so. The greatest misclassifications (>15 percent in the highest quintile on questionnaires and in the lowest quintile on food records) were found for high-fat milk (23 percent) and inner organs (18 percent) in food groups and for thiamin (18 percent) among nutrients. On average, 6 percent of foods and 5 percent of nutrients were grossly misclassified into extreme quintiles.

DISCUSSION

To effectively study the putative effects of the maternal diet during pregnancy on the development of type 1 diabetes in the offspring, we needed a dietary instrument that could be administered after delivery when the genetic disease susceptibility of the offspring had already been determined. We developed a 181-item FFQ that was sufficiently accurate for measuring the diet of the mother during the third trimester of pregnancy. In the validation study, the correlation coefficients between the second questionnaire, completed 1 month after delivery (the same arrangement as in the study proper), and the food records were similar to those obtained between the first questionnaire, completed during the period of interest (eighth month of pregnancy), and the food records. As in most other validation studies, the FFQ overestimated food consumption and intakes of nutrients. The quintile notation was acceptable for most of the foods and nutrients of special interest. The questionnaire also showed a good degree of reproducibility. The sample size of 113 subjects is reasonable for a validation study (8). The subjects came from the population for which the questionnaire was designed. In the validation study, the follow-up rate was 80 percent, which is rather high, since keeping food records for 10 days and completing a relatively cumbersome questionnaire twice requires motivation.

In reproducibility studies, the correlation coefficients have generally ranged from 0.5 to 0.7 for nutrient intakes (8). Our results compare well with those findings, except for ethanol. Moreover, in the Finnish study of nonpregnant women (12), the intraclass correlation for alcohol was weaker than in other reports. Intraclass correlations from our questionnaire for food items had greater variability than did those for nutrients, which is consistent with the results of earlier surveys (8). In the validation study, in which the subjects also completed the questionnaire twice, the mean intra-class correlation for nutrients was similar to that in our reproducibility study.

The absolute intakes estimated by using questionnaires were 30–40 percent higher than the food record estimates. When we compared our results with those of a Finnish validation study among nonpregnant women (12), it seemed that with the FFQ pregnant women tended to overestimate their food consumption more than nonpregnant women did. Earlier validation studies among pregnant women have reported similar overestimates using FFQs compared with food records or 24-hour recalls (3, 56–7). Only in one survey (4) were contradictory results reported.

Overestimation may reflect difficulties in comparing the standard portion size offered with the portion that is actually consumed. The use of an FFQ both before and after the food record provided minimal and maximal estimates of true validity; the process of keeping a food record might have improved the accuracy in completing the latter questionnaire. However, according to the intraclass correlations between foods and nutrients in the first and the second questionnaires (data not shown), the effect, if any, of increased awareness was minimal. The results of earlier studies have shown the difficulty of measuring fruit and vegetable consumption accurately using FFQs (1920–21). Our questionnaire overestimated vegetables and fruit juices, but not fruits, relatively more than other foods. Overestimation is not necessarily problematic in epidemiologic studies if the ranking of the persons according to their dietary intake is valid (22).

The crude correlations were, in general, higher for foods than for nutrients. The energy-adjustment method and correction for attenuation were used to avoid misleading conclusions on differences in the total energy intake between subjects and random within-person variation. Energy adjustment did not markedly improve the correlations for foods, but did strengthen the correlations for nutrients to a level similar to that observed for foods. In validation studies, correlation coefficients lower than 0.4 will attenuate associations between exposure and outcome to a significant degree (8, 23). After energy adjustment and correction for attenuation, the correlation coefficients of six nutrients and eight foods were lower than 0.4. Most of those foods were not consumed frequently, and therefore, their consumption appears to be difficult to measure using an FFQ. We also recorded weak correlations for various meats, which might reflect difficulties in recognizing and grouping meat categories.

We calculated quintiles by using energy-adjusted variables, since we plan to use energy-adjusted dietary intakes as exposure measures in the study proper. Comparisons of quintiles are more informative than a correlation coefficient when reporting the capacity of an assessment method to rank persons in relation to their intake (24). In our study proper, we will take a special interest in milk and cereal products among the foods and in total protein, antioxidative vitamins, and minerals among the nutrients, and in nitrates, nitrites, and heavy metal contaminants (25, 2626). It appears that by using our FFQ, we will be able to rank subjects according to their intake of the foods and nutrients listed above. Furthermore, our questionnaire provided accurate estimates for a large range of other foods and nutrients.

The assessment of the dietary intake of pregnant women is complicated because of various factors depending on the period of pregnancy. Poor correlation between instruments may be partly explained by appetite fluctuations and nausea, which may also influence the long-term diet reports (27). Five other validation studies with pregnant women have been performed during the last decade (3–7). They are difficult to compare with our study and with each other since they cover various periods of pregnancy and differ in the reference method used and in the number of days of dietary recording. Robinson et al. (3) and Brown et al. (44) used the food record as a reference method, whereas the 24-hour recall was used in the others (5–7). The studies have calculated only the intakes of nutrients, except for the study of Robinson et al., in which the contribution of selected food groups to energy and energy-yielding nutrients was also assessed. There are also statistical differences between the studies. Besides having a larger selection of nutrients, our questionnaire provides new information about the validity of the questionnaire for measuring food consumption and intakes of some food additives and contaminants during pregnancy. In general, our correlations for nutrients seem to be comparable with the results of other validation studies of pregnant women, which range from −0.02 to 0.55 for energy-adjusted nutrients (3–7). Questions about the frequency of eating out and about personal choices for fats used in cooking are among the strengths of our FFQ compared with most of the others (2323).

The reproducibility and relative validity of the FFQ assessed in this study were comparable with the results of earlier studies. We conclude that our questionnaire is a reasonably good method for assessing the diet of pregnant women in a prospective study focusing on the development and determinants of type 1 diabetes.

Supported by the Finnish Academy of Sciences, the Finnish Diabetes Research Foundation, and the Yrjö Jahnsson Foundation.

The authors thank Anne Nyberg for data entry, Mikko Virtanen and Heli Tapanainen for statistical guidance, and Catarina Ståhle-Nieminen and Jonathan Robinson for language editing.

REFERENCES