Liquori T, Koch PD, Contento IR, Castle J. The Cookshop Program: Outcome evaluation of a nutrition education program linking lunchroom food experiences with classroom cooking experiences. Journal of Nutrition Education 1998; 30: 302-313.
To evaluate the effectiveness of the Cookshop program, and to compare the Cookshop program to a participatory educational strategory without cooking, called the food and environment lessons (FEL), in promoting the targeted behavior of increased consumption of whole grains and vegetables.
From among the willing schools in the school district, schools were included in the study based on the following criteria:
- Representative of the relatively homogenous, urban, low-income school district
- Has at least three classes per grade to assign to different conditions
- Not already involved in too many other special programs
- Only students who took both pre- and post-tests were included in the final analysis.
- The CS program involved full class participation and therefore only required passive consent
- No parent refused to allow their children to participate.
- Not representative of the homogenous, urban, low-income school district
- Less than three classes per grade
- Already involved in multiple special programs
- Parent refusal.
- Schools were recruited from a school district and were selected if they fit inclusion criteria. Two schools were selected.
- Parents were recruited from the parents' associations. College students volunteered from contacting the appropriate college nutrition department faculty, who were asked to make announcements and post flyers.
- The evaluation study used a quasi-experimental, pre-/post-intervention-comparison group factorial design. Thirty-nine classes were assigned approximately equally to four conditions: Cookshop (CS) plus food and environment lessons (FEL), CS only, FEL only and a comparison condition (Com; group received neither CS nor FEL).
- All conditions received the school lunch intervention and parents in all conditions received the monthly general community newsletter. In addition, parents in the CS and CS + FEL groups received specific newsletters about the CS on a weekly basis.
- The outcome evaluation consisted of two components: A pencil-and-paper outcome evaluation questionnaire and a plate waste by visual estimate to approximate the consumption of targeted foods. The questionnaire was read to the students in class.
Dietary Intake/Dietary Assessment Methodology
Dietary intake of targeted foods (whole grains and vegetables) was assessed by visually estimating plate waste during school lunch. The amount of food left uneaten was based on an average of observations for two different CS foods. The plate waste data were obtained for each class on two separate days within two weeks before the intervention and again over a two-day period within two weeks after the intervention. Visual examination of the trays was used to identify the proportion of grains and vegetables served still remaining on each child's plate (none, one-third, two-thirds, all). This method was pilot tested.
The authors of this study evaluated the following intervention strategy. Classes were assigned to one of four conditions:
- CS only
- CS + FEL
- FEL only
- Neither intervention (comparison group).
Major components of the Cookshop and/or Food and Environment Lessons Program:
- Targeted foods were selected based on the following criteria: Grains, beans, rice or vegetables that were minimally processed with no added salts, fats or sugars; single, not combination-type, foods were preferred
- CS foods included broccoli, cauliflower, spinach, collard greens, vegetable salad, winter squash, and sweet potatoes (representing different parts of the plant); four breads (whole wheat, pumpernickel, rye and pita); one rice; and one bean
- Classroom component which included: 10 Cookshop classroom activities if assigned to group where CS was implemented and CS "classroom-friendly" recipes; 10 food and environment lessons if assigned to group where FEL was implemented; experimental activities in the form of 17 class trip to the local community garden.
- Parent component: Parent workshops, newsletters, and "family-friendly" CS recipes sent home with children
- Community component: Field trips to the local community garden; distribution of expanded "Diets and Dollars" newsletter to households of all children in CS schools, 22,000 household in Harlem, and another 80,000 households city-wide; series of articles about CS foods in "Diets and Dollars" community newsletter.
- Program effects were evaluated using a 2 x 2 ANCOVA. In the factorial design, the two factors were CS (present or absent) and FEL (present or absent).
- post-test scores were controlled for pre-test scores
- Main effects and interactions examined
- Unit of analysis was class
- Only students who took both pre- and post-tests were included in final analysis
- T-tests compared pre-test scores between schools.
Timing of Measurements
- Outcome evaluation consisted of two measures: An outcome evaluation questionnaire and a visual estimation of targeted foods consumed
- Pre-test measures were administered over a two-day period within two weeks prior to the intervention
- Post-test measures were administered over a two-day period within two weeks after the intervention.
The dependent variables that were derived from questionnaire data differed by grade. There were two questionnaire versions: One for children in K-3 (short version), and one for children in grades four to six (long version). Evaluation measures were tested during the pilot testing of the program and were modified accordingly.
- Preferences for plant foods (range 1-5): Included six and eight questions on the short and long versions, respectively. The Cronbach alpha reliabilities for this scale were 0.66 for short version and 0.60 for the long version.
- Attitudes (range 1-4): The attitude scale had nine and 17 questions on the short and long versions. Questions were from the following three subject areas: Health and cooking, taking care of the earth, and cooperation. Cronbach alpha scores were 0.55 and 0.70.
- Knowledge score (range dependent on grade level: Range 0-12, K-3; 0-25, 4-6): Tested curriculum-specific knowledge using a multiple choice format. Twelve questions were on the short version and 25 on the long version. Cronbach alpha scores were 0.52 and 0.56.
- Self-efficacy in cooking (range 1-4): Included two and three questions on short and long versions. Cronbach alpha scores were 0.35 and 0.42.
- Behavioral intentions [food intentions (range 1-4), paired food choice (range dependent on grade level: Range 0-5, K-3; 0-6, 4-6)]: Behavioral intentions to eat plant foods included nine and 14 questions on short and long version. This questionnaire had two subscales: Questions about future behaviors (Cronbach alpha scores were 0.57 and 0.56) and paired-food choice where the student would indicate which food he or she would choose (one being more healthful and one less healthful; Cronbach alpha scores were 0.23 and 0.52).
- Food intake/visual estimate (percent food remaining): Measurement described in dietary intake methodology section above. Inter-rater agreements for observation on amounts remaining were 0.80.
Independent variables included the presence or absence of CS, and the presence or absence of FEL.
Post-test scores were evaluated while controlling for pre-test scores.
- Initial N: 39 classes participated (19 in one school and 20 in another)
- The classes were relatively evenly distributed among the grades
- 590 students completed both pre- and post-tests and were included in the analysis
- Attrition (final N): 39 classes
- 23 classes of younger children (grades K-3)
- 16 classes of older children (grades four to six)
- Ethnicity: In each school, 85% of the students were African American and 15% were Hispanic
- Other relevant demographics: Four participating classes were bilingual. The instructor translated the material into Spanish when necessary.
- Anthropometrics: None
- Location: Harlem, New York City, NY.
Student outcomes: Post-test Scores and F-values for ANCOVA Conducted for the Cookshop Main Effects and the Food/environment Main effects.
|Post-test: Younger Children,
|Post-test: Older Children,
CS + FEL
F for Main Effects: CS
CS + FEL
F for Main Effects: CSFEL
Preference for plant foods
|Self-efficacy in cooking
Food intentions (range 1-4)
Paired food choice
1.99 + .53
|Food intake/visual estimate
(percent food remaining)
CS + FEL: Cookshops and food/environment lessons; CS only: Cookshops only; FEL only: Food/environment lessons only; Com: Comparison group. Higher scores are always "better."
^P≤0.10; *P≤0.05; **P≤0.01; ***P≤0.001
- There were no significant interactions for any of the scales in the analyses of variance so only the main effect results are reported
- Classes that received CS had higher mean food preference scores than classes that did not receive CS. This was consistent in both younger and older classes. FEL did not have an effect on food preference scores.
- Neither CS nor FEL impacted attitude scores
- CS and FEL each had a positive effect on knowledge in both younger and older classes. The impact of the intervention was similar among younger classes, however, CS appeared to have a much larger positive impact on knowledge than FEL in the older classes.
- The CS had a positive impact on self-efficacy in cooking in the older children. FEL did not impact this scale.
- For the younger classes that received CS, behavioral intention scores were higher than for classes that did not receive CS. This was true on both the food intentions subscale and paired food choice subscale.
- Main effects were found for CS for both younger and older classes. There were no main effects for FEL for either age group. Those receiving both CS and FEL left the least of the targeted foods on their plates, 79% and 74% for the younger and older age group. Those receiving CS only left 84% and 78%, respectively. Those receiving FEL left about 90% of the targeted foods while those receiving no classroom intervention left most of the targeted food untouched, 94% and 97% for younger and older classes.
The results suggest that actual cooking experiences and eating food with peers, accompanied by cognitive learning, may provide a promising approach to nutrition education, especially for younger children. However, lack of randomization of classes to a condition was a major limitation of the study.
The primary goal of the CS program in terms of student outcomes, was to increase children's consumption of minimally processed whole grains and vegetables. Secondary goals were to enhance the children's preferences for, attitudes toward, self-efficacy, and knowledge about these foods. The intervention activities were based on the social cognitive theory.
Research Design and Implementation Criteria Checklist: Primary Research
|1.||Would implementing the studied intervention or procedure (if found successful) result in improved outcomes for the patients/clients/population group? (Not Applicable for some epidemiological studies)|
|2.||Did the authors study an outcome (dependent variable) or topic that the patients/clients/population group would care about?|
|3.||Is the focus of the intervention or procedure (independent variable) or topic of study a common issue of concern to nutrition or dietetics practice?|
|4.||Is the intervention or procedure feasible? (NA for some epidemiological studies)|
|1.||Was the research question clearly stated?|
|1.1.||Was (were) the specific intervention(s) or procedure(s) [independent variable(s)] identified?|
|1.2.||Was (were) the outcome(s) [dependent variable(s)] clearly indicated?|
|1.3.||Were the target population and setting specified?|
|2.||Was the selection of study subjects/patients free from bias?|
|2.1.||Were inclusion/exclusion criteria specified (e.g., risk, point in disease progression, diagnostic or prognosis criteria), and with sufficient detail and without omitting criteria critical to the study?|
|2.2.||Were criteria applied equally to all study groups?|
|2.3.||Were health, demographics, and other characteristics of subjects described?|
|2.4.||Were the subjects/patients a representative sample of the relevant population?|
|3.||Were study groups comparable?|
|3.1.||Was the method of assigning subjects/patients to groups described and unbiased? (Method of randomization identified if RCT)|
|3.2.||Were distribution of disease status, prognostic factors, and other factors (e.g., demographics) similar across study groups at baseline?|
|3.3.||Were concurrent controls used? (Concurrent preferred over historical controls.)|
|3.4.||If cohort study or cross-sectional study, were groups comparable on important confounding factors and/or were preexisting differences accounted for by using appropriate adjustments in statistical analysis?|
|3.5.||If case control or cross-sectional study, were potential confounding factors comparable for cases and controls? (If case series or trial with subjects serving as own control, this criterion is not applicable. Criterion may not be applicable in some cross-sectional studies.)|
|3.6.||If diagnostic test, was there an independent blind comparison with an appropriate reference standard (e.g., "gold standard")?|
|4.||Was method of handling withdrawals described?|
|4.1.||Were follow-up methods described and the same for all groups?|
|4.2.||Was the number, characteristics of withdrawals (i.e., dropouts, lost to follow up, attrition rate) and/or response rate (cross-sectional studies) described for each group? (Follow up goal for a strong study is 80%.)|
|4.3.||Were all enrolled subjects/patients (in the original sample) accounted for?|
|4.4.||Were reasons for withdrawals similar across groups?|
|4.5.||If diagnostic test, was decision to perform reference test not dependent on results of test under study?|
|5.||Was blinding used to prevent introduction of bias?|
|5.1.||In intervention study, were subjects, clinicians/practitioners, and investigators blinded to treatment group, as appropriate?|
|5.2.||Were data collectors blinded for outcomes assessment? (If outcome is measured using an objective test, such as a lab value, this criterion is assumed to be met.)|
|5.3.||In cohort study or cross-sectional study, were measurements of outcomes and risk factors blinded?|
|5.4.||In case control study, was case definition explicit and case ascertainment not influenced by exposure status?|
|5.5.||In diagnostic study, were test results blinded to patient history and other test results?|
|6.||Were intervention/therapeutic regimens/exposure factor or procedure and any comparison(s) described in detail? Were interveningfactors described?|
|6.1.||In RCT or other intervention trial, were protocols described for all regimens studied?|
|6.2.||In observational study, were interventions, study settings, and clinicians/provider described?|
|6.3.||Was the intensity and duration of the intervention or exposure factor sufficient to produce a meaningful effect?|
|6.4.||Was the amount of exposure and, if relevant, subject/patient compliance measured?|
|6.5.||Were co-interventions (e.g., ancillary treatments, other therapies) described?|
|6.6.||Were extra or unplanned treatments described?|
|6.7.||Was the information for 6.4, 6.5, and 6.6 assessed the same way for all groups?|
|6.8.||In diagnostic study, were details of test administration and replication sufficient?|
|7.||Were outcomes clearly defined and the measurements valid and reliable?|
|7.1.||Were primary and secondary endpoints described and relevant to the question?|
|7.2.||Were nutrition measures appropriate to question and outcomes of concern?|
|7.3.||Was the period of follow-up long enough for important outcome(s) to occur?|
|7.4.||Were the observations and measurements based on standard, valid, and reliable data collection instruments/tests/procedures?|
|7.5.||Was the measurement of effect at an appropriate level of precision?|
|7.6.||Were other factors accounted for (measured) that could affect outcomes?|
|7.7.||Were the measurements conducted consistently across groups?|
|8.||Was the statistical analysis appropriate for the study design and type of outcome indicators?|
|8.1.||Were statistical analyses adequately described and the results reported appropriately?|
|8.2.||Were correct statistical tests used and assumptions of test not violated?|
|8.3.||Were statistics reported with levels of significance and/or confidence intervals?|
|8.4.||Was "intent to treat" analysis of outcomes done (and as appropriate, was there an analysis of outcomes for those maximally exposed or a dose-response analysis)?|
|8.5.||Were adequate adjustments made for effects of confounding factors that might have affected the outcomes (e.g., multivariate analyses)?|
|8.6.||Was clinical significance as well as statistical significance reported?|
|8.7.||If negative findings, was a power calculation reported to address type 2 error?|
|9.||Are conclusions supported by results with biases and limitations taken into consideration?|
|9.1.||Is there a discussion of findings?|
|9.2.||Are biases and study limitations identified and discussed?|
|10.||Is bias due to study’s funding or sponsorship unlikely?|
|10.1.||Were sources of funding and investigators’ affiliations described?|
|10.2.||Was the study free from apparent conflict of interest?|