Respond to question posed in the exercise. 1. What are the key independent/antecedent/predictor variables? a. Is there an active, attribute, or change over time independent variable? 2. What is the key dependent or outcome variables? For each, what is the level of measurement? 3. What is the main research question or hypothesis? 4. What is the research approach (i.e. descriptive, associational, comparative, quasi-experimental, or randomized experimental) for each question? 5. What is the general design classification if the approach is randomized experimental, quasi-experimental, or comparative? 6. What is the specific experimental design name if the approach is randomized experimental or quasi-experimental? 7. Is the measurement reliability for each key variable acceptable? a. What types of evidence for reliability are presented? b. Is the evidence or support for each key variable acceptable? 8. Is the evidence for measurement validity for each key variable acceptable? 9. What is the overall rating of measurement reliability and statistics? a. Is the overall measurement reliability of the variables acceptable? b. Is the power appropriate? c. Is the choice/use of statistics appropriate? d. Is there adequate presentation of the statistical results, including effect size? e. Is the interpretation of statistical results appropriate? 10. What is the evaluation of equivalence of the groups on participant characteristics? 11. What is the evaluation of the control of extraneous experience and environment variables? a. Was the study conducted in a controlled environment? b. Did the groups have equivalent environments? c. Was there a no-treatment (placebo) or usual-treatment comparison group? d. Were there adequate attempts to reduce other extraneous influences? 12. What is the evaluation of the construct validity of the intervention? If there is no active independent variable, the question is skipped as not applicable.) a. Is the intervention (active independent variable) operationally defined and implemented appropriately based on an existing body of empirical or theoretical research? b. Is the intervention described in enough detail for it to be replicated? c. Is there a manipulation check or verification to be sure that the intervention was presented as planned? 13. What is the overall evaluation of the construct validity of the outcome measures (dependent variables) and any attribute independent variables? 14. What is the evaluation of the overall population external validity? Base the rating on answers to the following: 15. What is the evaluation of the overall ecological external validity? The rating is based on the following: a. Is the setting (or conditions) natural or representative of the target setting? b. Is the rapport with testers or observers good? c. Are the procedures or tasks natural and representative of the behavioral concepts of interest? d. Is the timing and length of the treatment or intervention appropriate (not applicable [NA] if not an experiment because not intervention is done)? e. Will the results apply to more than the specific time in history that the study was done? 16. What is the evaluation of the extent to which important participants subgroups were tested or compared? a. Are gender differences analyzed or compared? b. Are two or more ethnic or racial groups analyzed or compared? c. Are two or more age groups analyzed or compared? d. Are other important groups (e.g. cultures, geographic regions) compared?