We discussed the main issues in the design and analysis of computational experiments. We have removed some of the more specialized details of least-squares estimation from Chaps.
Principles and Techniques
Design: Basic Principles and Techniques .1 The Art of Experimentation.1The Art of Experimentation
- Replication
- Blocking
- Randomization
This is an "experiment". The experimenter has control over a possible cause of the difference in output quality between machines. We call them "repeated measurements." The variation recorded in repeated measurements taken at the same time reflects the variation in the measurement process, while the variation recorded in repeated measurements taken over a time interval reflects the variation in the individual subject's response to the drug over time.
Analysis: Basic Principles and Techniques
Grouping the numbers through six rows and columns allows a random starting location to be obtained using five rolls of a fair die. In this book we use linear models to model our response and the method of least squares for obtaining estimates of the parameters in the model.
Planning Experiments
Introduction
A Checklist for Planning Experiments
A list should be made of the exact questions to be addressed by the experiment. Revision should start at step (a), as the scope of the experiment usually needs to be narrowed.
A Real Experiment—Cotton-Spinning Experiment
Thus Salvadori concludes: "The prudent engineer should not only be careful about material properties, but above all be aware of human behavior." It was decided that a suitable measure for comparing the effects of the treatment combinations was the number of pauses per hundred pounds of material.
Some Standard Experimental Designs
- Completely Randomized Designs
- Block Designs
- Designs with Two or More Blocking Factors
- Split-Plot Designs
Samples to be assigned to treatments are "nested within batches" and batches are "nested within suppliers". Randomization of samples at the level of treatment factors is performed separately from batch to batch. Blocks and experimental units are assigned to treatment factor levels—drug subject and task time intervals.
More Real Experiments
- Soap Experiment
- Battery Experiment ChecklistChecklist
- Cake-Baking Experiment
Note that the experimenter has no control over the age of the soap used in the experiment. An equal number of observations are made at each of the three treatment factor levels.
Exercises
Although the experimenters expected differences in the ovens and in different series of the same oven, their experience showed that the differences between the shelves in their industrial ovens were very small. This precaution was taken so that if one of the ovens failed on the day of the experiment, the treatment combinations could still all be observed twice each.
Designs with One Source of Variation
Introduction
Randomization
In step 3, columns 1 and 2 are sorted so that the entries in column 2 are in ascending order. The randomized treatments are then, and experimental units 1-7 are assigned to the treatments in this order.
Model for a Completely Randomized Design
It will be seen in Section 3.4 that unique parameter estimates cannot be obtained in the second formulation of the model. A complete model statement for each experiment should include a list of error assumptions.
Estimation of Parameters
- Estimable Functions of Parameters
- Notation
- Obtaining Least Squares Estimates
- Properties of Least Squares Estimators An important property of a least squares estimator is thatAn important property of a least squares estimator is that
- Estimation of σ 2
- Confidence Bound for σ 2
Each solution of the normal equations gives a minimum value of the sum of the squared errors (3.4.2) and provides a set of least-squares solutions for the parameters. The theorem tells us that for the one-way analysis of variance model (3.3.1), the least squares estimator.
One-Way Analysis of Variance
- Testing Equality of Treatment Effects
- Use of p-Values
To calculate the error sum of squares, ssE0, we need to determine the value of μ+τ that minimizes the error sum of squares. Source of Variation Degrees of Freedom Sum of Squares Mean Square Ratio Expected Mean Square.
Sample Sizes
- Expected Mean Squares for Treatments
- Sample Sizes Using Power of a Test
If the guess forσ2 is too small, the power of the test will be lower than the specifiedπ(). The power of the test depends on the sample size through the distribution of MST/MSE, which depends on δ2.
A Real Experiment—Soap Experiment, Continued
- Checklist, Continued
- Data Collection and Analysis
- Discussion by the Experimenter
- Further Observations by the Experimenter
From the examination of the soap packages, it was found that for deodorant soap and moisturizing soap, water is listed as the third ingredient, while ordinary soap claims to be 99.44% pure soap. The regular soap eventually lost most of the water it held, and the average weight loss (due to dissolution) was less than that for the other two soaps.
Using SAS Software .1 Randomization.1Randomization
- Analysis of Variance
- Calculating Sample Size Using Power of a Test
The value Pr > F is the p-value of the test to be compared to the chosen level of significance. DEL) to be detected (i.e.), the assumed largest value of the error variance (SIGMA2), the significance level of the test (ALPHA), and the range of values to investigate.
Using R Software
- Randomization
- Reading and Plotting Data
- Analysis of Variance
- Calculating Sample Size Using Power of a Test
The command head(soap.data, 5) on line 3 displays the first five rows of the data set shown on lines 4–9. 64 3 Designs with one source of variation Table 3.12 Calculation of sample sizes using test power.
Inferences for Contrasts and Treatment Means
Introduction
Contrasts
- Pairwise Comparisons
- Treatment Versus Control
- Difference of Averages
- Trends
They form a subset of the pairwise differences, so we can use the same formulas for the least squares estimate and the estimated. Clearly, the estimate of the linear trend is extremely large compared to its standard error.
Individual Contrasts and Treatment Means .1 Confidence Interval for a Single Contrast.1Confidence Interval for a Single Contrast
- Confidence Interval for a Single Treatment Mean
- Hypothesis Test for a Single Contrast or Treatment Mean
- Equivalence of Tests and Confidence Intervals (Optional)
The symbols "ciτi ∈" mean that the interval includes the true value of contrast ciτi with 100(1−α)% confidence. The null hypothesis H0:ciτi =h will be rejected at significance level α in favor of the two-sided alternative hypothesis HA:ciτi =h if the corresponding confidence interval for ciτi cannot contain h.
Methods of Multiple Comparisons .1 Multiple Confidence Intervals
- Bonferroni Method for Preplanned Comparisons
- Scheffé Method of Multiple Comparisons
- Tukey Method for All Pairwise Comparisons
- Dunnett Method for Treatment-Versus-Control Comparisons
- Combination of Methods
- Methods Not Controlling Experimentwise Error Rate
Details of the confidence intervals obtained by each of the above methods are given in Sects. The formulas for simultaneous confidence intervals are based on the joint distribution of the estimators Yi.−Y1.ofτi −τ1(i =2, . . . , v).
Sample Sizes
Alternatively, Scheffé's method could have been used with α=0.10 for all contrasts, including the three pre-planned contrasts. The formula for each of the simultaneous confidence intervals for pairwise comparisons using Tukey's method of multiple comparisons is given by (4.4.27) p.
Using SAS Software
- Inferences on Individual Contrasts
- Multiple Comparisons
The parameter estimates and standard errors can be used to construct confidence intervals by hand, using the critical coefficient for the selected multiple comparison methods (see also Sect.4.6.2). Similarly, replacing PDIFF with PDIFF=CONTROLU('1') requires simultaneous lower bounds for the treatment versus control contrastsτi −τ1 by Dunnett's method and is useful for 'upper tail' alternative hypotheses - namely to show which treatments have a greater effect than the control treatment (coded 1).
Using R Software
- Inferences on Individual Contrasts
- Multiple Comparisons
Theconfintfunction in line 15 will display the least-squares mean and corresponding individual 90% confidence intervals for the treatment mean (not shown). In Example 4.4.3 (page 89), Tukey's method is used to obtain a set of 95% simultaneous confidence intervals for the pairwise differencesτi−τs.
Checking Model Assumptions
Introduction
Strategy for Checking Model Assumptions
- Residuals
- Residual Plots
If the model assumptions are correct, the standardized error variables t/σ are independently distributed with an N(0,1) distribution, such that the observed values i t/σ = (yi t −(μ+τi))/σ would represent independent observations from standard normal distributions. A residual plot is a plot of standardized residuals against levels of another variable, the choice of which depends on the hypothesis being tested.
Checking the Fit of the Model
Checking for Outliers
If the conclusions of the experiment remain the same, the deviation can safely be left in the analysis. If the experimenter decides on the former, the analysis must be reported without the outlying observation.
Checking Independence of the Error Terms
Conversely, if groups of observations on different treatments (analogous to observations in the same block) have positively correlated errors, but errors associated with other pairs of observations (analogous to observations in different blocks) are independent, this tends to inflate the mean squared error up. and deflate the test effect, causing the true significance levels for tests under model (3.3.1) to be lower than indicated, and causing the true confidence levels for confidence intervals to be higher than indicated. Had the experimenter in the balloon experiment anticipated a run order effect, she could have chosen an analysis of covariance model prior to the experiment.
Checking the Equal Variance Assumption
- Detection of Unequal Variances
- Data Transformations to Equalize Variances
- Analysis with Unequal Error Variances
Figure 5.1 (p. 105) shows a plot of standardized residuals against treatment factor levels for the trout experiment. Checks of other model assumptions for the transformed data also reveal no major problems.
Checking the Normality Assumption
Other normal outcomes are calculated in a similar way, and the corresponding normal probability plot is shown in Fig.5.10. Interpreting a normal probability plot, such as the one in Fig. 5.10, requires a basis of comparison.
Using SAS Software .1 Residual Plots.1Residual Plots
- Transforming the Data
- Implementing Satterthwaite’s Method
The values of the NSCORE variable calculated by this procedure are the normal results for the values of Z. The SORT procedure and theBYstatement sort the observations in the original MUNGBEAN data set using the values of the TRTMT variable.
Using R Software .1 Residual Plots.1Residual Plots
- Transforming the Data
- Implementing Satterthwaite’s Method
If a transformation is required, select the best form transformation (5.6.3) and recheck the assumptions. Do the treatments seem to have different effects on the melting of the frozen orange dessert.
Introduction
Models and Factorial Effects .1 The Meaning of Interaction.1The Meaning of Interaction
- Models for Two Treatment Factors
- Checking the Assumptions on the Model
Both treatment factors have fixed effects, as their levels were specifically chosen (see section 2.2, p. In graph (b), all presentation formats achieved higher exam results with subject structure 1 than with structure 2, but the presentation formats themselves differed to average exam results look very similar.
Contrasts
- Contrasts for Main Effects and Interactions
- Writing Contrasts as Coefficient Lists
We can then compare the average effects of different levels of A (averaged over levels of B). This has the list of coefficients[−1, 1] in terms of the effects α∗1,α∗2 of the task levels, but the list of coefficients.
Analysis of the Two-Way Complete Model
- Least Squares Estimators for the Two-Way Complete Model
- Estimation of σ 2 for the Two-Way Complete Model
- Multiple Comparisons for the Complete Model
- Analysis of Variance for the Complete Model
Otherwise, Tukey's method would be used at the 99% level for the pairwise comparison of the levels of B (cue time), and a single 99% confidence interval would be obtained for the comparison of the two levels of A (cue stimulus). For the main effect of B, for example, a pairwise comparison of levels and Court factor Bis of the form.
Analysis of the Two-Way Main-Effects Model .1 Least Squares Estimators for the Main-Effects Model.1Least Squares Estimators for the Main-Effects Model
- Estimation of σ 2 in the Main-Effects Model
- Multiple Comparisons for the Main-Effects Model
- Unequal Variances
- Analysis of Variance for Equal Sample Sizes
- Model Building
So the least squares estimate of the difference in the dissolution times of the two solvents is ˆ. The minimum value of the sum of squares of the estimated errors for the two-way main effect model is.
Calculating Sample Sizes
Small Experiments .1 One Observation Per Cell.1One Observation Per Cell
- Analysis Based on Orthogonal Contrasts
- Tukey’s Test for Additivity
- A Real Experiment—Air Velocity Experiment
We write the sum of squares for the qth orthogonal contrast in the complete set assscq, where Thus, the sum of squares for these three contrasts could be used to estimate σ2 with 3 degrees of freedom.
Using SAS Software
- Analysis of Variance
The sums of squares for the other contrasts are calculated similarly, and the error sum of squares is calculated as the sum of the sums of squares of the three negligible contrasts. Using the Bonferroni procedure, each of the 14 hypotheses should be tested at a very small level of α.
