These correlations, it should be borne in mind, are for school systems, not for individual pupils. It might be that, though the system which devoted the most time to arithmetic did not show corresponding superiority in the product over the system devoting only half as much time, the pupils within the system did achieve in exact proportion to the time they gave to study. Neither correlation would permit inference concerning the effect of different amounts of time spent by the same pupil. Stone considered also the printed announcements of the courses of study in arithmetic in these twenty-six systems. Nineteen judges rated these announced courses of study for excellence according to the instructions quoted below:-- CONCERNING THE RATING OF COURSES OF STUDY Judges please read before scoring I. Some Factors Determining Relative Excellence. (N. B. The following enumeration is meant to be suggestive rather than complete or exclusive. And each scorer is urged to rely primarily on his own judgment.) 1. Helpfulness to the teacher in teaching the subject matter outlined. 2. Social value or concreteness of sources of problems. 3. The arrangement of subject matter. 4. The provision made for adequate drill. 5. A reasonable minimum requirement with suggestions for valuable additional work. 6. The relative values of any predominating so-called methods--such as Speer, Grube, etc. 7. The place of oral or so-called mental arithmetic. 8. The merit of textbook references. II. Cautions and Directions. (Judges please follow as implicitly as possible.) 1. Include references to textbooks as parts of the Course of Study. This necessitates judging the parts of the texts referred to. 2. As far as possible become equally familiar with all courses before scoring any. 3. When you are ready to begin to score, (1) arrange in serial order according to excellence, (2) starting with the middle one score it 50, then score above and below 50 according as courses are better or poorer, indicating relative differences in excellence by relative differences in scores, _i.e._ in so far as you find that the courses differ by about equal steps, score those better than the middle one 51, 52, etc., and those poorer 49, 48, etc., but if you find that the courses differ by unequal steps show these inequalities by omitting numbers. 4. Write ratings on the slip of paper attached to each course. The systems whose courses of study were thus rated highest did not manifest any greater achievement in Stone's tests than the rest. The thirteen with the most approved announcements of courses of study were in fact a little inferior in achievement to the other thirteen, and the correlation coefficients were slightly negative.