Precision
Description
Students repeat a measurement and record data. They study the reproducibility of their own measurements and compare their results to those of classmates.
Go to Top
Set
The progress of science depends upon experimentation. Experimentation requires measurement. Measurement always involves uncertainty. Accuracy and precision are two concepts which convey the degree of certainty to which a measurement is known. Accuracy conveys how well a given measurement agrees with an accepted or true value. The accuracy of a measurement may be indicated by absolute error or relative error. Precision is a description of the agreement of a set of measures taken in the same manner. The precision of a measurement can be expressed using absolute deviation, relative deviation, significant figures, or tolerance. The precision of a measurement should always correspond to the precision of the equipment used to take the measurement. Ideally, the scientist strives to make measurements that have a high degree of accuracy and precision.
Go to Top
Procedure
Measure the length of a cylinder using a ruler; record the length.
- Most vernier calipers consist of two metric and two English scales. The fixed scales have a jaw at one end. The metric fixed scale is subdivided into centimeters and millimeters. The other jaw is attached to the sliding or vernier scale. When the jaws of the caliper are closed, the zero of the fixed scale exactly coincides with the zero of the vernier scale. To use the caliper, separate the jaws, place the object to be measured between them, and close the jaws firmly on the object.
- If there is a set screw, tighten it to keep the vernier scale from moving while it is being read. Centimeters and tenths of centimeters are read on the fixed scale opposite the zero mark on the vernier scale. Hundredths of centimeters are read from the vernier scale by locating the particular division on the vernier scale that coincides with a division on the fixed scale.
-
- Step through the slides at the end of the movie reading the vernier scales as you go. Record the reading on the last slide.
- Measure the length of a piece of wood to 0.01 cm with a caliper.
- Record the measurements for 3 to 5 metal cylinders that have been supplied for you.
- Measure the length of each metal cylinder using a student caliper to the nearest 0.01 cm. Record these values.
- Record the length of each metal cylinder measured in the chart provided by your teacher. Copy the class data into your laboratory notebook.
Go to Top
Handout
Name ___________________________ Class ________
Teacher__________________________
DoChem 010 Precision

Class Data Table

Go to Top
Teachers Guide
| Expt |
Sample # |
Length (in cm) |
| 1 |
| |
| |
| 2 |
| |
| |
| 3 |
| |
| |
Pooled Class Data
| Cylinder # |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
| length 1 |
| |
| |
| |
| |
| |
| |
| |
| |
| 2 |
| |
| |
| |
| |
| |
| |
| |
| |
| 3 |
| |
| |
| |
| |
| |
| |
| |
| |
| 4 |
| |
| |
| |
| |
| |
| |
| |
| |
| 5 |
| |
| |
| |
| |
| |
| |
| |
| |
| 6 |
| |
| |
| |
| |
| |
| |
| |
| |
| 7 |
| |
| |
| |
| |
| |
| |
| |
| |
| 8 |
| |
| |
| |
| |
| |
| |
| |
| |
Go to Top
Teachers Guide
- Examine the class data for the cylinders that you used in the experiment. Did your measurement agree exactly with the measurements of the same cylinders taken by other students? Why?
- Use cylinder #3 for these questions. Calculate the average length of that cylinder from the class data.
- Find the deviation of each measurement from the average value, DA, calculated in Question 2. This can be found by taking the absolute value of the difference between each measurement and the average value:
- DA = |X - M|,
- where X is the individual value, and M is the mean or average.
- Calculate the average of the absolute deviations: add the deviations and divide by the total number of trials.
- Find the relative deviation of the set of measurements for the cylinder chosen in Question 2. This value is calculated by dividing the average absolute deviation by the average length of the cylinder. To express this as a percent, multiply the result by 100.
- DR% = (DA/M) x 100%
- Should any measurements of the length of the cylinder have been discarded? Why?
Go to Top
Handout Makeup
Name ___________________________ Class ________
Teacher__________________________
DoChem 010 Precision
Watch the movies. Make sure you know how to read the vernier scale.
The last slide of the movie is unlabeled. Please record the measurement in centimeters on that last slide. _______
Use the data below to answer the questions that follow the data.
Your Data

Class Data Table

| Cylinder # |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
| Cylinder |
5.28 |
4.93 |
6.32 |
9.79 |
2.69 |
8.61 |
1.00 |
3.29 |
| length, cm |
5.25 |
4.99 |
6.31 |
9.80 |
2.50 |
8.67 |
1.00 |
3.28 |
| |
5.25 |
4.92 |
6.31 |
9.82 |
2.80 |
8.65 |
1.10 |
3.28 |
| |
5.25 |
4.92 |
6.30 |
9.80 |
2.60 |
8.66 |
1.02 |
3.27 |
| |
5.26 |
4.93 |
6.33 |
9.81 |
2.73 |
8.68 |
1.04 |
3.29 |
- Examine the class data for the cylinders that you used in the experiment. Did your measurement agree exactly with the measurements of the same cylinders taken by other students? Why?
- Use cylinder #3 for these questions. Calculate the average length of that cylinder from the class data.
- Find the deviation of each measurement from the average value, DA, calculated in Question 2. This can be found by taking the absolute value of the difference between each measurement and the average value:
- DA = |X - M|,
- where X is the individual value, and M is the mean or average.
- Calculate the average of the absolute deviations: add the deviations and divide by the total number of trials.
- Find the relative deviation of the set of measurements for the cylinder chosen in Question 2. This value is calculated by dividing the average absolute deviation by the average length of the cylinder. To express this as a percent, multiply the result by 100.
- DR% = (DA/M) x 100%
- Should any measurements of the length of the cylinder have been discarded? Why?
Go to Top
Teachers Guide
Purpose
To demonstrate the concept of precision.
Go to Top
Materials
(per 10 students working in pairs)
- 1 demonstration caliper
- 1 piece of wood about 30 cm long
- 10 ruler
- 10 10-cm student caliper
- 10 metal cylinder of 3/8-inch diameter and of various lengths
Go to Top
Lab Hints
A class-sized caliper may be obtained from the physics teacher. It may also be obtained from most scientific equipment suppliers.
Consult sketches below if you are not familiar with reading a vernier scale. Select the vernier scale line most closely aligned with any one of the lines on the main scale. The number of this line gives the last significant digit. For more practice, step through the slides with the |>.


A broom handle may be used as the piece of wood.
Aluminum is a good metal to use for the 3/8-inch cylinders. The cylinders should range in size from 4 cm to 8 cm in height or length. The school metal shop may be able to make the cylinders. The cylinder ends need to be machined until they are smooth and even. Aluminum rods of 3/8-inch diameter may be purchased from a local hardware store. Label the cylinders from 1 to 10 using enamel paint or nail polish. Use the cm-ruler to measure the length. Compare data from the ruler with that from the calipers.
Go to Top
Time
Teacher preparation: 30 minutes
Class Time: 20 minutes
Go to Top
Hazards
There are no unusual hazards in this experiment.
Go to Top
Precautions
No special precautions are required in this experiment. Follow routine laboratory precautions.
Go to Top
Disposal
Save all materials for future use.
Go to Top
Sample Data
| Cylinder number |
length (in cm) |
| 1 |
5.26 |
| 2 |
4.93 |
| 3 |
6.33 |
Sample Class Data Table
| Cylinder # |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
| Cylinder |
5.28 |
4.93 |
6.32 |
9.79 |
2.69 |
8.61 |
1.00 |
3.29 |
| length, cm |
5.25 |
4.99 |
6.31 |
9.80 |
2.50 |
8.67 |
1.00 |
3.28 |
| |
5.25 |
4.92 |
6.31 |
9.82 |
2.80 |
8.65 |
1.10 |
3.28 |
| |
5.25 |
4.92 |
6.30 |
9.80 |
2.60 |
8.66 |
1.02 |
3.27 |
| |
5.26 |
4.93 |
6.33 |
9.81 |
2.73 |
8.68 |
1.04 |
3.29 |
Go to Top
Closure
- The variation in data can lead to a discussion of precision in measurement. For example, data for Cylinder #3 shows measures of 6.32 cm, 6.31 cm, 6.31 cm, 6.30 cm, and 6.33 cm. Which length should be reported?
- There are two types of errors in measurement. Systematic errors result from a flaw in experimental technique or in the equipment used. This type of error is very difficult to detect. Random errors represent the unavoidable uncertainty in the limitation of equipment, and in the personal differences among observers. Random error can be detected by comparing a set of successive measurements.
- Significant figures are those digits measured with certainty, plus one estimated digit. It is obvious from the data that the last digit in each measure is estimated, and therefore uncertain. The use of significant figures is one of the methods which may be employed to indicate the precision of a measurement. For example, compare the data for Cylinder #5 with those for Cylinder #3.
Go to Top
Closure?
Closure Questions:
- Examine the class data for the cylinders that you used in the experiment. Did your measurement agree exactly with the measurements of the same cylinders taken by other students? Why?
- Select one of the cylinders. Calculate the average length of that cylinder from the class data.
- Find the deviation of each measurement from the average value, DA, calculated in Question 2. This can be found by taking the absolute value of the difference between each measurement and the average value:
- DA = |X - M|,
- where X is the individual value, and M is the mean or average.
- Calculate the average of the absolute deviations: add the deviations and divide by the total number of trials.
- Find the relative deviation of the set of measurements for the cylinder chosen in Question 2. This value is calculated by dividing the average absolute deviation by the average length of the cylinder. To express this as a percent, multiply the result by 100.
- DR% = (DA/M) x 100%
- Should any measurements of the length of the cylinder have been discarded? Why?
Answers to Closure Questions:
- The measurements of the length of each cylinder do not agree exactly. This reflects random error. Random error is the unavoidable uncertainty in using equipment and in personal differences among experimenters.
- Cylinder #3
- M = (6.32 + 6.31 + 6.31 + 6.30 + 6.33) cm/5 = 6.31 cm
- Deviations of Cylinder #3
- DA = |X-M| cm
- 6.32, 0.01
- 6.31, 0.00
- 6.31, 0.00
- 6.30, 0.01
- 6.33, 0.02
- Average absolute deviation for Cylinder #3
- Average deviation =
- (0.01 + 0.00 + 0.00 + 0.01 + 0.02 cm)/5 = 0.008 cm
- Relative deviation for Cylinder #3
- = (0.008 cm/6.31 cm) x 100% = 0.1%
- Measurements should only be discarded if they vary substantially from the average measurement.
Go to Top
Computer Use
Students may enter the class' data directly into a worksheet of a spreadsheet computer program.
Go to Top
Makeup Ans.
A. 3.65 cm
1-6 see closure questions above for the answers.
Go to Top
Key Words
- precision
- deviation
- average deviation
- error
- significant figures
- vernier
- random error
- systematic error
Go to Top