Skip to main content
Icons of Progress
 

Automated Test Scoring

 

When the IBM ® 805 Test Scoring Machine was launched commercially in 1937, it became the first in a long line of such devices that are still being used today for processing standardized educational tests. Since the 1950s, fill-in-the-bubble test score sheets and mark sense scanners have remained the dominant technology in measuring results in large-scale testing programs in the world.

Like other breakthrough inventions prompted by a desire to simplify an everyday, repeatable task, the original prototype of the 805 was designed to automate the laborious task of scoring student examination papers. And the invention got its start, appropriately enough, in the classroom.

In 1931, Reynold B. Johnson was teaching high school physics in Ironwood, Michigan, when be began experimenting with an electrical machine that could grade his students’ tests. The machine he designed could detect pencil marks on an answer sheet using tiny electrical circuits, and then compare them to an answer key set up on a machine.

At the same time Johnson was working on his invention, a Columbia University researcher and IBM consultant named Benjamin Wood was also exploring different ways to mechanize the scoring of tests. Wood headed Columbia’s Statistical Bureau, which in 1929 was established in the basement of a university building with the goal of finding innovative uses for IBM machines in educational purposes.

Like Johnson, Wood was experimenting with the electrical conductivity of pencil marks. But the Columbia professor struggled to come up with a fool-proof design; in his tests the amount of electricity conducted varied widely with the level of darkness of the pencil marks—hence, the lighter the mark, the less accurate the test score.

From his own basement in Michigan, Johnson solved the pencil-mark problem by introducing high-resistor units into the electrical circuits of his design, raising the total resistance to the point where pencil-mark variations no longer mattered.

Johnson learned of Wood's interest in automatic test scoring, and in 1934 sent him a description of his design. Excited by what he saw, Wood called up IBM. "Dr. Wood saw the possibilities in the test scoring machine, but had a tough time selling it to management until he took it to Thomas J. Watson. [Sr.],” Johnson explained in a 1971 interview with the IBM publication Think. “Mr. Watson immediately grasped the concept and its commercial possibilities."

In the fall of 1934, on the strength of Wood's recommendation, Johnson was hired as a senior engineer at the IBM Endicott Engineering Laboratory and immediately went to work in developing the first production model of what became known as the IBM 805 Test Scoring Machine.

In the final commercial product, tests to be scored on the 805 were answered by marking spaces on special “mark sense” cards developed by IBM. Inside the 805 was a contact plate with 750 contacts corresponding to the 750 answer positions on the answer cards. When the cards were fed into the 805 for processing, the machine read the pencil marks by sensing the electrical conductivity of graphite pencil lead through the contacts plates. A scoring key separated the contacts into two groups, the “rights” and the “wrongs.” When the operator manipulated the controls, the 805 indicated the scores.

The speed of the IBM 805 was limited only by the operator's ability to insert the answer cards into the machine and record the scores. An experienced operator could record scores at the rate of about 800 cards per hour.

The first large-scale use of the 805 was at the New York Regents exam in 1936.
During the decade that followed, the Educational Testing Service (ETS), which administers the SAT university admission test and several other large examinations, pioneered the use of mark sense technology and the 805. And during World War II, many tests used in the placement of recruits were scored by electro-mechanical means using the 805.

By the 1960s, the electrical conductivity method of the IBM 805 was replaced by optical mark recognition (OMR) systems for surveys and tests. Meanwhile, “mark sense,” the trade name used by IBM for electrographic forms and systems, went on to become a generic term for any technology that processed written marks through both OMR and electrographic technology.

The acceptance of the IBM 805 and subsequent breakthroughs in large-scale testing machines firmly established Johnson as one of IBM's most innovative engineers, earning 90 patents. The IBM inventor went on to develop several other machines and devices using electrographic mark sense technologies, and in 1952, moved to San Jose to manage IBM’s first West Coast laboratory. There, he led a team in the development of the world’s first commercial magnetic hard-disk drive, a technology that would revolutionize data storage. The successes of Johnson and his team helped define the San Jose (Silicon Valley) area as the United States center of technology development.

Johnson was named an IBM Fellow in 1965, and was awarded the 1971 Machine Design Award of the American Society of Mechanical Engineers (ASME). The award cited his "many outstanding contributions to the educational and data processing fields through his numerous ingenious inventions and innovations dating back to the development of the first electric test-scoring machine."