Lecturer's Précis - Seidenberg and McClelland (1989)

Copyright Notice: This material was written and published in Wales by Derek J. Smith (Chartered Engineer). It forms part of a multifile e-learning resource, and subject only to acknowledging Derek J. Smith's rights under international copyright law to be identified as author may be freely downloaded and printed off in single complete copies solely for the purposes of private study and/or review. Commercial exploitation rights are reserved. The remote hyperlinks have been selected for the academic appropriacy of their contents; they were free of offensive and litigious content when selected, and will be periodically checked to have remained so. Copyright © 2004, Derek J. Smith (Chartered Engineer).

The authors began by pointing to the "immediate", or "on-line", nature of successful text processing. By this they meant that the mind's centres of higher understanding allow themselves to become more or less totally dedicated to the task of interpreting what the eyes are looking at, even "as the signal is perceived" (p523). When the cognitive system focuses its attention in this way, it allows approximately five words per second to be decoded, and it achieves this speed by simultaneously activating more than one type of encoding principle. The paper then went on to present and explore a "computational model" of that encoding.  

Moving on to what they call "the scope of the problem", the authors reminded us just how complex are the tasks which need to be mastered by learners of English. These are .....

The theoretical issue was then how to identify what processes are responsible for coping with irregular and unfamiliar words, and the authors asserted that the connectionist approach was "ideally suited" to do this, thanks to its ability to reduce cognition to "weights on connections between simple processing units in a distributed processing network" (p525). A computational model was accordingly a "minimal model of lexical processing, in which as little as possible of the solution of the problem is built in and as much as possible is left to the mechanisms of learning" (p525). As to the proposed computational architecture, the authors went for "a single uniform procedure [note this phrase - Ed.] for computing a phonological representation from an orthographic representation that is applicable to irregular words and nonwords as well as regular words" (p525; bold emphasis added).

Before proceeding, the authors reviewed previous models of the reading process, paying due credit to the classic "PDP Model", of which McClelland himself was co-author (McClelland and Rumelhart, 1986; Rumelhart and McClelland, 1986). They specifically mentioned Morton's (1969) "Logogen Model" and Coltheart's (1978) "Dual Route Model", but presented their own "single uniform procedure" as a "single route" alternative to the dual route option [Coltheart replies to this criticism in Coltheart, Curtis, Atkins, and Haller (1993), if interested].

Seidenberg and McClelland then presented their model. This was a "hidden unit" connectionist design in which an "interlevel" of storage units existed only to maintain "weighted" associations between two more functionally specific layers, rather like the cheese in a sandwich. The full system contained 400 orthographic units designed to encode word spellings, 460 phonological units designed to encode word pronunciations, and 200 "hidden units" designed to link the one to the other. As with all connectionist models of the late 1980s, what went on inside it and what gave it its ability to learn was a piece of software known as a "back propagation algorithm" .....

In the present set-up, where the network was being used to simulate the act of reading out loud, the input array needed to be loaded with a representation of a written word at the same time that the output array was loaded with a representation of the corresponding sound patterns. Here is the authors' detailed explanation of what this entails .....

The model was trained in this way on a vocabulary of 2,897 words, of which 13 were "homographs", that is to say, words like "wind" which have two pronunciations (/wind/ and /wInd/), and which therefore required two separate entries, namely <"wind"/wind/> and <"wind"/wInd/>. This meant that 2,884 unique input words needed to be paired with 2,897 unique output pronunciations. Not all words were presented on every trial, however, in an attempt to simulate the effects of word frequency upon vocabulary growth [see Gathercole (1990) on the relationship between working memory skills and vocabulary growth in children, if interested]. Once the network had completed its training regime, its performance was tested in a number of different ways .....

The first set of tests probed the network's ability to "read" written words out loud, and in general terms only 77 of the 2,897 input words turned out to have been learned incorrectly. Moreover, when these 77 errors were inspected, it turned out that 14 of them were down to human mis-keyings in the input data rather than any inadequacy on the part of the neural network. The remaining 63 errors were then classified as follows .....

So what was it that the network had learned? Well at the physical level, as we have seen, all that had changed were the weightings of individual connection pathways. These weightings had been recalculated for a given stimulus pair every time it had been presented, generally in the direction of a more precise memory trace. Not surprisingly, therefore, given that words were presented different numbers of times, the general trend here was that "the main influence on the phonological output is the number of times the model was exposed to the word itself" (p540). This mirrors real life, where "common familiar words yield faster naming latencies than do uncommon less familiar words" (p533). 

The model was also deployed in a particularly clever way to simulate human reading problems. By careful technical analysis of the 200-unit hidden layer array, those units were identified which were ON for the irregular word "pint". 22 such units were identified, 10 of which were activated by "pint" alone, 8 by "pint" and "mint", 1 by "pint" and "said", and 3 by "pint", "mint", and "said". The pattern here was described as follows .....

Additional experiments were carried out with cut-down variants of the original network, in an attempt to simulate a child with specific learning difficulties in learning to read. The general result was as follows .....

The authors' general conclusion was that high levels of performance could be obtained using their "single uniform procedure" simulation, implying that a workable form of lexical memory can be provided by the weightings of a single hidden layer. This is in stark contrast to the "cog neuro" tradition mentioned above, where activations are distributed between the processing modules identified by a particular model .....

In other words, Seidenberg and McClelland's network is not built up from individual word nodes - "there are no logogens" (p560; bold emphasis added) .....

The authors gave especially detailed consideration to their lexical decision data because it is at the heart of the "dual route (DR) theory" of reading out loud [glossary], and in the light of their data were happy to discard the DR Model on the grounds that .....

 7 - Evaluation

Here are the key arguments put forward in this paper, in revision point format .....

• The authors trained a large neural network to associate 2,897 written words with their phonetic transcriptions.
• Only 77 errors were made, 14 of which turned out to be down to operator error.
• The pattern of the remaining 63 errors was reassuringly similar to the reading problems seen in human dyslexics.
• The pattern with the 2,820 correctly "spoken" words was that high frequency words are coped with more effectively than low frequency words. 
• Performance on cut-down networks was degraded in a manner similar to the performance of humans dyslexics.

As for the theoretical confrontation between single route connectionism and dual route cognitive neuropsychology, it is not possible - as of 1989 - to issue final judgement. Students need firstly to consider the reply from Coltheart, Curtis, Atkins, and Haller (1993), which (a) fielded an upgraded dual route model [complete with "broadband" internal communication channels!], and (b) identified a number of areas which Seidenberg and McClelland's model had not catered for. Even more significant are papers on "modular connectionism" which started to emerge in the early 1990s. Norris (1991) and Hinton, Plaut, and Shallice (1993) are typical of this sub-genre of artificial intelligence studies, and both obtain significantly better simulations of real-life cognition when their neural networks are strapped together into higher order architectures.

See the Master References List

[Home]