twitter

Wednesday, 22 June 2016

Conceptualization in pigeons: The evolution of a paradigm

Behavioural Processes
Volume 123, February 2016, Pages 4–14
Comparative Cognition: In Honor of Ed Wasserman

doi:10.1016/j.beproc.2015.09.010
Get rights and content

Abstract

Keller and Schoenfeld (1950) proposed a unique behavioral perspective on conceptualization. They suggested that concepts refer solely to an organism’s behavior and to the conditions under which it occurs; as such, conceptual behavior need be neither verbal nor uniquely human. Herrnstein and Loveland (1964) advanced that behavioral perspective by deploying an elegant training procedure to teach visual concepts to pigeons. Keller and Schoenfeld’s perspective and Herrnstein and Loveland’s methodology have inspired my own research into conceptualization by pigeons. Using a system of arbitrary visual tokens, my colleagues and I have built ever-expanding nonverbal “vocabularies” in pigeons through a variety of different concept learning tasks. Pigeons have reliably categorized as many as 2000 individual photographs from as many as 16 different human object categories, even without the benefit of seeing an item twice. Our formal model of conceptualization effectively embraces 25 years of empirical evidence as well as generates novel predictions for both pigeon and human conceptual behavior. Comparative study should continue to elucidate the commonalities and disparities between human and nonhuman conceptual behavior; it should also explicate the relationship between associative learning, object recognition, conceptualization, and language.

Keywords

  • Conceptual behavior;
  • Discrimination;
  • Generalization;
  • Vision;
  • Pigeon

1. Introduction

In 1964, Herrnstein and Loveland reported that pigeons readily learned to distinguish color photographs that depicted human beings from otherwise similar photographs that did not. The essence of their elegant training procedure was to provide pigeons with food after they pecked a small screen displaying photographs containing one or more humans, but not to provide food when the birds pecked at photographs without humans. Not only did the pigeons learn this go/no go discrimination with numerous and varied photographs, but they reliably transferred the discrimination to novel stimuli from the same two categories of photographs. Later research from Herrnstein’s Harvard laboratory (reviewed by Herrnstein, 1985) found that visual concept learning was not confined to stimuli with which pigeons were likely to be familiar (e.g., humans, trees, and water); such concept learning could also involve stimuli that had never before been seen by pigeons (e.g., underwater pictures of fish).
Herrnstein’s pioneering research markedly departed from traditional work on discrimination and generalization in that the controlling stimuli were complex and lifelike, and did not differ along easily identifiable and manipulable physical dimensions. His work suggested that conceptualization—at least involving such concrete stimuli as real life objects—is not solely a human intellectual ability, but one which is readily demonstrable in animals of such presumably primitive intellect as pigeons (Whitman, 1919).

2. Defining concepts

Herrnstein and Loveland provocatively entitled their innovative research report, “Complex visual concept in the pigeon,” thereby provoking both surprise and skepticism among critical readers. In fact, since C. Lloyd Morgan’s (1894) early consideration of the issue, comparative psychologists have struggled mightily with answering the challenging question: do animals learn concepts?
As is often the case with difficult issues in comparative cognition (Zentall and Wasserman, 2012), providing a clear operational definition of the cognitive process under consideration is the critical first step. Here, even a cursory examination of prior thinking about conceptualization in both psychology and philosophy discloses a raft of thorny problems and disputable distinctions.
Some cognitive psychologists define categorization as the mental process of grouping objects or events into classes and responding to these classes in a similar manner (e.g., Medin and Aguilar, 2001). Concepts, on the other hand, are often thought to be the elements of knowledge that assist categorization (e.g., Hampton, 2001 and Smith and Medin, 1981). Other authors suggest that the term category should be used to refer to the actual class of items, whereas the term concept should be used to refer to the mental representation of that class ( Laurence and Margolis, 1999). Still other authors propose that the term concept should be used to refer to well-defined classes that can be specified by a set of necessary and sufficient features, whereas the term category should be reserved for ill-defined or fuzzy classes with gradual membership ( Medin, 1998). In any case, concepts and categories are frequently treated as entities: things to be found either in the environment or in one’s mind.
Several years before Herrnstein and Loveland published their famous empirical report, Keller and Schoenfeld (1950) developed a behavioral definition of concepts that liberated the experimental analysis of conceptualization from the realm of mentalism and made concepts suitable for empirical investigation in animals. These authors began their unique behavioral analysis by noting that “one does not have a concept, just as one does not have extinction—rather, one demonstrates conceptual behavior, by acting in a certain way (p. 154).”
So, just what behavior is it that we conventionally call conceptual? Keller and Schoenfeld proposed that organisms exhibit conceptual behavior when they respond similarly to members of one class of stimuli and they respond differently to members of other classes of stimuli: “Generalization within classes and discrimination between classes—this is the essence of concepts (p. 155).” In other words, when a child says ‘puppy’ if she sees a dog but not if she sees a cat, or when a pigeon pecks the viewing screen if a human being is displayed but it refrains from pecking if no human being is displayed, we would say that the child and the pigeon have each behaved conceptually.
The Keller–Schoenfeld definition also suggests a useful distinction between conceptualization and discrimination: we speak of conceptualization when the organism discriminates among classes of multiple stimuli rather than among individual instances of each class. So, if an organism has learned to make one response to a single photograph of a car and to make a second response to a single photograph of a chair, then we say that the organism discriminates the car from the chair. But, if an organism has learned to make one response to multiple exemplars of cars and to make a second response to multiple exemplars of chairs, then we say that the organism conceptualizes cars and chairs. Conceptualization thus entails a subset of discriminations in which multiple stimuli are associated with a common response.
Suppose, however, that we were to successfully train an organism to associate a dozen photographs of cars with one response and to associate a dozen photographs of chairs with a second response. Is such learning in and of itself enough to claim that conceptual behavior has been exhibited? No, it would not, because the organism might master this task by merely memorizing all 24 photographs. Therefore, we need to elaborate our definition: true conceptual behavior ought to be generalizable from familiar to novel instances of the training categories. Only if the organism can produce the correct response in the presence of novel cars and novel chairs should we properly speak of conceptual behavior.
Yet, even this additional requirement may be not sufficient to define conceptual behavior. What if the novel cars and novel chairs shown in testing were perceptually undistinguishable from the familiar cars and familiar chairs shown in training? In that case, the organism’s performing the correct responses in the presence of the novel testing stimuli would be a trivial failure to discriminate. A fully embellished Keller–Schoenfeld definition of conceptual behavior requires that the organism learns to respond similarly to members of one stimulus class and to respond differently to members of another stimulus class, as well as to generalize these differential responses to novel and discriminably different members of these stimulus classes ( Wasserman et al., 1988).
Of course, the very notion of a class of stimuli raises the critically important question of what binds the class members together. The perceptual and associative origins of stimulus classes will be discussed later.

3. Comparative implications

Beyond these weighty definitional matters, it is important to appreciate that Keller and Schoenfeld also proposed that there was no compelling reason to believe that conceptual behavior is unique to verbal humans or, indeed, to human beings at all. This view reflects a longstanding and unsubstantiated bias:
It is curious to note the resistance that may be shown to the notion that the term concept need not be limited to matters capable of being verbalized or found only in the behavior of human adults. We seem to have here a problem in our own behavior. We have formed a concept of conceptual behavior which is based upon such factors as the age of the subject, his [or her] ability to verbalize, and the fact that he [or she] is human (p. 159).

Keller and Schoenfeld’s behavioristic proposal was provocative when it was offered and, unsurprisingly, it has failed to gain acceptance beyond the narrow realm of behavior analysis. Behavioristic approaches to cognition necessarily run against the grain of cognitive and mentalistic orthodoxy ((e.g., Griffin, 1992 and Ristau, 1991) also see Fodor and Pylyshyn, 1988 for the related distinction between eliminativist and representationalist vocabularies in psychological theorizing). Nevertheless, I believe that their proposal is indeed correct. I further believe that the evidence I will review resoundingly confirms the fact that, despite lacking language, animals too are quite capable of conceptualization.

3.1. Initial paradigms for investigating concepts

Much of the empirical research inspired by Herrnstein’s pioneering studies has continued to employ a single target category (for example, fish) together with its complementary category (for example, non-fish) in a go/no-go paradigm. Using this method, both pigeons and primates have been shown to be able to learn several different basic-level concepts and to transfer their performance to novel instances of the target concept (e.g., Aust and Huber, 2002, Matsukawa et al., 2004, Schrier and Brady, 1987 and Vogels, 1999).
Yet another simple categorization procedure simultaneously displays varied photographs from two categories and requires the animal to choose stimuli from just one of them: for example, choose dogs but not humans. Using this two-alterntive forced-choice method, pigeons, dogs, bears, and primates have succeeded in learning a variety of perceptual concepts and transferred their behavior to new examples of the target concept (Range et al., 2008, Roberts and Mazmanian, 1988 and Vonk et al., 2012).
Nevertheless, Premack (1976) argued that, although many different species of animals can learn such simple, dichotomous classifications, “only primates may sort the world, i.e., divide it into its indeterminately many classes (p. 215).” Yet, despite this apt and pointed criticism, these two simple paradigms are still the most frequently deployed in research on animal concept learning. Such an extremely limited base of empirical support renders research in the realm of animal conceptualization vulnerable to the criticism of irrelevance when parallels to human conceptual behavior are drawn.

3.2. More advanced paradigms for investigating concepts

My students and I took Premack’s criticism to heart and worked to develop much richer experimental paradigms with greater fidelity to the conceptual behavior of people. What my colleagues and I first devised was a method of teaching pigeons to acquire perceptual concepts that was analogous to a parent teaching a child to categorize the pictures in a book or magazine.
In the human “name” game, the parent opens a picture book, points to one of its many colorful illustrations, and asks the child, “What is it?” If the child makes the correct verbal response, then positive social reinforcement is provided. If the child makes the incorrect verbal response, then no reinforcement is provided; instead, the parent may ask the child to try again; and, if this request also fails to occasion the correct verbal response, then the parent may have to provide it. This human “name” game thus inspired our first series of experiments into concept learning in pigeons.

3.2.1. Four-alternative forced-choice procedure and evidence

Instead of requesting verbal behavior from our pigeons (an obvious impossibility), we arranged for the birds to report members of four different categories—cats, flowers, cars, and chairs—by pecking four differently colored circular keys surrounding a square viewing screen, each key being associated with a different category. The apparatus is depicted in Fig. 1.
The four-alternative forced-choice apparatus used in our original series of ...
Fig. 1.
The four-alternative forced-choice apparatus used in our original series of concept learning experiments. The pigeon’s four differently-colored report buttons are located at the corners of the square viewing screen. The food hopper is located directly beneath the viewing screen.
Figure options
In one experiment (Bhatt et al., 1988; Experiment 1B), for example, we showed pigeons color slides depicting 10 different examples from each of the four categories. Within each category, the slides differed from one another in the number, size, color, brightness, orientation, location, and context of the stimulus object, in order to capture a broad range of category instances in those places where humans would ordinarily find them. After 30 pecks to the viewing screen, on which one of the 40 training slides was randomly displayed, the four report keys were illuminated and a single choice response was permitted. If the response was to the correct key for reporting the stimulus category shown on the viewing screen, then the pigeon was given food reinforcement; if the response was to any of the three incorrect keys, then no reinforcement was given and an unscored correction trial followed. A particular pigeon might have to peck the top left key in response to pictures of cats, the top right key in response to pictures of flowers, the bottom left key in response to pictures of cars, and the bottom right key in response to pictures of chairs. Different pigeons received different category-key assignments.
Fig. 2 shows that the pigeons readily learned this categorization task, with accuracy increasing from 29% correct in the first 5 days of training (Block 1) to 76% in the final 5 days of training (Block 6). At no time in this experiment or in any of numerous others in our laboratory was there any sign that the pigeons categorized photographs of human-made stimuli more slowly or less accurately than they categorized photographs of natural stimuli, thereby showing no inherent bias in pigeons' forming “natural” and “artificial” concepts.
Results from Experiment 1A of Bhatt et al. (1988). (Left) Acquisition of ...
Fig. 2.
Results from Experiment 1A of Bhatt et al. (1988). (Left) Acquisition of discriminative responding across six 5-session blocks of training. (Right) Choice accuracy to old and new stimuli during stimulus generalization testing sessions.
Figure options
Then, the pigeons were given two generalization testing sessions involving a total of 10 new snapshots of objects from each of the four categories. In testing, accuracy averaged 81% to the old pictures and 64% to the new ones—in each case, well above the 25%-chance level. Thus, the pigeons had acquired highly discriminative behavior, which enabled them to categorize a set of complex and lifelike stimuli they had seen only 30 times before and still other stimuli they had never seen before.
It is noteworthy that categorization accuracy was reliably lower to the novel testing stimuli than it was to the familiar training stimuli. This generalization decrement can be explained by a host of different theories of conceptual behavior—from exemplar models to prototype models (Smith and Medin, 1981; also see Astley and Wasserman, 1992). This behavioral fact suggests that the pigeons memorized some or all of the photographic stimuli they had seen during training, although nothing in the training regimen required them to do so.
Further evidence of individual stimulus learning and memory comes from a later project (Bhatt, 1988; see Wasserman and Bhatt, 1992) in which three groups of four pigeons each were given 48 daily training trials comprising: 12 copies of 1 example from the categories cat, flower, car, and chair (Group 1); 3 copies of 4 examples from the categories (Group 4); or 1 copy of 12 examples from the categories (Group 12). The speed of learning here was an inverse function of the number of examples per category. The mean number of days to reach a criterion of 70% accuracy on 2 successive days was: 6 for Group 1, 11 for Group 4, and 22 for Group 12. Either the smaller number of stimulus repetitions or the greater number of stimuli to be remembered with increasing numbers of examples per category can account for this learning function.
Of additional importance were the results of a generalization test with 32 novel stimuli: 8 from each category. Here, accuracy was a direct function of the number of examples in training. The mean percentage of correct choices on generalization testing trials was: 27% for Group 1, 45% for Group 4, and 62% for Group 12. Thus, although increasing the difficulty of original learning, greater numbers of training examples per category enhanced the accuracy of generalization performance, perhaps because of the increased likelihood that any given testing stimulus resembled one or more of the remembered training stimuli ( Smith and Medin, 1981). Not only are these learning and testing data orderly, but they neatly correspond with categorization in humans (reviewed by Homa et al., 1987) and with discrimination performance in pigeons involving bird and mammal sketches (Cook et al., 1990).
All of the research described so far has entailed stimuli that were repeated, either between daily sessions or both between and within daily sessions of training. Is such stimulus repetition necessary to support successful category learning and generalization? To investigate this issue, a large library of snapshots was created from four categories: people, flowers, cars, and chairs (Bhatt et al., 1988; Experiment 3). With 2000 snapshots (500 from each of the four categories) and 40 trials per session, pigeons could be trained for 50 sessions with no stimulus ever being repeated; each trial was thus both a training trial and a testing trial. The results of the experiment were clear-cut: pigeons came to respond discriminatively to stimuli from the four different categories of pictures even when those individual examples were never repeated. After beginning at the chance level of 25%, categorization accuracy of a group of four pigeons rose to a mean level of 70% over Days 41 to 50 of training.
The prior experiment convincingly showed that stimulus repetition was not necessary for category learning. The next experiment (Bhatt et al., 1988; Experiment 4) further pursued the matter. Here, a set of 40 slides, 10 each from the categories person, flower, car, and chair, was chosen randomly from our library of 2000. Four different pigeons were trained with this set of slides on repeating sessions that alternated with nonrepeating sessions, in which the birds were trained with new sets of slides that were never used again in another session. So, the pigeons were trained with the repeating 40-slide set on Days 1, 3, 5,…95 while being trained with the novel non-repeating slide sets on Days 2, 4, 6,…96. The results showed that discriminative responding rose faster and attained higher final levels of accuracy to the repeating slide set than to the nonrepeating slide sets. Performance on the repeating set rose from a mean of 29% in the first 4-day block to a mean of 85% in the last 4-day block; performance on the nonrepeating sets rose from a mean of 26% in the first 4-day block to a mean of 66% in the last 4-day block. Although unnecessary for category learning, stimulus repetition promotes the process.
Given all of the above data on conceptual discrimination and generalization in pigeons, one might wonder whether the differential reinforcement that we so assiduously arranged in those experiments really created the conceptual behavior that the pigeons exhibited. This matter may appear to be an odd one to raise, but Herrnstein and de Villiers (1980) speculated that differential reinforcement may not produce, but merely disclose already-existing concepts: “Something in the pigeon’s perceptual dynamics ties [stimuli] together as a class, prior to differential reinforcement (p. 87).” This argument is tantamount to saying that primary stimulus generalization is the root of conceptual behavior, an assertion for which there is substantial empirical support in both human and nonhuman animals ( Harnad, 1987).
It seems quite reasonable to hypothesize that many, if not most, basic-level human conceptual categories comprise highly similar stimuli. To our eyes, cats resemble one another much more than they resemble flowers, cars, or chairs. This perceptual similarity may be an important and inborn factor responsible for the emergence of the very concepts that we are considering, a possibility stated most clearly and emphatically by the philosopher Quine (1969):
If then I say that there is an innate standard of similarity, I am making a condensed statement that can be interpreted … in behavioral terms. Moreover, in this behavioral sense it can be said equally of other animals that they have an innate standard of similarity too. It is part of our birthright. And, interestingly enough, it is characteristically animal in its lack of intellectual status (p. 11, italics added).

Quine proceeded to suggest that the origin of perceptual similarity as well as the concordance of similarity relations from person to person is due to the operation of evolutionary mechanisms. “If people’s innate spacing of [perceptual] qualities is a gene-linked trait, then the spacing that has made for the most successful inductions will have tended to predominate through natural selection (1969, p. 13).” Anderson (1991) later expanded on Quine’s thesis and proposed that the main force behind perceived similarity is physical similarity: “the mind has the structure it has because the world has the structure it has (p. 428).”
Quite apart from the origins of perceived similarity (Spinozzi, 1993), one can ask, is such categorical similarity perceived by nonhuman animals? And, if it is, then how can one tell? To answer these questions, several different lines of inquiry have been pursued, each suggesting that animals similarly group stimuli into coherent categories without ever being required to do so by the prevailing contingencies of reinforcement (Astley and Wasserman, 1992, Fujita and Matsuzawa, 1986 and Sands et al., 1982).
In one such project, Wasserman et al. (1988, Experiment 2) examined the coherence of categories and their concordance in pigeons and people by comparing the relative speeds of pigeons’ learning to sort the same pictorial stimuli into human conceptual categories (true-categorization) or into absolutely arbitrary collections (pseudo-categorization). If all of the slides in the total pool of cat, flower, car, and chair stimuli were equally discriminable from one another, then pigeons trained on the true-categorization task should learn at the same rate as pigeons trained on the pseudo-categorization task, in which equal numbers of cats, flowers, cars, and chairs are associated with the four different report responses. However, if, to pigeons, members of the human conceptual categories more closely resemble one another than they resemble members of the other conceptual categories, then learning of the true-categorization task should proceed faster than learning of the pseudo-categorization task.
This prediction follows from the fact that correct responding in the true-categorization task should be bolstered by direct strengthening of responding to a specific key in the presence of a specific stimulus and by indirect strengthening due to similar stimuli in the same category occasioning the same response. But, in the pseudo-categorization task, correct responding will be bolstered primarily by direct strengthening of responding to a specific key in the presence of a specific stimulus; greater generalization within the conceptual categories here should produce an equal likelihood of pecking all four keys, reducing the accuracy of discriminative performance.
The evidence unequivocally supported the latter possibility. Over Days 37–40 of discrimination training, pigeons on the true-categorization task averaged 79% correct, whereas pigeons on the pseudo-categorization task averaged only 44% correct. These results (and those of Edwards and Honig, 1987, Herrnstein and de Villiers, 1980 and Pearce, 1988) implicate differential within- versus between-class resemblance as a key feature of visual categorization in animals.
Wasserman et al. (1988, Experiment 1) deployed a new technique to explore the stimuli that, to pigeons, constitute a class or category of objects. In any particular 40-trial session, pigeons were given a split-category discrimination, in which they viewed 20 cat slides and 20 flower slides (or 20 cat slides and 20 chair slides, or 20 car slides and 20 flower slides, or 20 car slides and 20 chair slides). For each pigeon, half of the cat slides required a peck to one key (Key 1) and the other half of the cat slides required a peck to a second key (Key 2), whereas half of the flower slides required a peck to a third key (Key 3) and the other half of the flower slides required a peck to a fourth key (Key 4). [Cat-chair, car-flower, and car-chair sessions were similarly constructed, with different pigeons having different key assignments.] If the cat slides in the first set were equivalently discriminable from the 30 other slides in the illustrative session, then errors should be randomly distributed to Keys 2, 3, and 4. However, if the 10 slides in the first set of cats are more similar to the 10 slides in the second set of cats than they are to the 20 flower slides, then more errors should be made to Key 2 than to Keys 3 or 4.
The pigeons’ pattern of choice errors was clearly consistent with the latter possibility. Over Days 105–112 of training, a mean of 56% of all errors were within-category in nature (33% was the chance level of errors, since there were three keys on which errors could be made).
This split-category project was relevant to another important issue: namely, the discriminability of the training and testing stimuli in our initial categorization study (Bhatt et al., 1988). Had our original pigeons generalized their responding to the novel test stimuli merely because the testing stimuli could not be discriminated from the training stimuli (a trivial possibility)? Or had generalization occurred to discriminably different stimuli (a nontrivial possibility)? Because the different split categories used by Wasserman et al. (1988) corresponded to the training and testing sets used by Bhatt et al., we could definitively decide the matter.
Over Days 105–112, the pigeons’ split-categorization accuracy averaged 72% (here, the chance of a correct response was again 25%). Because the training and testing stimuli in the Bhatt et al. project were clearly discriminable to pigeons, the reliable generalization obtained in that study was not due to the birds' inability to discriminate the new from the old sets of stimuli. Our original stimulus generalization result thus passes a most stringent test of conceptualization.
Overall, the research reviewed using our four-alternative forced-choice procedure strongly suggests that nonhuman animals very ably master perceptual or basic-level concepts. Such mastery appears to rely on the familiar behavioral principles of discrimination and primary stimulus generalization. The roots of conceptualization thus appear to lie deep in the perceived similarity of external stimuli. Differential similarity influences the responses of nonhuman animals in much the same way as it influences the speaking of humans. Although it may not always be the case that humans and nonhuman animals categorize stimuli in the same way (e.g., see Fujita, 1987, Roberts and Mazmanian, 1988, Yoshikubo, 1985 and Wasserman and Castro, 2012), based on the results presented here, one can suggest that conceptual behavior and its underlying cognitive substrates are generally similar in humans and nonhuman animals.

3.2.2. 16-alternative forced-choice procedure and evidence

When we first published the results of our four-alternative categorization project (Bhatt et al., 1988), we noted that the pigeons’ behavior in our study closely resembled that of the chimpanzees in the well-known Gardner and Gardner (1984) study. There, chimpanzees were trained to make different American Sign Language (AMESLAN) gestures to members of different object categories. When the chimpanzees were later shown novel stimuli from these categories, they made the appropriate AMESLAN responses. Thus, the chimpanzees mastered and generalized a multiple-category classification task just as did our pigeons. Interestingly, the Gardners entitled their study “A Vocabulary Test for Chimpanzees.” Given that the results of our pigeon study formally resembled those of the chimpanzee study, perhaps it might not have been too impetuous for us to suggest that pigeons had also acquired a “vocabulary” containing nonverbal “words” or “general signs” (Wasserman, 2002).
Nevertheless, we resisted that temptation. After all, our pigeons had learned only 4 “words,” whereas the Gardners’ chimpanzees had learned some 32 “words.” Furthermore, who would really have taken us seriously if we had claimed that pigeons were capable of learning even a small four-item vocabulary?
Still, we continued to be intrigued by the possibility that pigeons might acquire many more that four visual categories, if only we were to use suitable behavioral methods to train them. We had subsequently had the opportunity to conduct unrelated research with bonobos at Great Ape Trust in Des Moines, IA (Nagasaka et al., 2010) and were impressed by Kanzi’s categorization skills using lexigram symbols ( Rumbaugh, 1977 and Savage-Rumbaugh et al., 2009). So, we modeled a new pigeon project along those lines.
In order to up the ante in terms of the number of “words” or categories that pigeons might be able to learn, Wasserman et al., (2007) taught pigeons to sort a total of 128 black-and-white photographs into 16 categories—baby, bottle, cake, car, cracker, dog, duck, fish, flower, hat, key, pen, phone, plane, shoe, tree—using stimuli including those depicted in Fig. 3 and the response panel depicted in Fig. 4. Each of the 16 categories was introduced in a stagewise manner, beginning with 1 and ending with 16. With each category that was introduced, a new report button was correspondingly introduced with its associated pexigram (we were now studying pigeons, after all). This paradigm far more effectively captures the richness of natural categorization as well as the possible stepwise learning of categories observed with human children and with nonhuman animals studied in several well-known “word-learning” projects (e.g., Pepperberg, 2002 and Pilley and Reid, 2011).
Exemplary training and testing stimuli from 2 of the 16 object categories used ...
Fig. 3.
Exemplary training and testing stimuli from 2 of the 16 object categories used in Wasserman et al. (2007) and Wasserman et al. (2015). (Top) Duck photographs. (Bottom) Shoe photographs.
Figure options
The response panel used in Wasserman et al. (2007). All 16 pexigrams for one of ...
Fig. 4.
The response panel used in Wasserman et al. (2007). All 16 pexigrams for one of the pigeons are arrayed around the viewing screen in different fixed spatial locations. The numbers represent the sequential training order of each of the 16 categories which are denoted alongside those numbers. The numbers and labels were not presented to the pigeons.
Figure options
The results of our experiment were clear and, we believed, compelling. Fig. 5 shows that all three of our pigeons learned the categorization task to high levels of performance through the first 14 successively-trained categories. Through the first 8 categories, all of the individual pigeons took about the same number of sessions to attain criterion performance, achieving a d-prime score of 2.0 to the newest-trained category and an average d-prime score of 2.0 across all of the previously-trained categories to that point; thereafter, two of the birds (12Y and 68W) took longer to attain criterion, especially with categories 13 and 14. The behavior of bird 9W was exceptional; it showed no signs of slowing its speed of learning across the first 14 categories.
The mean number of training sessions for three individual pigeons to attain ...
Fig. 5.
The mean number of training sessions for three individual pigeons to attain criterion across blocks of two sequentially-presented categorization problems in Wasserman et al. (2007).
Figure options
Birds 68Y and 9W subsequently proceeded to master all 16 categories (bird 12Y advanced no further). Fig. 6 shows that generalization testing after category 15 had been learned yielded clear transfer performance by both pigeons to four new exemplars from each of the photographic categories; although accuracy to the novel photographs (38% correct) was lower than to the training photographs (76% correct), it did exceed chance accuracy (now being only 7% correct).
Choice accuracy of two pigeons to old and new stimuli during stimulus ...
Fig. 6.
Choice accuracy of two pigeons to old and new stimuli during stimulus generalization testing sessions in Wasserman et al. (2007).
Figure options
We were certainly pleased with our pigeons’ performance on this new and much more demanding categorization task. Nonetheless, we had to concede that this 16-alternative forced-choice procedure suffered from several shortcomings, rendering it unsuitable for further exploitation as a productive analog to human word learning. First, as the number of categories was increased beyond eight, it became increasingly difficult for the pigeon to locate the newly added pexigram, even though the pexigrams were placed in fixed positions on the touchscreen. Second, increasing numbers of pexigram report alternatives decidedly decreased the chance likelihood of an organism’s choosing the correct response, making it more difficult than usual for us to assess the statistical significance of the results. And, third, although children may learn words in a stepwise manner, they are not necessarily exposed to words in that manner.

3.2.3. 16-category, 2-alternative forced-choice procedure and evidence

We thus sought another way to train pigeons with a prodigious number of categories, while at the same time streamlining the process of recording and analyzing their categorization behavior. We developed and deployed another 16-category paradigm to achieve those ends (Wasserman et al., 2015).
Specifically, we again trained three pigeons to learn the same 16 categories of objects as in the prior project by pecking the same 16 arbitrary pexigrams. Each category contained the same 8 black-and-white photographs (128 total images) in training plus the same 4 black-and-white photographs (64 total images) in testing. Here, it was important overcome the need to progressively add categories/pexigrams when training with so many different categories (Wasserman et al., 2007) in a simultaneous training paradigm. In order to do so, we borrowed an approach from human word learning, in which children and adults are simultaneously trained on all of the items, but they receive only a subset of the available words as responses on any given trial ( Creel et al., 2006, Magnuson et al., 2003 and Yu and Smith, 2012). This experimental tactic simplifies the response options on each trial while still permitting a large number of categories to be taught from the outset of training. Thus, our new method of concurrently training a large number of categories using a simple 2-alternative forced-choice task represents the culmination of several developmental steps toward a realistic, yet tractable way to experimentally explore the associative substrates of natural categorization.
As shown in Fig. 7, the items to be categorized appeared in center of a small screen. On either side of the training item were two report buttons depicting two colored pexigrams. One pexigram corresponded to the category of the displayed stimulus; the second was a foil which corresponded to a different, randomly-selected category; correct pexigrams and foils randomly appeared to the left and right of the training item across trials. Correct choices delivered food reinforcement; incorrect choices did not deliver food and required the pigeons to complete correction trials until the correct response was performed. Pigeons remained on this procedure until their performance had stabilized. Because generalization is the key behavioral hallmark of concept learning, we tested the breadth of the pigeons’ categorization behavior by showing them 4 novel exemplars from each of the 16 categories in later testing sessions.
The response panel used in Wasserman et al. (2015). Here, only two pexigrams ...
Fig. 7.
The response panel used in Wasserman et al. (2015). Here, only two pexigrams were shown alongside the viewing screen on any given trial: one was the correct option for the presented photograph and the other was a randomly-chosen foil from the 15 incorrect categories for the presented photograph.
Figure options
To explore the course of category learning, we examined the first 44,700 training trials. Each of the pigeons showed clear evidence of learning, although they reached different levels of accuracy: bird 45W performed best, reaching accuracy levels exceeding 80% correct; bird 39Y reached nearly 70% correct; and, bird 66W reached an accuracy level of 65% correct.
Because there were so many different training categories, strong discrimination performance on this task could have been due to better-than-chance performance in classifying all of the categories or to exceptional performance in classifying only a few of the categories. To examine this issue, for each bird, we conducted a (one-tailed) binomial test against chance for each category in every block of training. These tests revealed that birds 45W and 39Y each attained the maximum score of 16 categories midway through training and that bird 66W attained a maximum score of 14 categories two-thirds of the way through training. Thus, all three birds learned something about virtually all of the categories and were not merely masters of only a few of them. The modest percentage correct scores of Birds 39Y and 66W may therefore underestimate their learning about the large number of training items and categories.
During subsequent testing sessions, in addition to the 8 training stimuli, 4 new transfer exemplars from each of the 16 categories were presented. Accuracy to the familiar training stimuli averaged 73% correct and accuracy to the novel testing stimuli averaged 69% correct. Thus, even when they were trained with a large number of categories, the pigeons were able to transfer discriminative responding to novel stimuli with only a modest decline in accuracy.
These latest results clearly document a substantial capacity for concept learning in pigeons. All three birds learned to discriminate all 16 categories. More detailed analyses of the birds' behavior showed that they learned to categorize 97% of the 128 individual exemplars. In addition, the pigeons learned these categories as coherent stimulus collections, reliably generalizing their discriminative responding to new exemplars. Finally, and of key importance, evidence of this large categorization capacity was achieved while the birds were learning the 16 photographic categories in parallel, a training task that may be substantially more demanding than learning categories sequentially as in our earlier project ( Wasserman et al., 2007).
We believe that we now have “proof of concept” of the merits of using our expanded operant conditioning task to elucidate the basic associative principles that may participate in children's word learning. This new paradigm surely advances our understanding of the complexity and capacity of associative learning processes in animals and promotes the application of those processes to important problems lying well beyond the traditional realm of animal conditioning. Moreover, the tight laboratory control that is offered by this paradigm also permits the conduct of detailed trial-by-trial analyses that may be capable of disclosing the intricacy and richness of associative learning, possibly affording us the opportunity to study the interplay between the strengthening and weakening of associative connections (for details of these encouraging results, see Wasserman et al., 2015).

4. Summing up

Our research into pigeon concept learning has now spanned more than 25 years. In that time, we have developed experimental methods that are fully capable of empirically assessing Keller and Schoenfeld’s (1950) innovative insights into the nature and generality of conceptual behavior. Like people, pigeons learn concepts. Pigeons can reliably categorize as many as 2000 individual photographs from as many as 16 different human object categories, even without the benefit of seeing an item twice, although repetition does promote the learning process. Pigeons’ categorization behavior is not specific to familiar training stimuli; it reliably generalizes to novel testing stimuli. Pigeons can progressively add categories to their behavioral repertoire as training requires or they can learn multiple categories in parallel. Basic-level or perceptual concepts leverage the greater visual similarity among members of the same category than among members of different categories. And, varying the number of exemplars in the training categories affects the behavior of pigeons in the same way as it affects the behavior of humans: increasing the number of exemplars slows acquisition, but enhances generalization.

4.1. Theoretical interpretation

Beyond the methodological and empirical products of our evolving conditioning paradigm, we have also developed a principled theoretical account of conceptualization (Soto and Wasserman, 2010a), which elucidates the associative mechanisms that give rise to the details of concept learning that we reviewed in previous sections. This empirical evidence can be explained by a connectionist model implementing two simple assumptions.
The first assumption is that exemplars from any given category are represented by a large collection of common features or “elements,” with different categories involving different probabilities that an exemplar from the category will activate each of those common elements. When the probability of activation of an element is high in a particular category, that element is activated by several different exemplars from that category, rendering it relatively category-specific. When the probability of activation of an element is low in a particular category, only a few exemplars from the category activate the element, rendering it relatively stimulus-specific.
The second assumption is that concept learning proceeds by strengthening connections between such elemental stimulus representations and responses through error-driven learning. On each trial, the model selects an action that is likely to maximize predicted reinforcement, usually the action with the strongest connections to active stimulus elements. The difference between the predicted reinforcement and the reinforcement obtained after the response is performed—so-called prediction error—determines how much the connection between the active stimulus elements and the chosen action should be modified ( Rescorla and Wagner, 1972).
Our model specifies the conditions leading to the control of actions by category-specific elements, yielding categorization learning; it also specifies the conditions leading to the control of actions by stimulus-specific elements, yielding identification or discrimination learning. For example, all instances of strong categorical control discussed in previous sections (as in the case of the split-category task of Wasserman et al., 1988) can be seen to be the result of differences in the rate at which category-specific and stimulus-specific elements are presented in typical categorization tasks. Because category-specific elements are shared by many exemplars, they are presented often and their connections with responses can be modified faster. Stimulus-specific elements are presented less often and they support slower learning. In short, category-specific elements enjoy a repetition advantage over stimulus-specific elements. [For a detailed explanation of other empirical effects, see Soto and Wasserman (2010a)].
Additional experiments have yielded further supportive evidence for the model. Some of this work has deployed a blocking design in both pigeons ( Soto and Wasserman, 2010a) and people (Soto and Wasserman, 2010b). Other work has deployed an overshadowing design in pigeons ( Soto and Wasserman, 2012a). We have additionally proposed ( Soto and Wasserman, 2012b and Soto and Wasserman, 2014) that underlying the many reported behavioral similarities between pigeons and people is an evolutionarily conserved learning mechanism that may be implemented in the homologous neural structures of birds and mammals.

4.2. Application of the paradigm to superordinate concepts

Most of the previously cited research concerned basic-level conceptualization. Basic-level concepts (like tables and sofas) are commonly contrasted with superordinate concepts (like furniture) which comprise several perceptually diverse basic-level concepts. Unlike basic-level concepts, superordinate concepts are not believed to be based on perceptual similarity among their members (Hampton, 2001).
In a seminal paper, Rosch et al., (1976) found that children learn to classify images at the basic level more quickly and at an earlier age than at the superordinate level. Later investigations showed, however, that infants may be sensitive to broad, global categories (e.g., vehicles versus animals), but not to narrower, basic-level categories within the superordinate category (e.g., rabbit versus dog or car versus truck; Mandler and McDonough, 2000). Some researchers have suggested that whether an infant selectively attends at the global or basic level may depend on the nature of the task (Mareschal and Quinn, 2001).
These findings on human conceptualization lead to the question: can animals also learn superordinate concepts? The answer is “yes.” Animals, including pigeons, learn to respond similarly to pictures from different superordinate-level categories (Roberts and Mazmanian, 1988, Vonk and MacDonald, 2002 and Wasserman et al., 1992), treating them as functional equivalents of one another (also see Zentall et al., 2014).
So, even pigeons can categorize photographs at both basic and superordinate levels when each discrimination task is given alone. But, humans can concurrently and flexibly categorize the same stimulus at multiple levels, referring to a particular object as a Mercedes Benz, a car, a vehicle, or an artifact. Flexible classification at different levels is thought to be one of the most important and possibly unique features of human categorization ( Markman, 1989). Can animals flexibly classify the same stimulus at both basic and superordinate levels, depending on task demands? And, if they can, then might they differentially learn these basic and superordinate tasks? Lazareva et al., (2004) sought to answer these questions in a study that involved the concurrent training of both basic and superordinate categorization of the same stimuli in the same sessions.
The pigeons saw eight stimuli from each of four basic-level categories: cars, chairs, flowers, and people. The stimuli were color photos of a single target object on a solid grey background in order to control for inadvertent cues (Edwards and Honig, 1987 and Greene, 1983), such as the area, orientation, and dominant color of the target image. From these four basic-level categories, two superordinate categories were arranged: natural stimuli (flowers and people) and artificial stimuli (cars and chairs).
During training, each photograph randomly required categorization at both the basic and superordinate levels. For example, if a photograph of a car was shown along with four choice keys, then the pigeon was required to select the key that was associated with all of the car stimuli. Alternately, if a photograph of the same car was shown along with two different choice keys, then the pigeon was required to select the key that was associated with all of the artificial stimuli (cars and chairs). Both types of categorization trials occurred equally often within the same training sessions.
Pigeons successfully mastered both categorization tasks, thus exhibiting the ability to classify the photos in a flexible way. The birds also mastered the basic-level task more quickly on average than the superordinate-level task. Finally, after the pigeons had mastered both tasks, they were shown eight novel stimuli from each of the four basic-level categories; the birds exhibited similar, robust transfer to novel photographs at both levels of categorization, although performance was lower to the testing images than to the training images. These results suggest that pigeons can flexibly categorize the same photographs, having apparently formed open-ended classes of stimuli at both levels of categorization.
Later research by Lazareva et al., (2006) explored several possible perceptual contributors to basic-level and superordinate-level categorization. Left-right reflection had no effect on pigeons’ classification behavior, whereas top-bottom inversion, blurring, and quartering/scrambling of the photographs differentially impaired their categorization of natural and artificial stimuli. In addition, Lazareva et al., (2010) reported that between-category similarity affects the relative speeds of pigeons’ learning basic-level and superordinate-level tasks; these data may help to explain the occasionally contradictory results that have been reported in the developmental literature (e.g., Mandler and Bauer, 1988, Mareschal and Quinn, 2001, Oakes and Rakison, 2003 and Rosch et al., 1976).

4.3. Application of the paradigm to shape recognition

We have also used our categorization paradigm to investigate the perceptual mechanisms of visual object recognition by pigeons. Since 1992, collaborative work with Irving Biederman (reviewed by Wasserman and Biederman, 2012) has sought to determine how pigeons recognize complex visual objects. Our strategy for stimulus selection and variation was inspired by Biederman, 1987, Biederman, 2007 and Hummel and Biederman, 1992 prominent theory of shape recognition.
The basic assumption of Recognition By Components (RBC) theory is that a small set of geometrical primitive components, called “geons,” can be derived from contrasts of viewpoint-invariant properties of edges—such as straight versus curved or whether pairs of edges are or are not parallel—in the two-dimensional image and the vertices formed at the cotermination of image edges. Detecting these properties is generally invariant over viewing position and image quality. This invariance allows robust object perception when: (a) the image is degraded, (b) the same object is seen from a novel distance or viewpoint in depth or is occluded by another object or surface, or (c) a new instance of the same kind of object is seen. Although RBC has been vigorously tested with humans, we have evaluated its generality with pigeons. A convergence of the data from people and pigeons would strongly suggest the operation of similar perceptual-cognitive processes.
The results of our comparative investigations have been quite revealing. Pigeons show strong control by the individual components of multi-part objects; they are highly sensitive to the spatial organization of an object's several parts; they show some degree of rotational invariance while simultaneously attending to view-specific features of shape stimuli; and, they not only learn about shape, but also encode information about such surface properties as color, brightness, and shading. Because the visual discrimination behavior of pigeons closely resembles that of human beings, it is becoming increasingly plausible that the basic processes of object recognition are mediated by common neurobiological mechanisms that do not depend on linguistic competence or the human brain (Logothetis and Sheinberg, 1996). The pigeon may well become a powerful model system for both behavioral and biological studies of complex visual processing (Cook and Wasserman, 2004, Soto and Wasserman, 2012b, Soto and Wasserman, 2014 and Wasserman, 1991).
Comparative investigations can be expected to continue elucidating the commonalities and disparities between human and nonhuman visual recognition behavior; it should also explicate the relationships between associative learning, object recognition, conceptualization, and language.

4.4. Final philosophizing

I began this paper by introducing Keller and Schoenfeld’s (1950) behavioral analysis of concepts. I later discussed Quine’s (1969) related speculations concerning the role of perceptual similarity in human conceptualization. Permit me to close by adding the ideas of another philosopher who also endorsed the centrality of perceptual mechanisms in conceptualization and who further bolstered Keller and Schoenfeld’s analysis of conceptual behavior in terms of discrimination and generalization—Rand (1990).
Borrowing heavily from Gotthelf’s (2007) excellent summary of Rand’s philosophy, one can outline her perspective on human concepts. Critically, Rand contends that concepts are objective; although concepts are the products of a cognitive method of classification, their content is dictated by reality.
Specifically, Rand proposes that conceptual groupings are based on perceptual resemblances; these she deems to be primitive, unanalyzable similarities, which we select from those that we encounter in our daily experience. Detecting perceptual similarities inherently occurs against a background of perceptual disparities. So, we detect that two plates are similar to one another only against the background of other disparate, yet comparable objects, such as bowls, whose similarity to plates occurs along some quantitative, more-or-less, axis. The plates are perceived to be more similar to one another than they are to the bowls and vice versa.
It must therefore be the case that the perceptually similar items share with the contrasting dissimilar items some comparable property, such as shape in the case of plates and bowls. Rand terms such a comparable property a Conceptual Common Denominator. The formation of concepts is therefore based in reality, with these Conceptual Common Denominators representing the specific, objective axes of comparison.
A concept, for Rand, is thus the result of an integrative cognitive process, which blends a multitude of individual units into a single, integrated unit of thought. This integration creates a new retainable and functional abstract mental unit by being assigned a word. Hence, the essence of humans’ cognitive capacity is the ability to subsume an enormous amount of information via vastly fewer lexical units. Substituting a single, abstract word for the many perceptual exemplars it represents is a process that Rand believes to be uniquely human.
We remain a long way from being able to decide whether Rand’s claim of human uniqueness is correct. Yet, to the degree that we can specify the requisite experiential conditions, we may be able to come much closer to understanding the relationship between word and concept (Wasserman et al., 2015). I would like to believe that future exploitation of our experimental paradigms will help us achieve that goal. Doing so should not be a matter for philosophy; it should be and will be a matter for psychological science.

Acknowledgement

This research was supported by National Institute of Mental Health Grant MH47313 and National Eye Institute Grant EY019781.

References

    • Bhatt, 1988
    • R.S. Bhatt
    • Categorization in pigeons: effects of category size, congruity with human categories, selective attention, and secondary generalization
    • Doctoral Dissertation, University of Iowa (1988) Unpublished
    • Cook and Wasserman, 2004
    • R.G. Cook, E.A. Wasserman
    • Behavioral physiology: visual perception mechanisms
    • Marc Bekoff (Ed.), Encyclopedia of Animal Behavior, Greenwood, Phoenix, AZ (2004)
    • Gotthelf, 2007
    • A. Gotthelf
    • Ayn Rand on concepts. Another approach to abstraction, essences, and kinds
    • Nat. Classif. (2007) Birmingham, UK 12–14 October
    • Greene, 1983
    • S.L. Greene
    • Feature memorization in pigeon concept formation
    • M.L. Commons, R.J. Herrnstein, A.R. Wagner (Eds.), Quantitative Analyses of Behavior: Discrimination Processes, Ballinger, Cambridge, MA (1983), pp. 209–231
    • Griffin, 1992
    • D.R. Griffin
    • Animal Minds
    • University of Chicago Press, Chicago (1992)
    • Hampton, 2001
    • J.A. Hampton
    • The role of similarity in natural categorization
    • U. Hahn, M. Ramscar (Eds.), Similarity and Categorization, Oxford University Press, London (2001), pp. 13–28
    • Full Text via CrossRef
    • Harnad, 1987
    • Categorical Perception: The groundwork of Cognition
    • S. Harnad (Ed.)Cambridge University Press, Cambridge (1987)
    • Keller and Schoenfeld, 1950
    • F.S. Keller, W.N. Schoenfeld
    • Principles of Psychology: A Systematic Text in the Science of Behavior
    • Appleton-Century-Crofts, New York (1950)
    • Markman, 1989
    • E.M. Markman
    • Categorization and Naming in Children: Problems of Induction
    • MIT Press, Cambridge. MA (1989)
    • Matsukawa et al., 2004
    • A. Matsukawa, S. Inoue, M. Jitsumori
    • Pigeon’s recognition of cartoons: effect of fragmentation, scrambling, and deletion of elements
    • Behav. Processes, 65 (2004), pp. 23–34
    • Medin and Aguilar, 2001
    • D.L. Medin, C. Aguilar
    • Categorization
    • R.A. Wilson, F.C. Keil (Eds.), MIT Encyclopedia of the Cognitive Sciences, MIT Press, Cambridge, MA (2001), pp. 104–106
    • Morgan, 1894
    • C.L. Morgan
    • An Introduction to Comparative Psychology
    • Walter Scott, Ltd., London (1894)
    • Oakes and Rakison, 2003
    • L.M. Oakes, D.H. Rakison
    • Issues in the early development of concepts and categories: An introduction
    • D.H. Rakison, L.M. Oakes (Eds.), Early Category and Concept Development: Making Sense of the Blooming, Buzzing Confusion, Oxford University Press, New York (2003), pp. 3–23
    • View Record in Scopus
       | 
      Citing articles (10)
    • Pepperberg, 2002
    • I.M. Pepperberg
    • The Alex Studies
    • Harvard University Press, Cambridge. MA (2002)
    • Premack, 1976
    • D. Premack
    • Intelligence in Ape and Man
    • John Wiley & Sons, Hillsdale, NJ (1976)
    • Rand, 1990
    • A. Rand
    • Introduction to Objectivist Epistemology (Expanded Second Edition)
    • H. Binswanger, L. Peikoff (Eds.)Meridian, New York (1990)
    • Rescorla and Wagner, 1972
    • R.A. Rescorla, A.R. Wagner
    • A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement
    • A.H. Black, W.F. Prokasy (Eds.), Classical Conditioning II: Current Theory and Research, Appleton-Century-Crofts, New York (1972), pp. 64–99
    • View Record in Scopus
       | 
      Citing articles (68)
    • Ristau, 1991
    • Cognitive Ethology: The Minds of Other Animals
    • C.A. Ristau (Ed.)Erlbaum, Hillsdale, NJ (1991)
    • Rumbaugh, 1977
    • Language Learning by a Chimpanzee: The Lana Project
    • D.M. Rumbaugh (Ed.)Academic Press, New York (1977)
    • Smith and Medin, 1981
    • E.E. Smith, D.L. Medin
    • Categories and Concepts
    • Harvard University Press, Cambridge, MA (1981)
    • Soto and Wasserman, 2014
    • F.A. Soto, E.A. Wasserman
    • Mechanisms of object recognition: what we have learned from pigeons
    • Front. Neural Circuits, 8 (22) (2014)
    • Wasserman, 1991
    • E.A. Wasserman
    • The pecking pigeon: a model of complex visual processing? A review of quantitative analyses of behavior (vol. XIII); behavioral approaches to pattern recognition and concept formation
    • M.L. Commons, R.J. Herrnstein, S.M. Kosslyn, D.M. Mumford (Eds.), Contemporary Psychology, 36 (1991), pp. 605–606
    • View Record in Scopus
       | 
      Citing articles (1)
    • Wasserman and Biederman, 2012
    • E.A. Wasserman, I. Biederman
    • Recognition by components: a bird’s eye view
    • O.F. Lazareva, T. Shimizu, E.A. Wasserman (Eds.), How Animals see the World, New York, Oxford University Press (2012)
    • Wasserman et al., 2007
    • E.A. Wasserman, D.I. Brooks, O.F. Lazareva, M.A. Miner
    • A vocabulary test for pigeons (Columba livia)
    • Psychon. Soc. (2007)
    • Whitman, 1919
    • C.O. Whitman
    • H.A. Carr (Ed.), The Behavior of Pigeons, vol. 3, Carnegie Institution of Washington (1919) Publication 257
    • Zentall and Wasserman, 2012
    • T.R. Zentall, E.A. Wasserman
    • Oxford Handbook of Comparative Cognition
    • Oxford University Press, New York (2012)

Fax: +1 319 335 0191.
Copyright © 2015 Elsevier B.V. All rights reserved.
Posted by at