Darwin turned wholeheartedly to the problem of evolution. Ever since his Beagle trip he had been convinced that the difference between what naturalists called 'varieties' and what they called 'species' was much less significant than previously thought. If pigeon breeders could create varieties as different as pouters, runts, and fantails, what would prevent nature from doing the same? And, given millions of years, wasn't it possible that a pigeon could be turned into something so radically different we would no longer be willing to call it a pigeon–or even a bird?
Darwin was not the first to have these kinds of thoughts. Seventy years before, his grandfather, Erasmus, had devoted a whole section of his book Zoonomia to the issue of evolution. In 1844, Robert Chambers anonymously published his controversial book, The Vestiges of the Natural History of Creation, a sweeping history of the cosmos that came down strongly on the side of evolution, largely on the evidence of fossils (the 'vestiges' of creation). The book was flawed, but popular, and it brought the idea of evolution into the public eye. The opposition to evolution was still strong, but it included among its number a wide range of opinions, from those who thought that all species had been created at the beginning of the world in the same form as they now had, to those who thought that new species were being continuously created to fill new environmental niches, to those who thought that variation within species was within Nature's power but the creation of new species remained in God's hands.
Darwin had two things to contribute to this debate: a wealth of observations on adaptation, and, more importantly, a theory that could explain how new adaptations arose without the guiding hand of a divine Creator. His observations were gained by his own experience on the Beagle, his eight painstaking years of work on barnacles, and the advice and expertise of friends like Hooker. His theory was his own creation. Darwin solved the problem of evolution by pointing to a mechanism that depended on nothing but variation and chance: natural selection. Many more individuals were born than could be supported by the environment, which meant that some had to die. Which ones died? Obviously, those that were least well adapted to the environment. Given that there is variation in the population, and that that variation is heritable (i.e. can be passed from one generation to the next), it was clear that the most useful adaptations would be preserved. If enough of those adaptations were accumulated, a new species could arise.
Although this sketch of the theory was already in place in Darwin's notes, in 1854 he was still struggling with a few pieces that he had yet to make sense of. One had to do with the population of islands like the Galapagos. According to his theory, animals and plants had arrived on the islands millions of years before and had slowly adapted to fit the unique environments on each island. But how had they gotten there in the first place? He was unwilling to accept the possibility that these volcanic islands had once been closer to the mainland of South America. Instead, he tried to prove that seeds and even eggs might have been transported on ocean currents from the mainland. He conducted experiments: soaking seeds in salt water for weeks to see if they would still germinate (most did) and figuring out which seeds would float (most didn't). The second puzzle piece was why there was such a great diversity of life in the world. If every species was continuously adapting to fit the environment as best it could, why didn't all species converge to the same form? Shouldn't there be some 'best species' that would dominate all others? Darwin solved this problem by drawing an analogy with modern industry. It was not true that there was a single 'best job' in any task. In fact, production got more efficient the more specialized each worker became. The same held true in the natural world: species specialized so that they could capitalize on particular aspects of the environment. In fact, a species that was failing in the competition in one particular environmental niche could become startlingly successful if it simply shifted niches so that it was no longer in head-to-head competition similar species.
To shore up his understanding of variation under artificial selection–the kind of selection that had produced pets and domestic farm animals–Darwin started to learn all he could about pigeons. Breeding pigeons was a pastime that few aristocrats threw themselves into, but Darwin eagerly built a shed in the yard behind Down House. He started making trips into London to speak to the professionals, downing beers with them while they boasted about how they could see differences of 1/16th of an inch between two pigeon's beaks. Darwin studied not only living, breeding pigeons, but also dead ones; for a while, his workshop became a shop of horrors as he killed and 'skeletonized' pigeons of all varieties and ages, not to mention the occasional rabbit or chicken, studying the sometimes striking differences in structure between different varieties.
Meanwhile, Darwin was testing out his theory of evolution on friends like Hooker and zoologist Thomas Henry Huxley. No one was entirely convinced yet, but some progress was being made. When Lyell, the geologist whose Principles of Geology had so inspired Darwin while on the Beagle, found out about Darwin's theory, he urged him to publish it as soon as possible, if only to ensure that he would not be scooped. Darwin demurred: he was much more interested in producing a watertight case for evolution by natural selection than in gaining credit for a theory that could be easily dismissed. Nonetheless he started working on a manuscript in May of 1856. It started as a sketch, but it quickly became obvious that, when completed, it would rival Lyell's Principles in length, and would probably have to be published in multiple volumes.