top of page

Defending the Mitonuclear Compatibility Species Concept

In early March, The Auk: Ornithological Advances, published a paper in which I presented a new species concept based on a new model for the process of speciation. This new speciation model focuses on the interactions of coadapted sets of mitochondrial and nuclear genes. This speciation model was not a completely novel conception on my part. Several previous paper highlighted the importance of mitonuclear interactions in the process of speciation, most notably:

Lane, N. (2009). On the origin of bar codes. Nature 462:272–274. doi: 10.1038/462272a

Gershoni, M., A. R. Templeton, and D. Mishmar (2009). Mitochondrial bioenergetics as a major motive force of speciation. Bioessays 31:642–650. doi: 10.1002/bies.200800139

Burton, R. S., and F. S. Barreto (2012). A disproportionate role for mtDNA in Dobzhansky–Muller incompatibilities? Molecular Ecology 21:4942–4957.

But I further developed the ideas and presented them as a new species concept applied to birds:

Hill, G. E. 2017. The Mitonuclear Compatibility Species Concept. The Auk 134:393-409

I limited my argument to birds not because I thought that speciation is fundamentally different in birds compared to other complex animals. (As an aside, I do think that speciation is fundamentally different in plants, but plants are weird eukaryotes. And, I think that the concept of species should not be applied at all to prokaryotes). I limited my argument to birds because I do think that the process of speciation in birds is simpler and less confounded by complicating variables (like Wolbachia parasites) than in other taxa of complex animals. A key rule of thumb in a scientific investigation is start with the simplest systems and move to more complex systems once a foundation of understanding is established. I started with birds because avian speciation is as basic as speciation gets.

A few days after my paper came out, Dr. Jerry Coyne, who is a professor at the University of Chicago and widely regarded as one of the most knowledgeable scientists regarding speciation, wrote a critique of my ideas in his widely-read blog, Why Evolution is True. I hesitate to start a blog war about speciation, but Dr. Coyne has critiqued my species concept in a blog, so I’ll respond in my blog (The Ornithologists’ Blog), which to this point has been much more about birdwatching and bird photography than science. I guess it is time to turn my blog more toward science and what better way to start than a blog exchange with the great Jerry Coyne. So here goes.

I’ll start by stating that I have great respect for Dr. Coyne. He has made substantial contributions to evolutionary biology and particularly speciation and his is not an opinion to dismiss lightly. In this case, however, I contend that his critique is based on an unfair and inaccurate presentation of my ideas, and he's wrong.

I’ll take the statements line by line. Dr. Coyne’s comments are highlighted in orange font to make it easier to follow who said what.

Coyne: Hill’s idea is that speciation in birds proceeds largely through the mitochondria of one isolated population evolving divergently from the nuclear genomes of another population, so when the populations encounter each other after a long period of isolation, the mitochondrial genes of one species are mismatched with some nuclear genes from the other, and the hybrids become either sterile or inviable. That would make them different species if the hybrid problems are severe as gene flow between the populations would be very low.

Yes, divergence of coadapted sets of mitonuclear genes is the foundation of speciation. But “severe” incompatibility is not invoked. Certainly, sterility is not invoked. As Trevor Price has documented very thoroughly, birds remain fertile, almost universally, across species boundaries. They remain fertile very commonly across entire genera. They frequently are fertile even in between-genera crosses. So, if infertility were invoked relative to birds, the mitonuclear compatibility species concept would be a non-starter. The mitonuclear compatibility species concept requires that hybrids have reduced fitness in the natural environments. The amount of time needed for divergence is also not explicitly stated in the theory. If adaptive divergence is invoked, then divergence in coadapted mitonculear gene complexes can be very fast on an evolutionary scale (I present that idea in an earlier essay ( Ecol Evol. 2016 Aug; 6(16): 5831–5842).

Coyne: The strongest evidence Hill has for his hypothesis is that for the two bird “species,” the blue-winged warbler and golden-winged warbler have very low divergence in the nuclear genes (0.03% to be exact), but the mitochondria differ much more strongly—3%. They are considered species because they have different markings and maintain their marking distinctness when they meet.

This is the statement is a serious mischaracterization of what I wrote. The Blue-winged and Golden-winged Warbler complex is far from the strongest evidence for the mitonuclear compatibility species concept (although in Dr. Coyne’s defense, I did include an entire section on these species complex as an example of how my theory would apply to the toughest and most confusing situations in birds). The strongest evidence for the mitonuclear compatibility species concept is that 95% of bird species (the boring species) have a distinctive mt DNA barcode sequence, very rarely hybridize, and have species-typical song and plumage color that transition in perfect sync with a DNA barcode gap. The strongest evidence for my idea is that between most sister taxa of birds, there is introgression of nuclear but not mitochondrial or z-linked genes.

Blue-winged and Golden-winged Warbler present a tough case for any species concept, including, as I point out in my paper, the Biological Species Concept and the Phylogenetic Species Concept. These two wood warblers are species, literally, because a committee voted that they are distinct species. Across a century of committee meetings, that vote has been unanimous because Blue-winged and Golden-winged Warblers have very distinct plumage, very distinct song, and different ecological niches that are quite distinct in niche modeling. From a non-genomic-based perspective, they are as different as any pair of sister wood warbler taxa.

I contend that it is in this species complex in which the value of the mitonuclear compatibility species complex is clearly revealed. Coyne’s genomic summary is correct. A recent study shows that there is almost complete introgression of nuclear genes but 3% divergence in mitochondrial genotype.

Toews, D. P., S. A. Taylor, R. Vallender, A. Brelsford, B. G. Butcher, P. W. Messer, and I. J. Lovette (2016b). Plumage genes and little else distinguish the genomes of hybridizing warblers. Current Biology 26(17):2313-8

Coyne: Hill’s “mitonuclear compatibility species concept” proposes that when a certain degree of genetic difference between mitochondria of different groups is seen, that is indicative of mitonuclear incompatibility, and the groups should be called different species:


Coyne: But he doesn’t say how much difference between mitochondrial DNA would mandate a diagnosis that two populations are different species.

I state that speciation has occurred when populations reach a point of having uniquely coadapted mt and NO–mt genes, where “uniquely coadapted” is defined by loss of fitness in the hybrid offspring that carried mt genotypes. Thus “how much difference in mt DNA” is defined not by a percent divergence but rather by the functional consequences of specific divergences. As little as a single nucleotide change could be the key to speciation. In practice, however, we can expect that percent divergence is a good proxy for divergence in mitonuclear compatibility.

Coyne: But why couldn’t the speciation of these groups have involved sexual selection, so that they’ve diverged in both male color and female preference, rather than hybrid inviability due to mt/nuclear DNA divergence?

Dr. Coyne is asking here why prezygotic mechanisms of isolation (aversion to mating with the wrong species) could not explain the pattern. Mitonuclear incompatibilities are post-zygotic (after mating) isolating mechanisms. My entire career has been focused on female mate choice relative to coloration in birds, so we are in my wheelhouse here. Just I as I would never presume that Dr. Coyne had not considered Haldane’s rule in discussion of gene flow (see below), it would be unfair to think that I hadn’t considered female mate choice based on male coloration when considering avian speciation. I came to my ideas regarding speciation from a male coloration/female mate choice perspective.

If the patterns among birds were consistent with the idea that pre-zygotic isolating mechanisms drive speciation, I would have embraced that explanation. I would have never needed to invoke mitonuclear incompatibilities. But the pattern is not consistent with what we would expect if sexual selection drove the divergence of populations.

Perhaps most directly to the point, a detailed study of social pairing and both social and extra-pair matings was conducted in Ontario on a population of Golden-winged Warblers into at the edge of the Golden-winged/Blue-winged hybrid zone.

Vallender, R., Friesen, V.L. & Robertson, R.J. Behav Ecol Sociobiol (2007) 61: 1797. doi:10.1007/s00265-007-0413-3

This study revealed that pre-zygotic isolation is working very poorly in these warblers. The species-typical plumage pattern seemed to have little effect on pairing success. Poor pre-zygotic reproductive isolation is the basis for the high proportion of birds in this population showing mixed phenotypic characters as well as introgressed nuclear genotypes. Breakdown of prezygotic isolating mechanism is thus documented in field observations of breeding warblers as well as in the nucleotide sequences of the warblers. Toews et al (reference above) found evidence that exchange of nuclear genes has been occurring for millennia. So how do we explain a 3% divergence in mt DNA between these two populations in the face of breakdown of reproductive isolation between the species? Dr. Coyne knows the answer to this better than anyone. He literally wrote the book on it. Post-zygotic mechanisms of isolation.

Coyne: To Hill, the warbler mtDNA divergence suggested that the big divergence of mitochondria played the major role in preventing gene flow between these species. But this is problematic for several reasons.

First, for a deleterious mitochondrial mt/nuclear DNA interaction, the nuclear DNA would have to have diverged as well in some places. We know that there are six regions in these species that have marked divergence in nuclear DNA, and these include the genes for body color (there are also ecological differences). But why couldn’t the speciation of these groups have involved sexual selection, so that they’ve diverged in both male color and female preference, rather than hybrid inviability due to mt/nuclear DNA divergence?

The problem with pre-zygotic isolation is that it is empirically falsified as stated above. But Dr. Coyne’s comment gets to a central issue and highlights a clearly testable prediction of my idea. For the current situation to work, there must be divergence not just in mitochondrial genotype but also in at least one (and presumably a set of) nuclear genes that are coadapted with the mitochondrial genotype. There should also, in general, be linkage between species-typical ornamentation and coadapted nuclear genes. I predict that N-mt genes and ornament genes will be Z-linked, and I’ll make that argument again, below, when we get into Haldane’s rule. From the Toews paper, we know that at least some key genes for species-typical plumage coloration in this warbler complex are NOT Z-linked. But then we also know that this is an exceptional species pair for high hybridization and flow of nuclear genes. What is interesting is that among the nuclear genes that are differentiated among species, a disproportionate number are on the Z-chromosome. Are any nuclear genes that interact with mitochondrial genes (N-mt genes) among these diverged Z-linked genes? Nobody knows at this point but it is now a straightforward prediction to test.

Coyne: Or maybe ecological differentiation plays a role. One problem with Hill’s theory, pointed out by evolutionists Darren Irwin and Brian Sidlauskas in the Monitor piece, is that it assumes that nuclear/mtDNA incompatibility is the cause of speciation, when it could (if it even exists; see below) might have followed speciation that had already occurred through sexual selection or other processes.

All of this arm waiving fails to face up to the empirical reality of these warblers: there is divergence in mitochondrial and Z-linked genes and almost complete introgression of autosomal nuclear genes. These observations are best explained by coadapted set of mt and N-mt genes with the N-mt genes Z-linked.

Coyne: But if that’s the case, why are the mitochondria so diverged compared to the nuclear DNA? I have my own theory on that, which I imparted to Ms. Botkin-Kowacki, but which she didn’t mention in her piece. In birds, females are the heterogametic sex (the sex with unlike sex chromosomes), having ZW females and ZZ males. This is the reverse of the situation in mammals and insects, in which males are heterogametic, with XX chromosomes in females and XY chromosomes in males (Lepidoptera are like birds in this respect.)

We also know, in a phenomenon called “Haldane’s Rule” (after biologist J.B.S. Haldane), that in hybrids between species and populations, if only one of the two sexes is sterile or inviable, it is almost invariably the heterogametic sex. So in species hybrids in birds and butterflies, the females are often sterile or inviable, while in mammals and insects it’s usually the males. I spent much of my career working on this phenomenon, and my work and that of others have suggested some explanations, which I won’t go into here.

If Haldane’s rule applies in these warblers, then the female hybrids could be more sterile or inviable than males. Since females but not males pass on mitochondrial DNA, this would—because female hybrids couldn’t mate with males of either parental species—prevent the mitochondrial DNA from moving between the two species. This wouldn’t apply to nuclear DNA, which could move between species when male hybrids mated with either parental species. But those male hybrids wouldn’t pass on their mitochondrial DNA, which is transmitted only by the female parent. This phenomenon alone would account for the disparity in mitochondrial versus nuclear DNA divergence, without having to invoke any bad interactions between mitochondrial and nuclear genes. It’s simply a phenomenon of genetics–if part of the DNA can’t move between species, then that part will diverge faster. And that would knock down Hill’s strongest evidence. (His evidence for his theory isn’t very strong anyway, though we have seen the phenomenon in some copepods).

I agree with Dr. Coyne here (not that “evidence for his theory isn’t very strong anyway”) that Haldane’s rule is playing a big role in avian speciation. Mitonuclear interactions actually provide a mechanism for how Haldane’s Rule is working. The copepod example, which Dr. Coyne briefly mentions, provides some insight. Copepods don’t have sex chromosomes—all nuclear genes are on autosomes. For that reason, in the F1 generation, both male and female hybrids do ok. All F1 hybrids get at least one maternal copy of all nuclear genes. That means that there is a compatible gene to match mitochondrial genes across the genome. However, because of recombination, by the F2 generation, in some individuals end up with two copies of the paternal nuclear genes at any given locus and if that locus holds nuclear genes that interact with mitochondrial genes then there will be mitonuclear incompatibility and hybrid breakdown. In one of a series of brilliant papers, Dr. Ron Burton called this the “sorry state of F2 hybrids”. Very tellingly in copepods, hybrid dysfunction is a result, specifically, of poor function of Complexes I, II, IV, and V of the electron transport system, all of which require coordinated function of mitochondrial and nuclear genes. Complex II, which has no mitochondrial subunits, functions fine in hybrids. Moreover, aside from hybrid dysfunction from mitonuclear incompatibilitieis, there is a fitness advantage from nuclear hybrid gene combinations. In other words, there is hybrid vigor from novel nuclear gene pairings. Hybrid dysfunction comes from mitonuclear incompatibilities. Burton and his students have spent over a decade demonstrating these key observations in fantastic detail. Here’s a great summary of the work:

Burton, R. S., R. J. Pereira, and F. S. Barreto (2013). Cytonuclear genomic interactions and hybrid breakdown. Annual Review of Ecology, Evolution, and Systematics 44:281–302.

Birds do have sex chromosomes, and as Dr. Coyne described, females are the heterozygous sex (ZW) while males are the homozygous sex (ZZ). So in F1 hybrids, females get all of the Z-linked genes from their daddy while they get all of their mitochondrial genes from their mommy. As Dr. Coyne explains, it has long been known that in hybrid crosses the heterogametic sex does not do as well as the homogametic sex. J. B. S. Haldane wrote about this 100 years ago and it is known as Haldane’s Rule.

Dr. Coyne dismisses a direct role of mitochondrial genes in creating this pattern and instead argues that there must be nuclear incompatibility factors on the Z chromosome that cause loss of fitness when expressed with nuclear genes from the other species. This loss of fitness in ZW hybrid individuals (females) would stop the introgression of mitochondrial genes because mitochondria are maternally transmitted. The identity of these nuclear incompatibility factors is not discussed.

As I counter argue, I propose that the patterns that we see-- lack of introgression of Z-linked and mitochondrial genes but lots of introgression of autosomal genes—are better explained by interacting mitochondrial and nuclear genes. If nuclear genes that interact with mt genes are Z-linked, females from hybrid crosses will do poorly because they will have mismatched nuclear and mt genotypes. Males will do ok for the same reason that F1 copepods do ok. This opens the door to gene flow, because these F1 males can mate. But mt and Z-linked genes cannot introgress into the foreign population because they become low-fitness alleles in the foreign mitochondrial and nuclear background. Hence the common pattern in birds: introgression of nuclear autosomal but not mitochondrial or Z-linked genes.

Coyne: There are other problems with Hill’s theory:

  • It’s subjective: how much divergence between mitochondrial DNA of two groups would make them count as different species? While the Biological Species Concept (BSC), which counts species as different if gene flow between them is severely impeded, is subjective in some cases of incomplete gene flow, in many others it’s objective: humans can’t exchange genes with chimps, or Drosophila simulans with D. melanogaster (hybrids are completely sterile or inviable).

I strongly disagree here. Previous species concepts are highly subjective. Why do you think we need a committee and a vote to decide whether or not a bird population is a species or not? I would argue that the Biological Species Concept is among the most subjective major theories in all of biology. In contrast, my ideas can be falsified in a straight-forward manner with data. The predictions are clear. The falsification would be definitive. And, by the way, humans probably can exchange genes with chimps—here prezygotic processes are working just fine.

  • Hill’s species concept is merely a subset of the BSC: it’s just one of many ways that gene flow can be interrupted, and we now of many species, like ducks, that maintain distinctness through other reproductive barriers not involving mt/nuclear DNA problems.

In my opinion, the Biological Species Concept is a subset of the Mitonuclear Compatibility Species Concept. The Biological Species Concept is an archaic construct proposed before the age of modern genetics and before the mitochondrial genome was even discovered. It was a brilliant idea in 1940. After 80 years of advances in genomics, it is time to move on from this theory.

  • Hill’s species concept isn’t general: we know of many species in other groups for which gene flow is prevented not by hybrid sterility or inviability, but things like differences in ecology, mating preference, or time of mating, or use of different pollinators.

This is a backwards argument. The need for mitonuclear coadaptation is fundamental to all eukaryotes. The hypothesis that I propose is the most general species concept ever proposed. To counter a species concept based on fundamental characteristics of all eukaryotes with a laundry list of special cases is not convincing.

  • As Hill admits, there is no direct evidence in any bird species for hybrid problems being caused by deleterious interactions between the mitochondrial DNA of one species and the nuclear DNA of another. It’s a a purely speculative theory based on observations, like the warbler data, that have other and better explanations.

The circumstantial evidence for mitonuclear speciation in birds is substantial. An idea cannot be tested until it is proposed. My idea will be quickly rejected if it is wrong. It will change the game if it is right.

  • In other groups like mammals and flies, it is the mitochondria that move easily between species while the nuclear DNA is more divergent. This is explained by Haldane’s Rule (in those groups fertile hybrid females can move mtDNA between species), but not by Hill’s idea that mitochondria are genetically incompatible with nuclear DNA.

The problem with this statement is that species boundaries have been created with incorrect concepts. Birds are a great starting place in revising speciation theory because we can see and hear the sexual signals that demarcate avian species. If we stripped away all color and sound from birds, we would have a taxonomy as screwed up as the taxonomy of most animals, and we would claim many more cases of mitochondrial introgression. I think the mitonuclear compatibility species concept will work very well in Drosophila for instance.

bottom of page