For one thing, if this were true, why hasn't the effect already been overwhelming? Religion isn't exactly a modern phenomenon.
ericjamesstone offered some speculation in an interesting response:
One possible explanation would be that up until the birth control era, men and women who were not predisposed toward religiosity still had a lot of offspring, due to engaging in offspring-producing behavior.
Another would be that up until concerns about overpopulation caused people to start limiting the number of children they had based on ideology, non-religious people may not have had any additional reason to limit the number of their offspring as compared to religious people. So while concerns about the expense of having children, etc., might affect religious and non-religious alike, concerns about limiting human impact on the environment might reduce the reproductive rate of the non-religious more than the religious.
While neither especially agreeing nor disagreeing with Eric (unusually so for the two of us, in fact), his remarks got me to thinking. I find I question some of the assumptions he proposes.
Prior to the Industrial Revolution, for most people their best available labor source as well as their most likely form of eldercare (as it were) came from having as many children as possible. Infant mortality rates meant that if you wanted surviving adult children, you didn't stop to celebrate after the first birth or two and focus on raising them. Offspring-producing behavior was highly incented regardless of one's faith holding. So I think I'm good with his first 'graf.
However, I'm not sure it's ever been true that large number of people limit the number of children they have based on ideology. There's a very strong inverse correlation between economic status and family size that's demonstrable over and over again across many societies. I won't pretend to have any deep understanding of that, but I would hazard a speculation of my own that the marginal cost of another child rises as your planned investment in each child rises. Another mouth to feed can be critically expensive if you don't have enough food to go around, but if you're living well enough above subsistence to be worried about things like eventual college tuition (to stretch the point), another mouth is also very expensive within the resource calculus of that family.
But I don't think that's ideology. I think that's response to one's perception of one's own resource constraints. What Eric refers to above as "concerns about the expense of having children". While surely people have restricted child bearing for ideological or religious reasons (when's the last time you met a Shaker?), so far as I'm aware those positions have never been taken up by mass movements with enough force to impact large-scale demographics. Eric's "concerns about limiting human impact on the environment", in other words.
I don't believe the genetics holds up in any case. So far as I know there's a tendency to normalize back to the center. This is the central intellectual error of the eugenics movement. I'm also not sure the sociology holds up. To be clear, I'm riffing here. I don't have research to back any of this up. I'm just responding to speculation with speculation of my own. But it's definitely an interesting question.