If children are regenerated by the act of baptism, of course it is otherwise; but, not believing this, I am clear that the baptized child of a believer differs from any other unregenerate child, who is not baptized, only in this: that God looks upon it with peculiar interest and love, and that it is surrounded with special and peculiar privileges, opportunities, promises, and hopes, with regard to its being brought to repentance and saving faith in Christ; and by baptism it is initiated into special relationship to the people of God.

The church also has special duties with regard to it.
Some of my brethren give great occasion to those who resist children's baptism, to argue against it as Romish in its nature and effect, by not discriminating clearly in using the words members and membership in connection with children.

Read almost any modern book against infant baptism, and you will find that its main force is directed against the practice as a "church and state" institution, and as making persons members of the church by means of sacraments.

Let us who are really free from such imputation, assert the truly spiritual nature and object of this ordinance.

I wish to see it divested of all that does not belong to it, made eminently spiritual, expressed in terms which cannot easily be misunderstood, and appealing to the natural affections, the understandings, the consciences, of spiritual men and women, as, in its sober and legitimate use, God's great appointment, from the call of Abraham to the millennium, for the increase and perpetuity of his church.[2] [Footnote 2: This subject is discussed by itself, and more at large, in another part of this book.] You are aware that the great question, which has made most of the trouble in the Christian church from the beginning, relates to the meaning and use of sacraments and ordinances, or what we call Symbolism.
The tendency of the human mind, even in Paul's day, as indicated by him, with other things belonging to it, under the name of "the mystery of iniquity, which doth even now work," was, to increase the number of sacraments and ordinances, and make them bear an essential part in the work of regeneration.