As Reformed Christians, we confess that Baptism replaces circumcision. Belgic Confession Article 34:
“We believe and confess that Jesus Christ, in whom the law is fulfilled, has by his shed blood put an end to every other shedding of blood, which anyone might do or wish to do in order to atone or satisfy for sins.”
Jews who were circumcised had the sign of belonging to the covenant people, the sign that looked forward to the coming of Christ. Now that Christ has come, they receive the sign that looks back to what Christ has done (Baptism). They are still part of the covenant of grace, but no longer under the old administration of Moses. They are now in the new administration of the covenant of grace, the new covenant (see Hebrews 8). Thus, Paul tells the Jews who come to faith that they must be baptised (Acts 2:38).
“We believe and confess that Jesus Christ…is the end of the law.” So begins Article 34 of the Belgic Confession citing Paul’s words to the church in Rome (Romans 10:4). Because our Lord is the goal (Greek, telos) of all the Old Testament laws and ceremonies, he “has made an end, by the shedding of His blood, of all other sheddings of blood which men could or would make as a propitiation or satisfaction for sin.” Since all the sacrificial rites of the Old Testament were subsumed by our Lord and fulfilled in His work on our behalf, the New Testament writers teach that the sacrifices are ended (e.g., Hebrews 9:23-28). Among the many bloody rituals of the Old Testament was circumcision. Although itself not a sacrifice, the fact that it involved blood showed that it belonged to that epoch of redemptive history in which everything looked forward to a final shedding of blood by the seed whose heel would be bruised by crushing the serpent’s head (Genesis 3:15).
We see this applied in the New Testament when circumcision was being demanded of Gentile converts to Israel’s Messiah (Acts 15:1, 5). The Church rejected this “yoke…that neither our fathers nor we have been able to bear” in its pastoral letter to the Gentiles (Acts 15:23–29). The Confession follows the apostle Paul in saying that circumcision has been “abolished,” for as Paul says, circumcision no longer “counts for anything” (Galatians 5:6; cf. Romans 2:25–29).
In the place of circumcision, Jesus “has instituted the sacrament of baptism” for the New Covenant. What this teaches us is that both circumcision and baptism’s place is that of an initiatory sign of membership in the covenant. There is one covenant of grace and this covenant always has a sign, yet, the administration of the one covenant of grace is multifaceted. Under the Old administration, circumcision was the rite of initiation, while under the New, baptism is the sign of initiation.
The covenant of grace is an everlasting covenant (Genesis 17:7) of which circumcision was a sign during the old administration (Genesis 17:11) and baptism is a sign in the new administration (Colossians 2:11). The sign initiates one “into the Church of God,” that is, into the visible, covenant community. As an initiatory rite, baptism separates us “from all other people and strange religions” as the sign “that we may wholly belong to Him whose mark and ensign we bear.” Under the Old administration of the covenant, slaves bought with money as well as foreigners who desired to join Israel were marked out with the sign and shown to belong to the LORD (Genesis 17:12–13; Exodus 12:43–48). We, too, have been marked out in baptism, in which “the honorable name” was placed upon our foreheads (James 2:7; Revelation 22:4).
These truths, that circumcision has been abolished, along with all other sacrificial rites of the Old Testament, and that baptism takes the place of circumcision, were vital for the ancient church fathers in their apologetics against Judaism. The most famous usage of these themes is found in Justin Martyr’s Dialogue with Trypho the Jew. In it he said, “Wash therefore, and be now clean, and put away iniquity from your souls, as God bids you be washed in this laver, and be circumcised with the true circumcision” (18 cf. 19). He then described that we have not received “carnal circumcision” but “spiritual circumcision” through baptism because the blood of Christ has made the blood of circumcision obsolete (43 cf. 24). Cyprian also used these truths to counter the argument that infant baptism had to be done legalistically on the eighth day just like circumcision in his Letter 58:4.
(Answer adapted from Daniel Hyde's comments on Belgic Confession Article 34)
In some ways, we could affirm that Baptism is an expression of faith, certainly for those who come to faith as adults. The same was true for circumcision, especially for those who were circumcised as adults (e.g. Abraham in Genesis 17 and the second generation of Israelites in Joshua 5).
Abraham believed in the LORD (Genesis 15:6) before he was circumcised (Genesis 17; Romans 4:11). However, he was given the sign of circumcision not as a testimony to his faith, but a sign and seal of the righteousness he had by faith (Romans 4:11). That is, circumcision was a sign and seal of God’s promise to Abraham that he would be his God and the God of his descendants, a promise that through faith in God, Abraham would be made righteous in God’s sight (Genesis 15:6).
See below for further reflection on Colossians 2.
Colossians 2:11-12 is a tricky passage but the analogy suggests that baptism and circumcision have the same spiritual meaning. For Paul, in the new covenant, our union with Christ is our circumcision. In baptism, we are identified with Christ’s baptism (his death) and Christ’s circumcision (Isaiah 53:8) as it were, on the cross. Baptism and circumcision both signify Christ’s death. Circumcision pointed forward to Christ’s death (a bloody ritual), Baptism looks back on Christ’s death.
By faith, we are united to Christ’s circumcision and by union with Christ we become participants in his circumcision/baptism. That is, through faith, the death of Christ is considered as our death (payment for sin) (See also Romans 6:1-11 and Heidelberg Catechism Question 43). Paul’s point here is to teach us about our union with Christ, but along the way we see how he thinks about baptism and circumcision and his thinking should inform ours.
Our fundamental understanding from this passage is that baptism has replaced circumcision as the mark of belonging to God’s people (Heidelberg Catechism Question #74; Belgic Confession Article 34).
John Piper, a prominent Baptist, actually affirms that he also understands baptism to have replaced circumcision: “It’s probably right, therefore, to say that baptism has replaced circumcision as the mark of being part of the people of God. In the Old Testament, men were circumcised to signify membership in the old-covenant people of God, and in the New Testament men and women are baptized to signify membership in the new-covenant people of God.”*
*https://www.desiringgod.org/messages/what-is-baptism-and-how-important-is-it
Strictly speaking, ‘circumcision of the heart’ is invisible and not a covenant sign. It refers to true faith which only God the Holy Spirit can work in the heart of his people (Romans 2:29). The people of Israel were taught from Old Testament times that the sign of circumcision (in the flesh) does not make a person right with God. The sign itself called for repentance and faith (Deuteronomy 10:16; Jeremiah 4:4), a faith which only the LORD can truly work in our hearts (Deuteronomy 30:6).
Reformed Christians do not consider Baptism to primarily be a sign of our faith in Jesus Christ, rather, we understand Baptism (as a sacrament) to be a sign and seal of the promises of God. The focus is on what God has promised (eternal life for those who believe in Jesus Christ) and Baptism is a sign and seal of that promise. According to the sacramental nature of baptism, God is the actor and the recipient is the passive beneficiary.
Acts 21:17-24 narrates the instructions of the Jerusalem church council to Paul. They want him to act in a way that shows that he still has respect for old covenant ceremonial laws thereby confirming that Christians from a Jewish background are free to keep such laws if they wish to do so.
Regarding Baptism, the council explicitly mention that Paul has been teaching Jews not to circumcise their children (Acts 21:21). Clearly it was common knowledge in Jerusalem that Paul had been teaching Jewish Christians that the sign of circumcision was no longer necessary.
In his letter to the Galatians, Paul makes it clear that old covenant ceremonial laws are not binding on Christians and must not be kept for religious reasons. They must not be seen as a means of justification or a testimony to faith. They must not be imposed on Christians from a Gentile background. When Paul submits to the Jerusalem council’s instruction, he is not contradicting himself and suddenly affirming the continued validity of ceremonial laws. Rather, by becoming all things to all men (1 Corinthians 9:19-20), he does what he can to allay the concerns of Jewish Christians and promote peace in God’s church.
The wisdom of Paul’s actions are subject to debate. There is a suggestion that his desire to come to Jerusalem (Acts 20:22) was not in alignment with God’s will (cf. Acts 20:23; 21:4,11; 22:17, 20). There is also the fact that being zealous for the law is seen negatively in Acts 22:3-4 and Galatians 1:14. Whatever the case may be, neither Paul nor the Jerusalem council fought to keep circumcision in place as the sign of belonging to God's people.