One of the highly recommended books in regard to the doctrine of Scripture, is B.B. Warfield's The Inspiration and Authority of the Bible (P&R Publishing). As to the specific question concerning the formation of the Canon, from that book, you can read what Warfield wrote here: The Formation of the Canon of the New Testament.
Warfield writes,
Quote
But from the time of Irenæus down, the church at large had the whole Canon as we now possess it. And though a section of the church may not yet have been satisfied of the apostolicity of a certain book or of certain books; and though afterwards doubts may have arisen in sections of the church as to the apostolicity of certain books (as e. g. of Revelation): yet in no case was it more than a respectable minority of the church which was slow in receiving, or which came afterward to doubt, the credentials of any of the books that then as now constituted the Canon of the New Testament accepted by the church at large.
The canon of the New Testament was never universally accepted by the Early Church. Doubts were expressed regarding Hebrews, 2 Peter, 2,3 John, James, Jude, and Revelations. These books could be read in the churches but could not be used independently to formulate doctrine. Francis Pieper writes in Christian Dogmatics, Vol. 1,
Quote
For the Scriptures of the New Testament we have the historical witness of the Early Church (ecclesia primitiva). Its witness is unanimous as to the Four Gospels, the Acts of the Apostles, the thirteen Epistles of Paul, the First Epistle of John, and the First Epistle of Peter (homologoumena). But as to the canonicity of the Epistle to the Hebrews, the Second Epistle of Peter, the Second and Third Epistles of John, the Epistle of James, the Epistle of Jude, and the Apocalypse, doubts, more or less strongly expressed, were entertained (antilegomena). Eusebius in his Church History lists the homologoumena and the antilegomena.The historical fact that the Early Church differentiated between the homologoumena and the antilegomena cannot be changed by a resolution of the later Church. Luther, too, abides by this judgment of the primitive Church; he says, appealing to Eusebius (Church History III, 25), that in ancient times the Epistle to the Hebrews, the Epistles of James and Jude, and the Apocalypse “had a different reputation.” He finds much excellent instruction in the antilegomena, grants that the offensive passages may be explained acceptably by “glosses,” and will keep no one from appraising them as he sees fit. But he will not class them with the “right certain chief books of the New Testament.” As for himself, he will let the doubt entertained by the Early Church remain. Chemnitz denounced the action of the Roman Catholic Church in declaring the Apocrypha of the Old Testament and the antilegomena of the New Testament a part of the canon of Scripture by a mere decree and in anathematizing all those who refused to accept the canon fixed in the Vulgate, as anti-Christian.
I'm not sure about other protestant denominations but Lutherans continued to have reservations regarding the antilegomena books well into the 17th century. Pieper quotes C. F. Walther, one of founders of the Missouri Synod,
Quote
For our dear fathers in the faith, with hardly an exception till after the time of the Formula of Concord, regarded and declared all or at least some of the antilegomena as not belonging to the canon; and they did that not from hastiness or levity toward the Word of God, but, on the contrary, because they were very conscientious with regard to the Word of God. Luther’s opinions on the antilegomena are not a “blot” on our Church, but they rather bear witness how careful our Church once was in determining the standard and norm of our faith and life. The summary decrees of the Papists and the Reformed that all the antilegomena must be received as canonical by all Christians on pain of losing their salvation are so little a testimony for the high regard of these denominations for the Word of God that they rather demonstrate how easy it is for those to add something to the canon who hold that the Scriptures are to be interpreted either, in a blind collier’s faith, according to the whim of the Church (that is, of the Pope) or according to the principles of reason. It will therefore not be improper to submit here the testimony of our fathers, particularly of the 16th and the first half of the 17th century; not that we personally hold these opinions, but in order to show that doubts as to the canonicity of the disputed books were held also by men whose orthodoxy no Lutheran would dare to deny, and thus to clear a man like Luther of the suspicion that he had brazenly, in his subjective pleasure, passed judgment on books which had been received into the New Testament Canon.