Q: How was the Bible put together? How did they decide that they needed a new testament and what was included/left out?
A: What a great and foundational question! It is truly unfortunate that more Christians don't ask this question and simply assume that the collection of writings that we base our life and eternal destiny on are reliable and worthy of our trust!
It's important to note at the beginning that not everyone who calls themselves a "Christian" group believes in the same collection of writings referred to as "the Bible." Most Protestant groups agree that "the Bible" is the collection of 66 books found in most modern English translations (e.g. the NIV, ESV, NASB, etc.) which contains 39 books in the "Old Testament" and 27 books in the "New Testament." Some other groups, like the Mormons, add to the canon of Scripture by adding additional books outside of the Bible to the authoritative writings. This isn't an addition to "the Bible" per se, but is an attempt to add to "the canon" which is the group of authoritative books for our faith.
Other groups, like the Roman Catholic and Orthodox churches, have additional books within the Bible itself, in addition to added sections to books held in common. The most common additions to the Bible are referred to as "The Apocrypha" by Protestants and "Deutero-canonical" (literally, "second-canon") by Roman Catholics. These different terms refer to the same sections of the Bible that are found in Roman Catholic translations (e.g. The New American Bible) that are absent in other translations (i.e. the versions listed above and many others).
There is a helpful chart available on wikipedia that shows the range of what is included in the various "canons" of what is considered to be the sacred writings (or Scripture) along denominations and groups that all claim to follow Christ.
One of the reasons that this question is so good, is because it strikes at the heart of what genuinely belongs in the Canon of Scripture ... clearly, what is included in "the Bible" varies depending on which group you are talking to!
For the purpose of this post, it will be impossible to answer the question in a historically comprehensive manner which would explain why each varying iteration or version of the canon exists across the spectrum of Christian traditions.
As a Protestant (for those who like labels), I believe that the canon is limited to the 66 books included in the modern English translations such as the New American Standard and New International Version. However, it is important to understand that this conclusion is based on my reading of history, not simply because of my upbringing! In fact, I was raised in a Christian Tradition that held the Deuterocanonical books to be Scripture.
It is the claim of some groups, that the Church decides what is Scripture. This is not an accurate claim. A more accurate claim is that the Church discerns or discovers what is properly in the canon.
Part of the claim of what is actually "Scripture" is that it was written under the inspiration of the living God. It is wrong to deny either the human or the divine element of such writings, but to be inspired by God means that God (in some sense) gave us these words and is the author. If this claim is true, then clearly only God-authored books are rightly included in the canon.
If this is the case, then no group can make a book "inspired" by including it in a list of books ... God must inspire a book for it to be rightly included in the canon! Therefore, the process of canonization was really a process that attempted to discern all of the books that were properly included in the inspired writings and to eliminate everything that is not inspired from being considered authoritative (at least, as authoritative as those writings that are actually inspired by the living God).
Therefore, the process of canonization extends beyond "the Church" (which began at Pentecost, recorded in Acts 2) to include God's people throughout history. While most groups accept the same list of 27 New Testament books (the Ethiopian Orthodox church has a longer canon that accepts additions to the New Testament writings), the majority of disagreement over the canonical books occurs over the inclusions in the Old Testament (or Hebrew Bible).
Part of the disagreement relies on the fact that the Jewish people rejected the apocryphal books, but many early church councils included these books in their list of canonical writings. Unfortunately, there is no easy way to settle the argument over who is right!
The deciding factor for me personally relies in two major points: 1) the Jewish people are the ones to whom the "Old Testament" writings were entrusted to (e.g. Romans 3:1-2), therefore, they were in the best position to evaluate the canonicity of the writings and the ones to whom this task was entrusted in the first place; and 2) the decisions of the early church to include these writings in the canon still came after Jesus' earthly ministry and the lives of the Apostles! It is incredibly difficult for me to believe that Jesus and the Apostles were using a deficient set of Scriptures!
It is argued by some that the most often quoted version of the Old Testament in the New is the Septuagint (LXX), which was a translation of the Hebrew scriptures into the Greek language. Included in the LXX were the books commonly referred to as the apocryphal or deuterocanonical writings. Therefore, some conclude, that Jesus and the Apostles were giving approval to these additional books and sections which Protestants have ignored and eliminated from their canon.
However, this is not as clear cut as it may seem at first. First of all, it is not certain that the version of the LXX which Jesus and the Apostles used and quoted from did contain the Apocryphal sections since the earliest Greek manuscripts date from the 4th century AD. More important, however, is that even if the Apocryphal sections were included, they were never once quoted by Jesus or the Apostles as authoritative! Even one direct citation in an authoritative manner (e.g. "The Lord says..." or "It is written...") would be a strong piece of evidence for the inclusion of these Apocryphal sections. Unfortunately, for those who claim these sections are authoritative, they have no claim from Jesus or the Apostles. Instead, they have to base their claim on the decision of Christians in the 4th century which is dubious.
In fact, to base this decision on this historical fact is to ignore other historical evidence to the contrary. Several prominent church fathers denied the inclusion of the Apocrypha, including Athanasius, Origen, and Jerome (the translator of the Latin Vulgate).
Quite frankly, it is bold for the Christian Church (composed of believing Jew and believing Gentile) to modify the canon of the Hebrew Bible after the fact! Had Jesus or the Apostles included any citations from the Apocryphal sections as authoritative, I could probably be persuaded, but I am not convinced simply because of the actions of early Christians.
In fact, the second question -- how did they decide that they needed a new testament... -- is helpful for further making the case that the actions of the early church are not always "right"!
I think it would be difficult to make a case that anyone did decide that a "New Testament" was needed. Instead, the book of Acts records that the earliest ministry of the early Christians included them demonstrating that Jesus was the Christ from the Scriptures, which refers to the Old Testament! Never was anyone quoted as saying, "Bummer we don't have new writings to make our claims! How are we supposed to be Christians without a New Testament?"
Instead, the earliest Christians were taught the foundational truth of the Old Testament scriptures and listened to the eye-witness testimony of the Apostles! Over time, writings began to emerge from these Apostles and their companions (e.g. Luke, who was not an Apostle but ministered with Paul) for various reasons.
One of the primary reasons for most of the epistles was because the early Christians were making poor decisions and acting... well, un-Christian! Therefore, to claim early Christian practice as authoritative can be extremely dangerous because many early practices by Christians were wrong!
The epistles to the church in Corinth were at least partially a result of rampant sexual immorality in the church, in addition to abuses and misunderstandings of spiritual gifts and the practice of communion.
The epistle to the Galatians was a result of these Christians attempting to finish by the flesh their Christian walk after beginning by the Spirit. They were falling into a dead legalism, which is an error that is still prevalent in the Church today!
The epistles to the Thessalonians were in part written to address misconceptions about the return of Christ.
Many more examples could be given. The clear truth is that God's people (both Jew and Gentile) throughout history have gotten a lot of things wrong. In the early church, thankfully there were Apostles who were eyewitnesses to Jesus' ministry and who saw Him alive from the dead after He was resurrected bodily from the grave who could address these errors with the same authority that the Old Testament Scriptures carried. The Apostles could claim, "thus saith the Lord..."
Over time, these new writings began to be recognized as Scripture during the lifetime of the Apostles.
Two examples should suffice. First, 1 Timothy 5:18:
For the Scripture says, "YOU SHALL NOT MUZZLE THE OX WHILE HE IS THRESHING," and "The laborer is worthy of his wages." (NASB)
What interesting about this passage is that the Apostle Paul cites as Scripture two passages, the first from Deuteronomy 25:4 and the second from Luke 10:7! Somewhere along the line, the Apostle to the Gentiles recognized that the book his ministry partner wrote about the life of Jesus was rightly considered "Scripture."
The second example is found from the Apostle Peter's writings:
...and regard the patience of our Lord as salvation; just as also our beloved brother Paul, according to the wisdom given him, wrote to you, as also in all his letters, speaking in them of these things, in which are some things hard to understand, which the untaught and unstable distort, as they do also the rest of the Scriptures, to their own destruction. (2 Peter 3:15-16, NASB)
Notice that Peter is discussing the Apostle Paul's writings and mentions that people twist his (Paul's) writings as they do to the rest of the Scriptures! Peter is putting Paul's writings in the class of sacred writings and recognizing them as being authoritative and inspired by God, just like the Old Testament Scriptures.
The two most prominent Apostles put their stamp on these particular writings, recognizing that God was doing something through them and their ministry. They didn't sit down and decide that they needed to write a New Testament, they instead recognized during the process that what was being written was inspired by God and rightly added to the canon because of their inspiration.
This is in accordance with a promise that Jesus made to the Apostles prior to His ascension in John 14:26, "But the Helper, the Holy Spirit, whom the Father will send in My name, He will teach you all things, and bring to your remembrance all that I said to you. (NASB)
As the early church progressed, these writings were recognized virtually immediately as something special and early lists of the canonical books (keep in mind that these writings were written by hand well before the printing press or copy machines were invented ... therefore, lists of the authoritative books were prominent before the writings were ever actually collected together into single volumes like our modern Bibles!) include the same 27 books as our modern New Testament.
As early as AD 160, the church father Irenaeus was using the same four gospels that are included in our New Testament today and by the mid-2nd century the canon was clearly set. Although some discussion continued as to whether or not certain books belonged in the canon, the 27 books that are currently included rightly survived the disputes.
It is important to understand that the books that were disputed were most often disputed because of their difficult content or their perceived lack of Apostolic authority. The list of disputed books in the New Testament includes James, Hebrews, 2 Peter, 2 & 3 John, Jude and Revelation. Ultimately, the discussion for possibly excluding these books was not as strong as their original inclusion and recognition amongst the inspired writings.
I would highly encourage you to read this short transcript of a Q&A session with one of the most prominent scholars in the field of biblical studies, particularly as it relates to the historical reliability of the Old and New Testaments and the resurrection of Jesus of Nazareth. Dr. William Lane Craig gives a concise overview of virtually the same question as our Questioner and gives some different information that is helpful.
In conclusion, God has entrusted his oracles to His people throughout history and the people who received those writings were in the best place to recognize, preserve and protect those writings. The Jewish people preserved the 39 books included in our present, Protestant canon, which was validated by Jesus and the Apostles. The early church was entrusted with the writings of the Apostles and their ministry partners and were preserved despite intense persecution in the earliest stages of the church. Those brothers and sisters of our who were willing to lose life and limb for the Apostolic writings were not interested in losing either for spurious books that popped up such as the Shepherd of Hermas or the gnostic Gospel of Thomas.
The 66 books we have today in the canon are the full extent of the writings inspired by God and we should not expect new revelation to come or new books to be added. Our faith is a faith of revelation, completely dependent upon God revealing Himself and giving us the necessary information. He has delivered this faith once and for all in the Scriptures (Jude 1:3).
The God who created this world is able to reveal Himself and to preserve His writings for His people. Studying the history of canonization should inspire greater faith in our modern Bible as it is historically reliable and was assembled by those who were in the best position to evaluate and receive these truths for the glory of His name.