John 1.29-34

²⁹ The next day John saw Jesus coming toward him and said, “Look, the Lamb of God, who takes away the sin of the world! ³⁰ This is the one I meant when I said, ‘A man who comes after me has surpassed me because he was before me.’ ³¹ I myself did not know him, but the reason I came baptizing with water was that he might be revealed to Israel.”
³² Then John gave this testimony: “I saw the Spirit come down from heaven as a dove and remain on him. ³³ And I myself did not know him, but the one who sent me to baptize with water told me, ‘The man on whom you see the Spirit come down and remain is the one who will baptize with the Holy Spirit.’ ³⁴ I have seen and I testify that this is God’s Chosen One." John 1 NIV

The first thing I think of when I hear the term "Lamb of God" is a sacrifice, both the sacrifice of what Christ paid for us, but also the idea of the sacrificial lamb that Israel used throughout its history to cover the sins of the people. I also think of the skins that God covered Adam and Eve with at the Garden of Eden after they sinned which most scholars think were sheep. 
The Israelites would have of course thought of the sacrificial lamb and perhaps the Messiah, though most did not expect anything other than a great king who would take away the oppression of the Romans and bring Israel back to the people. 

He was oppressed and afflicted,
    yet he did not open his mouth;
he was led like a lamb to the slaughter,
    and as a sheep before its shearers is silent,
    so he did not open his mouth. Is 53.7 NIV (Vear(

Then is the example of the Passover lamb Exodus 12(Vear)

Later, in Revelation, John refers to Christ as a lamb who looked as though it had been slain: 
⁶ Then I saw a Lamb, looking as if it had been slain, standing at the center of the throne, encircled by the four living creatures and the elders. The Lamb had seven horns and seven eyes, which are the seven spirits^[a] of God sent out into all the earth. Rev 5.6 NIV

So while the Jews had some (depending I am sure on the person) confusion about what the Messiah would do, John the Baptist had no such reservations. I always picture him shouting this with great excitement. John the Baptist then makes the same sort of statement that John had made in the opening portion of this passage. This man who would take away the sins of the world had been before him. Eric Vear p 101 points out that Christ would have been humanly six months younger than John, if you read the other Gospels---but what John the Baptist is speaking of is the eternal quality, the out-of-time quality that God Himself possesses.  Of course Christ was before him! He was, and is and is to come! John says that he did not know at first that Jesus was the Christ, as I have heard that it was likely that they knew each other, being related.
The other Gospels have John questioning Jesus as to if He is the one to come, and then knowing He is the One and baptizing Him---this is the following day, so by now he is certain by all God has shown him that this is the One and Only. BereanCall
The subject of baptism should be covered as well. This was used if a Gentile wanted to become one of the people of Israel, and was part of the procedure of denouncing all false gods and washing away the past life. It was also part of the Jewish life, and was used for various washing away of sins. Vipers, Mack Women, for example, were required to have a mikvah (translates to baptism) after each menstral cycle to wash away and be considered ceremonially clean again. The people took mikvahs for other sin washings, as well as to repent as for what John was having the people do. It was a realization of the sins and drifting away from God, a reaffirmation that oh, yes! I want to be God's child. Also, these immersions were done at the beginning of new ventures, such as a man becoming a priest; any new life change was preceded by an immersion to wash away old sins, old life and start anew, a rebirth. (Vear 88-89, TruthStands  Wedding)
 "The baptism was accompanied by confession. In any act of returning to ADONAI, confession must be made to three people. First, we must make a confession to ourselves.... Secondly, we must make confession to those whom we have wronged...Thirdly, we must make confession to ADONAI.... It is not the one who wants to meet the LORD on equal terms who discovers forgiveness, but the one who whispers through their tears: God have mercy on me, a sinner (Luke 18:13b)."Mack
One last little discussion is the dove. Did the Spirit really descend as a dove? The Bible uses a lot of symbolism, as well as being real statements. It is interesting that "in the Jewish mindset of that day the Spirit would be connected with a dove." If the Spirit was in the actual symbol of a dove, it has meaning to us in that doves were always seen as loving, loyal creatures, full of gentleness. If it was "like a dove" as Jesus appeared in Revelation " a Lamb looking as if it had been slain", then the point is made just as clearly. One way or another, the Spirit made His very good and clear point---which is always the point. (Berean Call)
John was baptizing in water, something that was done over and over again apparently in most people's lives as they changed, drifted away and back to God. Christ's baptizing us with the Holy Spirit was a whole 'nother game. We went from constantly having to sacrifice for our sins to being covered for all the sins were have ever done and will ever do, as well as being given a constant Counselor, guide and, to me, policeman. Someone who nudges us more and more firmly if we attempt to continue to sin, Someone who points out to us what we ought to do for others, Someone to be there in the watches of the night as we cry out alone in fear of what life has in store. Really we went from one way of life to an incredible form of life that the Prophets of old had dreamed of seeing. And the people of Israel and Rome were witnessing the Man Himself walking on Earth, something the rest of us now dream of seeing once more.