This is not so different from adult second language acquisition. Many people try and fail to learn foreign languages as adults and an important reason many fail to achieve fluency despite spending many hours listening to tapes and studying is that they lack experience in failed interactions that they can learn from.

The babbling part is generally left out of ESL instruction because it's impractical to package up and sell in a profitable manner but it actually works. I found in my studies of Chinese as a second language that the students I attended class with at a very intensive institute would have less ability than people I met outside of class who didn't even intentionally study Chinese but had lovers who were native speakers. The latter would almost instantly develop fluency in cooing type language that gave them a base to explore other type of language interactions in public.

I myself was making very slow progress until one day I learned a phrase that is often mumbled in a jumbled together fashion which means something like "Is that so?" I thought this was a funny phrase to use because nobody could tell if I really knew what I was saying or just mumbling nonsense so I would randomly just repeat it to native speakers and they would assume that I was fluent and then speak back in a stream of words I couldn't understand but became motivated to recall and try to sort out later. From that point on, my progress was steady and I began to develop fluency. But it all went back to basically walking around murmuring at people not knowing what they were saying and trying to guess from the context which is surprisingly easy once you try it.