Benjamin David Goodman was born on May 30, 1909 in Chicago, Illinois. He was the eighth child of immigrants David Goodman and Dora Grisinsky Goodman, who left Russia to escape anti-Semitism. Benny's mother never learned to speak English. His father worked for a tailor to support his large family, which eventually grew to include a total of 12 children, and had trouble making ends meet.
When Benny was 10 years old, his father sent him to study music at Kehelah Jacob Synagogue in Chicago. There, Benny learned the clarinet under the tutelage of Chicago Symphony member Franz Schoepp, while two of his brothers learned tuba and trumpet. He also played in the band at Jane Addams' famous social settlement, Hull-House.