Shortly after the state legislature authorized Boston to establish and maintain a public library, some of the city's most prominent citizens began donating books. In 1852 one wealthy banker pledged $50,000 towards the acquisition of new books. By the time the library's first building, a two-room former schoolhouse on Mason Street, opened to the public in 1854, the library's collection numbered about 16,000 volumes.

The library needed more space almost immediately. On January 1, 1858, it moved to a new building on Boylston Street, where it remained for nearly 40 years. In 1887 the renowned architect Charles McKim won the commission to design a library in Copley Square in the city's newly fashionable Back Bay. McKim's "palace for the people" was completed eight years later with seats for 900 readers and shelf space for 628,297 volumes. Only the Library of Congress had a larger collection. The McKim Building, as it would be called, also housed some of the finest interior spaces and some of the most significant works of art created in late nineteenth century America.

The increase in the city's population, combined with the explosion of scientific and technical knowledge, meant that the library would inevitably outgrow the McKim Building. John Deferrari intended that his gift be used to fund a new wing for the Central Library in Copley Square.

Deferrari took pride in the fact that, as he liked to say, he "became a millionaire without the benefit of a banker, a secretary, a bookkeeper, an automobile, or a telephone." But not, he would have added, without the benefit of a public library. Deferrari grew up in a North End tenement during the 1860s. He left school at 13 to help his fruit peddler father. His job was to sell fruit to workers in State Street offices. Daily contact with successful businessmen inspired the young man to improve his lot. He bought a pushcart, then a horse and wagon, and finally a fruit store in Dock Square.

When the Boston and Providence Railroad station opened in Park Square, Deferrari moved to Boylston Street — a choice that would change his life forever. He set up his Quality Fruit Store next door to the building that then housed the Boston Public Library, and began to visit it regularly. There he read books on real estate, law, business, economics, and statistics. He took a keen interest in any subject that might help him become a successful investor and conducted his research with passionate attention to detail.

Over time, John Deferrari invested his earnings from the Quality Fruit Store in stocks, bonds, and real estate holdings. By the time he was 28, he had left the fruit business and was devoting himself to the accumulation and exchange of real estate and securities. He always paid cash and never had an office; he used General Delivery rather than a post office box. He had no faith in banks or lawyers, relying instead on what he could learn at the library.

Three years before he died, John Deferrari expressed his appreciation to the institution that had played such an important role in his life. The original plan to construct a Deferrari wing evolved into a major addition, designed by noted architect Philip Johnson. When the new building opened in 1972, it included a hall dedicated to John Deferrari; a bust of the self-made man was placed in the Boston Room. In July 2016, the library completed a $78 million renovation of the Johnson Building, which brought the building's spaces and services up to twenty-first century tastes and needs.