This dramatic shift in the nature of the neighborhood continued to drive out residents, and between 1860 and 1865 the Eighth Ward, which included the SoHo area, lost 25% of its population. After the Civil War and the Panic of 1873, in the 1880s and 90s, large manufacturers began to move into the area, especially textile firms, and the area became the mercantile and wholesale dry-goods trade center of the city, and was the subject of significant real-estate speculation. This phase came to an end by the close of the 19th century, and as the center of the city continued to move uptown, the quality of the area declined.

After World War II, the textile industry largely moved to the South, leaving many large buildings in the district unoccupied. In some buildings they were replaced by warehouses and printing plants, and other buildings were torn down to be replaced by gas stations, auto repair shops and parking lots and garages. By the 1950s, the area had become known as Hell's Hundred Acres, an industrial wasteland, full of sweatshops and small factories in the daytime, but empty at night. It would not be until the 1960s, when artists began to be interested in the tall ceilings and many windows of the empty manufacturing lofts, that the character of the neighborhood began to change again.

Soho, NYC is a shopping enclave full of high-end boutiques and buzzworthy pop-up shops, as well as a sea of street vendors touting designer knockoffs in the form of sunglasses, handbags and scarves. Just make sure to save some change—and your phone battery—so you can grab coffee and a pastry from one of the many Instagrammable cafes and eateries packed into the downtown nabe. But don’t write off Soho as a mere urban shopping mall. Though many of the art galleries that made Soho New York a contemporary-art hot spot in the 1970s and ’80s decamped to Chelsea and the Lower East Side, some excellent art spaces remain. Walk along the cobblestone streets and find great New York restaurants, bars and things to do in this downtown neighborhood.

After the abandonment of the highway scheme, the city was left with a large number of historic buildings that were unattractive for the kinds of manufacturing and commerce that survived in the city in the 1970s. The upper floors of many of these buildings had been built as commercial Manhattan lofts, which provided large, unobstructed spaces for manufacturing and other industrial uses. These spaces attracted artists who valued them for their large areas, large windows admitting natural light and low rents. Most of these spaces were also used illegally as living space, despite being neither zoned nor equipped for residential use. This widespread zoning violation was ignored for a long period of time, as the artist-occupants were using space for which there was little demand due to the city's poor economy at the time, and would have lain dormant or been abandoned otherwise.

Nevertheless, as the artist population grew, the city made some attempts to stem the movement, concerned about the occupation of space that did not meet residential building codes, and the possibility that the occupied space might be needed for the return of manufacturing to New York City. Pressured on many sides, the city abandoned attempts to keep the district as strictly industrial space, and in 1971, the Zoning Resolution was amended to permit Joint Live-Work Quarters for artists, and the M1-5a and M-5b districting was established to permit visual artists, certified as such by the Department of Cultural Affairs, to live where they worked. In 1987, non-artists residing in SoHo and NoHo were permitted to grandfather themselves, but that was the only extension to non-artists and was a one-time agreement.

The area received landmark designation as the SoHo-Cast Iron Historic District in 1973. In 2005, the construction of residential buildings on empty lots in the historic district was permitted. Nevertheless, with no enforcement of the new zoning laws by the City, beginning in the 1980s, in a way that would later apply elsewhere, the neighborhood began to draw more affluent residents. Due to rent protection and stability afforded by the 1982 Loft Law, in addition to the fact that many of the artists owned their co-ops, many of the original pioneering artists remained despite the popular misconception that gentrification forced them to flee. Many residents have lived in the neighborhood for decades. In the mid-1990s, most of the galleries moved to Chelsea, but several galleries remain as of 2013, including William Bennett Gallery, Martin Lawrence Galleries, Terrain Gallery, Franklin Bowles Gallery, and Pop International Gallery.

SoHo's location, the appeal of lofts as living spaces, its architecture, and its reputation as a haven for artists all contributed to this change. The pattern of gentrification is typically known as the "SoHo Effect" and has been observed elsewhere in the United States. A backwater of poor artists and small factories in the 1970s, SoHo became a popular tourist destination for people seeking fashionable clothing and exquisite architecture, and home to some of the most expensive real estate in the country.