Graniteville was originally named Bennett's Corners and Fayetteville. Quarries for trap rock were operated in the area from 1841 to 1896, leading to the community's name being changed first to Granite Village, then Graniteville.

According to College of Staten Island professor, geologist Alan Benimoff, this name is a misnomer, as the quarry actually contained no granite. Granite is made of potassium feldspar. The rock at the quarry is composed of sodium feldspar. Also, professor Benimoff discovered Trondhjemite at the site, a rare mineral.

Largely rural well into the 20th century, Graniteville was the scene of a notable fire in March 1942, when an explosion at the Consolidated Fireworks Company on Richmond Avenue claimed the lives of five employees.

Bisected from east to west by the Staten Island Expressway and with New York State Route 440 forming its eastern boundary, Graniteville's most notable landmark is Baron Hirsch Cemetery, an 80-acre (324,000 m²) Jewish cemetery founded in the late 19th century and still in active use; in August 2001 this cemetery became the focus of a local health-related story when it was one of six locations on the island where mosquitos carrying the West Nile virus were discovered.

Residential development in the area burgeoned soon after the Verrazzano-Narrows Bridge opened in November 1964, and today Graniteville stands at the north end of a busy commercial strip along Richmond Avenue; this strip, which extends to the vicinity of the Staten Island Mall approximately 4 miles (6 km) away, is the scene of some of the heaviest automotive traffic on Staten Island.