From the Pershing Field Garden Friends website:

As one enters Pershing Field on Summit Avenue, there is a large arch made of reddish colored stone; it is the only remnant of the Fourth Regiment Armory building that once stood at Montgomery Street and Bergen Avenue. After the armory building was destroyed in 1927, the arch was reconstructed in Pershing Field and provides a dramatic entrance to the passive side of the park. Both the Arch and the old Armory trace their history to that of the city’s Fourth National Guard Regiment whose origin goes back to just after the Civil War.

Read more about the history and current events on their website.