The Chronicle reported that the Department of Health and Human Services issued a memo last week calling for employees to "maximize the use of telework," due to conditions around the building.
Former House Speaker Nancy Pelosi has also reportedly expressed safety concerns about her staff who work there.
The plaza has been known for years as a place where drug use and sales are common.