The former Hospital for the Negro Insane of Maryland was once the state’s most crowded psychiatric institution. Pulling off a plan to redevelop it won’t be easy.