In 1749, William Washington, said to be a cousin of General Washington, came to Granville County to build his home. His youngest son John later inherited the home. In 1800, John built a large old mansion, called Elmosa, as a gift to his daughter Rowena, who married Dr. J. L. Wortham. It is unknown how the house received the name, but it is thought to have some family significance. Tucker provides a description of the building.