Stephen 1771-1797 was 23 years old when he married 16 year old Mary Carroll 1778-1832 in 1794. They had just two children before Stephen died at age 26 in Kentucky.

James William Castleman 1795–1854, dies at Castleman's ferry in Virginia, and Elizabeth Castleman 1798–1839. 

James William's son Stephen David plays a key role in obtaining pardons for family and friends as he lives in Washington DC from before 1860 until his death in 1889. Stephen David's son James T. also involved as he seems to be the person that copied the terms and conditions from the pardon that was then sent to Edward McCormick along with blank pardons for him to fill in and have notarized when they "take the oath" of allegiance to USA. This pardon is hanging on the wall at Clermont Farm in Berryville, VA.