Arlington, Bobby Lee, and the 'Peculiar Institution'

by Andy Hall

Library of Congress

It's taken as a matter of fact among many Southern apologists that Lee was adamantly opposed to the institution of slavery, or even that he had a strong abolitionist bent. (The phrase "Robert E. Lee was an abolitionist" pings several hundred hits on teh Google.) This position rests almost entirely on a single line plucked from a letter Lee wrote to his wife in December 1856,  reproduced in Douglas Southall Freeman's landmark, four-volume biography of Lee, published in the 1930s: "In this enlightened age, there are few I believe, but what will acknowledge, that slavery as an institution, is a moral & political evil in any Country." But it's a highly-selective, Breitbart-style quote that cuts against the context of the larger letter:

That full passage, long and florid as it is, encapsulates Lee's view of slavery, "a greater evil to the white man than to the black race." He dislikes the institution, finds it harmful and would prefer that it did not exist, but also sees it as a circumstance entirely beyond his or any other mortal's control. "We must leave the progress as well as the result in [God's] hands who sees the end; who Chooses to work by slow influences; & with whom two thousand years are but as a Single day." By contrast, the abolitionist who "means well to the slave, he must not Create angry feelings in the Master." And until the predetermined and unknowable day of day of eventual and inevitable emancipation, "the painful discipline they are undergoing, is necessary for their instruction as a race, & I hope will prepare & lead them to better things."

Shorter Lee: slavery sucks, sure, but it's God's will. It's good for you, too. You're welcome.

Freeman was an unabashed admirer of Lee—driving to his daytime job as a newspaper editor in Richmond, he routinely snapped a salute as he passed Lee's statue—and his 1935 biography is famous today as both a milestone of exhaustive research and for its hagiography of its subject. Freeman is virtually a Confederate hero in his own right, whose work added a solid, successful, Pulitzer Prize-winning endorsement of many of the central themes of the Lost Cause. In addressing Lee's view of slavery, Freeman argues that Lee had little familiarity with the institution as it was practiced outside of Virginia, on cotton and cane plantations in the Deep South. Apart from a posting to the Texas frontier in 1856-57, "all his reflective years had been passed in the North or in the border states. . . . Lee, in short, was only acquainted with slavery at its best and he judged it accordingly."

This is weak sauce indeed.

Fortunately, Elizabeth Brown Pryor helps round out the question of Lee and slaveholding in her tremendous biography, Reading the Man: A Portrait of Robert E. Lee Through His Private Letters. (A C-SPAN presentation and Q-and-A with Pryor was highlighted by TNC back in April.) Pryor gives a far more detailed picture of Lee, his philosophical beliefs on slavery, and how he put those beliefs into practice; what emerges is a far more rounded, complex picture, but one that is far, far darker, as well.

Lee first came to slave ownership in 1829 when, newly out of West Point, he inherited several slaves from his mother's estate. Lee quickly discovered, Pryor writes, that for him slaveholding represented "an uncomfortable stewardship." He  found supervision of the their work to be distracting from his own career, and disliked the daily details of managing and providing for them. He found slaves to be, in Pryor's words, "more trouble than they were worth." To relieve himself of the day-to-day responsibility for them, and to provide additional cash for his household, Lee soon took to hiring out his bondsmen and -women. This practice, common among slaveholders in Lee's circle, makes it difficult to track his ownership of slaves in detail over the next three decades. Freeman believed that Lee had divested himself of slaves by 1847, based on Freeman's failure to find any relevant tax records, and Lee's own son, Robert Jr., claimed that his father had manumitted all his slaves "a long time before the war." Pryor counters that Lee definitely owned slaves as late as 1852, considered buying more shortly before the war began, and throughout the war itself used slaves as personal servants. Whether Lee directly and personally owned slaves at a given point before or during the war, Pryor would argue, is almost immaterial, for presence of slaves and the benefit of their labor was an intimate and familiar part of Lee's daily life until the end of the Civil War.

Indeed, a much more instructive look at Lee's attitudes toward slavery and African Americans can be found in his stewardship of Arlington House between later 1857 and early 1861.

Lee was himself the son of a Revolutionary War hero, but he managed to make an even better marriage, in 1831, to Mary Anna Custis (1808-73). Mary was the great-granddaughter of Martha Custis Washington by her first husband, and so the step-great-granddaughter of the first president. Mary was the only one of four siblings to live to adulthood, and when her father, George Washington Parke Custis, died in October 1857, Mary Lee inherited the bulk of the Custis estate. Robert E. Lee was named executor, and as his father-in-law had foolishly drawn up his will without benefit of legal advice, Lee was forced to take an extended leave of absence from the army to return to Arlington to sort it all out. (It was a result of this situation that Lee happened to be the army's senior line officer available on short notice when John Brown and his band seized the national armory at Harpers Ferry in the fall of 1859.)

The Arlington that Lee inherited was unique in many ways. It was the centerpiece of a small network of farms, where cultivation of a variety of crops was done on a small scale. The estate included sixty-five or seventy slaves, and a number of freed bondsmen lived on the property as well. George W. P. Custis (left) and his wife, Mary, were far more lenient with their slaves than most large property owners, and took a high degree of paternalistic interest in them. Although it was not recognized by Virginia law, they encouraged slaves to marry and live in family units. Slaves who married partners from other estates were allowed regular visitation, and the Custises rarely broke up family groups on the auction block—in fact, they rarely sold slaves at all. Mrs. Custis organized a school for slave children that taught the rudiments of reading and writing, explicitly defying Virginia law even as her husband—fully cognizant of Mary's activities in this area—was serving as justice of the peace. There were small gifts for the children at Christmas and treats on other holidays. The Custises were keen on preserving their bondsmens' souls, as well, organizing and attending services on the estate. (There is some evidence that the required church services and schooling were not as appreciated as they might have been, given that they occurred on Sunday, the slaves' only day without field work.) George W. P. and Mary Custis viewed themselves as compassionate and caring slaveholders, acting in the best interest of their charges; their neighbors viewed them as lax, indulgent masters who'd ruined their slaves for getting a solid day's work out of them.

Running beneath this bright Potomac idyll was the dark and ugly undertow of miscegenation. Whether occurring by violent assault, threat or coercive persuasion, these encounters all amount to abuse, including forcable rape, by white men with virtually unlimited power over women of color. "The Custis family has a reputation for interracial dalliance," Pryor writes, "and many of the mulatto servants had clearly descended from illicit ties." People of mixed race represented roughly ten percent of the South's population in the decade before the war; they accounted for over half the slaves and all the free persons of color living at Arlington listed in estate records and the census of 1860. Rumor—more than mere rumor—had swirled around Custis men for decades, and there is ample circumstantial evidence that the practice reached down to George W. P. Custis himself. Over his lifetime he freed a handful of female slaves and their mulatto children; even the Congressional Record acknowledged this and suggested that Custis was showing something like a "paternal instinct" in the process (emphasis original). Pryor notes that "there is no evidence that Lee himself indulged in sexual activity with the slaves, but certainly he was aware of it." The latter could undoubtedly be said for all of Alexandria County and the District of Columbia, across the river.

It was to this complex world, then, that Colonel Lee returned from Texas in late 1857. The Arlington estate was in disarray; there was a mountain of debts, and the farms were losing money due to mismanagement. Lee set to work on making the estate profitable again. Ironically, for a man now widely hailed for his compassion and gentle demeanor, he had no compunction at all about overturning the Custis family's indulgent ways. It was a shock to the black men and women of Arlington, many of whom had lived nowhere else. As Pryor says in her C-SPAN interview (beginning about the 56:30 mark), Lee fundamentally believed the master-slave relationship was "the only relationship that could exist between the races; he had no grander vision, no ability to see beyond that. Master and slave was the only relationship and, unlike Mrs. Custis, he saw it very much as an economic relationship, that those slaves were there to work, and I think one of the reasons they thought he was mean is because he was very tough on them. He saw that he owned their labor. And I think it was a contrast to the situation they'd had a few years earlier."

Lee's admirers have often attributed to him in this area the cliched he-was-strict-but-fair trope. Perhaps, but one incident in particular at Arlington reveals a darker, vindictive side to Lee. Even under the relatively light hand of the Custis family, runaways had been a serious problem. The Arlington bondsmen had believed that George W. P. Custis had ordered them manumitted in his will; he had, but left the timing to the discretion of his executor. Lee told them they could be freed only after five years. Escapes increased and Lee, mindful of the value of the escapees both as property and as an example to others, determined to curtail it. In June 1859, two letters were published in the New York Tribune describing a case where three Arlington slaves, two men and a woman, escaped north into Maryland and had got nearly to the Pennsylvania line and freedom before being captured by the constable and hauled back to Virginia. When they arrived at Arlington, Lee angrily demanded they be whipped. The estate's overseer refused, and the constable took over. But after whipping the men, he declined to beat the woman. Lee, according to the letters, did that job himself.

Lee's biographer Freeman dismisses these accounts as "exaggerated" and "libel," an "extravagance of irresponsible antislavery agitators." There is, Freeman asserts, "no evidence, direct or indirect, that Lee ever had them or any other Negroes flogged. The usage at Arlington and elsewhere in Virginia among people of Lee's station forbade such a thing."

Wesley Norris disagrees. Wesley Norris was one of the men flogged at Arlington that day.

Norris makes no claim that Lee whipped his sister personally, but otherwise his account follows closely those letters submitted anonymously to the Tribune. In all, Pryor found seven accounts of the event, all of which corroborate the basic elements of the original claim. It's important to remember as well that Norris' story was published in an antislavery newspaper in 1866, the year after the war ended; Lee was in good health and serving as president of Washington College in Lexington at the time. He did not respond publicly, though privately he denied it. But Pryor makes the critical point abut Norris' story: "its veracity has been questioned by generations of Lee aficionados, and we might be tempted to dismiss it as the exaggerated ranting of a bitter ex-slave. Except for one thing: all of its facts are verifiable." Among the verifiable fact Pryor found was the receipt book showing payment to the constable.

In the end, though, it doesn't matter much whether Lee personally whipped Mary Norris, or even if he merely stood by, extolling the constable to "lay it on well." It doesn't matter because every day of his life, from birth to the end of the Civil War, he lived in a world where he failed to grasp the most fundamental notions that should transcend culture, custom, and even law. He failed to see the basic human nature of the people of color around him. As Pryor herself summarizes:

Image: Arlington House under Union occupation, June 28, 1864. Both images from the Library of Congress.