James Madison

Introduction

On August 10, 1769, recently arrived at the College of New Jersey, eighteen-year-old James Madison (1751-1836) sat in Nassau Hall and penned his first surviving letter. Addressed to his former tutor Thomas Martin, who had brought him to Princeton earlier that summer, Madison opened with personal enquiries but soon proceeded to business:

The Grammars, which Mr. Houston procured for you amount at 2/10 each to 17/. Your brothers account with Plumb.6 to 6/7. and Sawneys expences 4/2…Inclosed you have 15/. the overplus of which you may let Sawney have to satisfy those who may have been at any trouble on his account.

Back to top

The man called Sawney again appeared in Madison’s postscript: “P.S. Sawne[y] tells me that your Mother and Brothers are determined to accompany you to Virginia…I wish…them great Happiness.”[1] Entrusted with Madison’s accounting and charged with relaying family news, Sawney had known Madison since the future president’s birth. Sawney, too, had accompanied Madison to Princeton. Now, he was delivering books and settling accounts on his way back to Montpelier—the Virginia plantation where the Madison family owned him as a slave.

Back to top

Many details of Sawney’s life and journey to Princeton remain unknown: whether he traveled on his own (though his letters indicate that he did), the exact nature of his role on this trip to Princeton, or how long he stayed with Madison in New Jersey. Yet from 1769 to 1823, Madison and his associates wrote dozens of letters and notes referencing Sawney—attesting to the enslaved man’s continuing presence in his life and Madison’s own complex and conflicting attitudes towards slavery.

Back to top

This brief profile will trace Madison’s life through two key stages of his involvement with slavery: his lifelong relationship with Sawney and his other slaves as the master of Montpelier and in the White House, and his political views on the institution of slavery itself, expressed forcefully, and with much contradiction, near the end of his life. Madison seemed to have always believed that slavery was morally wrong, America’s “original sin,” yet he failed to act on these viewpoints to influence policy as President of the United States, and never abandoned slave labor as a Virginia landowner.

Back to top

Planter and Revolutionary

James Madison was born into a plantation society entirely predicated on slave labor, both sides of his family having made their fortunes in tobacco farming. . When his grandfather Ambrose Madison died under mysterious circumstances, three of his slaves were convicted for allegedly poisoning him.[2] Ambrose’s son and James Madison’s father, James Madison Sr., was at one time the richest landowner in Orange County, Virginia, owning 5,000 acres tended by slaves. When he married, his new wife Nelly Conway brought about a dozen more slaves to the household—the fourteen-year-old Sawney most likely among them.[3] In 1751, they had James Madison Jr., the first of twelve children, and raised him to be Montpelier’s heir.

Back to top

The choice of Sawney to join Madison, a seemingly hypochondriacal eighteen-year-old, on his 1769 trip to Princeton hints at the high level of trust the family placed in the enslaved man. In 1782, Madison Sr. made Sawney one of Montpelier’s overseers, responsible for a quarter of the plantation. Sawney lived in an overseer’s cabin rather than the slave quarter; he harvested his own tobacco, which he sold to the Madison family (recorded as “Sawney’s…own crop”), and Madison Sr. permitted him to buy “special ‘English’ shoes of the sort worn by white men.”[4] Perhaps Madison Sr. was trying to avoid the fate of his own father.

Back to top

James Madison returned to Montpelier after his graduation from Princeton in 1771 and quickly involved himself in local politics—employing his intellect and Princeton education to support the Revolutionary cause. In 1774 he became a member of the Orange County Commission of Safety and was elected to the Virginia legislature in 1776, forming a lasting friendship with Thomas Jefferson. At twenty-nine years old, Madison became the youngest delegate at the Continental Congress (1780-3), took a leading role in drafting the Constitution at the Constitutional Convention in Philadelphia (1787), and was elected to the first United States Congress, representing Virginia, in 1790. But even amid Madison’s many political engagements during these years, Sawney frequently appeared in his letters to Montpelier.

Back to top

As an overseer and Madison’s own trusted childhood attendant, Sawney held a position of responsibility on the plantation. Madison described Sawney’s authority as equal to Montpelier’s other overseer, Mordecai Collins—a white man. “I hope M. C. & Sawney will make ready for Harvest without waiting for my presence,” he wrote in 1794, “as I can not be sure of being there in time.”[5] Sawney also regularly made purchases for Montpelier.[6] In his November 1790 Instructions for the Montpelier Overseer and Laborers, Madison enumerated extensive directives for each overseer, almost without differentiation. Sawney was “To have a patent plow,” with “L. Collins [Mordecai’s brother] to do the wooden work.”[7] Yes Madison’s attitude towards Sawney did not extend to the other enslaved people at Montpelier, and Mordecai Collins received one final instruction that Sawney did not: “To treat the Negroes with all the humanity & kindness consistent with their necessary subordination and work.”[8]

Back to top

Over the course of these same years, Madison sometimes questioned the institution of slavery itself. At Princeton, his education under Reverend John Witherspoon had instilled an Enlightenment approach to learning, and throughout his life Madison would emphasize the use of reason in tackling philosophical debates—including questions over slavery.[9] One rare moment of action occurred in 1783, took place when Madison brought a twenty-four-year-old slave named Billey to attend to him at the Continental Congress in Philadelphia. Madison’s maternal grandmother had deeded Billey to eight-year-old Madison when Billey was an infant. In 1783, Madison wrote to his father that he had decided to leave Billey in the north:

On a view of all circumstances I have judged it most prudent not to force Billey back to Va. even if could be done; and have accordingly taken measures for his final separation from me. I am persuaded his mind is too thoroughly tainted to be a fit companion for fellow slaves in Virga. The laws here do not admit of his being sold for more than 7 years. I do not expect to get near the worth of him; but cannot think of punishing him by transportation merely for coveting that liberty for which we have paid the price of so much blood, and have proclaimed so often to be the right, & worthy the pursuit, of every human being.[10]

Back to top

Billey was the only slave Madison would ever emancipate, though he did not in fact free Billey directly. Rather, Madison sold him into another contract, an apprenticeship from which Billey would be freed after seven years. After his manumission, Billey took the name William Gardener and remained in contact with Madison until his death in 1795.[11]

Back to top

In Billey’s case, perhaps Madison was beginning to struggle with the inherent hypocrisy of forcing a man into slavery while at the same time proclaiming the right of all men to liberty. His convictions certainly seem to have been clearest at this time. Two years later, he wrote to a friend:

My wish is if possible to provide a decent & independent subsistence…Another of my wishes is to depend as little as possible on the labour of slaves. The difficulty of reconciling these views, has brought into my thoughts several projects from which advantage seemed attainable.[12]

Back to top

In 1787, at the Constitutional Convention, Madison argued that the slave trade was “dishonorable to the National character”; he further contended that slaves should not be taxed because it would be “wrong to admit in the Constitution the idea that there could be property in men.” Slaves, he reasoned, were “not like merchandize” and were “not consumed,” and thus could not be held as property.[13] In 1794, he married Dolley Payne Todd (1768-1849), a Quaker widow whose father had emancipated his slaves in 1783.

Back to top

“Yielding to the Necessity”

Madison’s antislavery thinking seems to have been strongest during the 1780s—at the height of Revolutionary politics. But by the early 1800s, when in a position to truly impact policy, he failed to follow through on these views. In February 1801 Madison Sr. died, leaving Montpelier and more than one hundred slaves to James Madison, as his eldest son. The following week, Thomas Jefferson became President of the United States and appointed Madison as his Secretary of State. Madison managed Montpelier from afar, yet took no concrete steps toward freeing his slaves or changing the plantation system. Upon becoming the fourth President of the United States in 1809, Madison brought slaves to serve him in the White House.

Back to top

One of these slaves was ten-year-old Paul Jennings, whose memoir about White House life—A Colored Man’s Reminiscences of James Madison—was published in 1865. Though Dolley Madison remains famous for rescuing Gilbert Stuart’s painting of George Washington from the White House during the War of 1812, the young Jennings was with her at the time and assisted her in salvaging the portrait. Jennings served as Madison’s valet until his death, returning to Montpelier in 1817 at the end of Madison’s second term as president. Jennings joined the elderly Sawney there, who, at age eighty-one, now cared for Madison’s ailing mother. Sawney would work past the age of ninety, serving Nelly Conway Madison until her death in 1829.

Back to top

During Madison’s retirement, from 1817 until his death in 1836, Montpelier continued as a slave plantation, and the Madisons faced increasingly dire financial straits. Despite pressure to follow George Washington’s example, Madison did not free his slaves in his will; instead, he left them to his wife Dolley, asking that she only sell them with their consent.[14] Dolley did not comply. Despite her Quaker background and father’s example, she, like her husband, never divested herself of her slaves. Instead, she sold them off to pay debts. Jennings eventually bought his freedom after Dolley sold him in 1846—despite an earlier provision promising to free him in her own will.

Back to top

Madison continued to voice a range of concerns about slavery until his death, even while these views ran in stark contradiction to his retired life at Montpelier, still dependent on slave labor. In 1816, he joined the newly formed American Colonization Society, which aimed to help free African Americans migrate to Liberia; he became its president in 1833. In 1820, writing to the Marquis de Lafayette, who was pressing him on questions of slavery, Madison conceded “the dreadful fruitfulness of the original sin of the African trade.”[15] In a letter to Francis Corbin the following day, he critiqued the system of slavery while failing to argue for a significant change:

I do not mean to discuss the question how far slavery and farming are incompatible. Our opinions agree as to the evil, moral, political, and economical, of the former. I still think, notwithstanding, that under all the disadvantages of slave cultivation, much improvement in it is practicable.[16]

Back to top

In 1829, Madison participated in a Virginia constitutional convention convened to decide if non-property-holding citizens should be able to vote—and took the opportunity to argue that African Americans, both enslaved and free, should be part of the count for Virginia’s House of Delegates. He took to the floor to argue that “it is due to…our character as a people, that they should be considered, as much as possible, in the light of human beings, and not as mere property.”[17] And yet, when facing financial strain in the 1830s, Madison did not accede to these principles, instead selling several of his slaves to make financial ends meet—a decision he described as “yield[ing] to the necessity.”[18]

Back to top

Madison’s conflicting views on the institution of slavery manifested in highly contradictory ways throughout his life—from his relationship with Sawney and his emancipation of Billey, to his failure to follow through on his Revolutionary principles or his own moral beliefs. Historians continue to grapple with these paradoxes. Michael Signer recently argued that “Madison applied a different political philosophy to matters concerning the state to those concerning his own house… At the national level … he believed slavery presented an existential threat to a country premised on freedom. At Montpelier, with slaves he felt he treated well, he saw no such conflict.”[19] Similarly, Jack Rakove has claimed that “Madison’s reluctance to challenge slavery outright or in public can thus be explained—if not justified—as a concession to political reality.”[20]

Back to top

Yet by the end of his life, Madison was no longer even convinced that emancipation was superior to slavery. In 1836, he ended a letter to the Editor of the Farmers’ Register with this unexpected conclusion:

It is most obvious, they [slaves] themselves are infinitely worsted by the exchange from slavery to liberty—if, indeed, their condition deserves that name.[21]

Ultimately, Madison’s personal dependence on slavery led him to question his own, once enlightened, definition of liberty itself.

Back to top