Tom Walker is a greedy and selfish man of a miser who worships money more than he does his equally miserly wife. They lived in a forlorn looking house, that stood long and had an air of starvation. This is until he takes a walk in the swamp at an old Indian fortress and starts up a conversation with the Devil incarnate (referred to as "Old Scratch" in the story.)

Old Scratch strikes up a deal with Tom Walker: he offers the riches hidden in the swamp by Captain Kidd in exchange for Tom's employment under Old Scratch. Tom agrees to think about it, and returns home.

Burdened with this secret, he mentions it to his greedy wife. When he is not there, Tom's wife takes all the valuables in the house and goes to make a deal with Old Scratch. When Tom goes in search of his wife, all he can find of her is her heart and liver in her apron tied to a tree. This conveyed two beliefs common among Irving's contemporaries: that the Devil finds you, rather than the other way around, and that women were held in contempt by the Devil.

Tom Walker then agrees to the deal with Old Scratch (his wife was abusive towards him and he considered her death at the hand of Old Scratch a good thing). Tom agrees to become a loan shark.

Tom constantly swindles people out of money, until he suddenly becomes fearful for the after-life. He then starts to become fiercely dedicated to God, always keeping a Bible at hand.

When, one day, a person who had borrowed money from him and is asking for clemency blames Tom for taking his money, Tom says, "The Devil take me if I have made but a farthing!" At this time, there are three loud knocks at the door. Tom is drawn towards the black cloaked figure and realizes, in horror, that he has left his Bible at his desk.

Tom Walker is then taken away by the Devil on the back of a black horse and is never seen again. All his assets vanished and his house burned to the ground.

The ghost of the miser haunts the site of the old fort.