Lucas Paqueta had given the Hammers the lead, but second-half goals from Amadou Onana and Morgan Rogers flipped the game on its head.

Paqueta almost got the Potter era off to a lightning-quick start with a stunning chance from range, but it swerved just wide of the far post with just 41 seconds on the clock.

Just eight minutes later, the Brazilian did get West Ham off the mark, dropping back to meet Crysencio Summerville's squared cross and guiding it into the bottom-right corner.

Villa did not manage a single shot on target in the first half but steadily grew into the game in the second half, getting some luck along the way.

Onana's skewed shot seemingly flew straight out of play, but the referee awarded a corner. Tyrone Mings flicked it onto Ian Maatsen, whose drilled shot was spilled by Fabianski, and Onana was on hand to fire in the rebound from close range.

Villa got its nose in front just five minutes later, with a blistering counter-attack. Ollie Watkins got the better of Konstantinos Mavropanos to fizz in a low cross, and Rogers was on hand to poke it home.

Jacob Ramsey almost rounded things off in style after Villa caught out its visitor with another quick counter, but his shot from the edge of the area smashed off the woodwork, and the hosts held on through West Ham's late flurry of chances.

Rogers believes anything can happen in a cup competition and questioned why Aston Villa could not win the trophy after the match.

The club's most recent trophy was the League Cup in 1996, while it hasn't won the FA Cup since 1957, finishing as runner-up twice since then.

"A bit of belief, we were a bit more patient [in the second half]. There was more forward-thinking," Rogers said.

"We created chances, a bit more mayhem in the final third, and that's where our goals came from.

"We know we can come back against anyone. We'll look to rectify the start of games, but we're happy with the way we've finished them.

"Why can't we win it? Anything can happen in a cup competition. You don't know who you'll draw, where you'll play."

Potter did not get off to the start he would have been hoping for, despite only being in the job since Thursday. 

Six of the past seven West Ham managers have lost their first game in charge, with Slaven Bilic the exception in a July 2015 UEFA Europa League qualifier.

"The players have had a big week, a difficult week," Potter said.

"We're disappointed to go out and to lose, but the application of the players has been really good. They've carried out everything we've asked them to do in a short space of time.

"We had some more chances at the end but in the end, Aston Villa were too strong. We pushed and pushed, but it was not meant to be.

"Niclas [Fullkrug's first-half injury] looked a bad one so that's going to be more time. Crysencio Summerville's is a bit more precautionary. We're hoping it's less severe."

West Ham has been eliminated from the FA Cup in the third round in consecutive seasons for the first time since doing so in 2011-2012, 2012-2013 and 2013-2014.

It was much better reading for Unai Emery, although his side might count itself lucky to come away with the win having not managed a first shot on target until the 69th minute.

Villa eliminated a Premier League side from the FA Cup for the first time since the 2014-2015 semi-final against Liverpool, which was also the most recent time it came from behind to win a match in the competition.