Stuttgart's Deniz Undav and Dortmund's Karim Adeyemi had chances in the opening half before centre-back Waldemar Anton, who left Stuttgart for the Westfalenstadion in the summer,  put the visitors in front in the 50th minute.

Stuttgart's Jeff Chabot put the visitors two goals up on the 61st-minute mark, knocking in a superb volley for his first Bundesliga goal.

Julian Brandt scored late but it was not enough, as Stuttgart beat Dortmund for a fifth straight time.

Kovac, named coach this week after Nuri Sahin was fired in January, has his work cut out for him, with Dortmund mired in 11th spot.

The coach praised his charges, saying "the boys gave it their all -- we shouldn't have left the field as losers," adding some poor decision-making had "deprived us of the fruits of our labour."

Freiburg climbed to sixth with a hard-fought 1-0 win over Heidenheim, with a first-half header from Vincenzo Grifo the difference.

Italy winger Grifo, known for his set-piece prowess, headed in a lovely Ritsu Doan cross to put his side on track for a third European qualification in four seasons.

Union Berlin chalked up a statement victory in their battle to avoid the drop, winning 4-0 at struggling Hoffenheim.

Union, who hosted Real Madrid in the Champions League last season but started Saturday's game in 14th, won thanks to a brace from Benedict Hollerbach and goals from Marin Ljubicic and Andrej Ilic.

The win took Union 10 points clear of Heidenheim, who sit in the relegation playoff spot.

Also on Saturday, Mainz and Augsburg played out a scoreless draw.