While Wenger railed against Taylor, appearing to push him as he was ordered from the touchline, Arsenal threw men forward in pursuit of an improbable late victor.
Wenger was sent to the stands as well for dissent, and he'll surely get a call from the FA this week after pushing the fourth official as he headed towards the tunnel.
Alexis Sanchez earned the last-gasp penalty for Arsenal in the match that produced two converted penalties in stoppage-time.
Yet Wenger denied there was any physical altercation with Taylor.
Speaking immediately after the game, though, Wenger categorically denied making contact with Taylor.
Premier League 22nd Match Arsenal FC vs Burnley FC Live Streaming is available on Sky Sports Channel.
Asked if he has spoken to the officials, Wenger answered: "Not yet because I have a press conference to make. I apologise for that". "I should have kept my control, even if it was in a hectic time".
Arsene Wenger has apologized for his behaviour after being sent to the stands by referee Jon Moss in Arsenal's thrilling 2-1 win over Burnley.
That was until referee Moss penalised Francis Coquelin's challenge on Ashley Barnes in the box, allowing Gray to draw the Clarets level.
The merits of whether Ben Mee's foot was high will be debated far and wide, but such an argument would have been deemed irrelevant if the linesman would have correctly flagged Laurent Koscielny offside as he was fouled by the Burnley defender.
"We have Welbeck back and Lucas Perez, before that I had Olivier Giroud on the bench when Theo Walcott was playing, I think they can give us something coming on". It was a big game and you have to stay 11 versus 11.
Last month, fans brainstormed ideas on how the banner might look, and the winning design has taken pride of place at the Emirates for the first time as Arsenal host Burnley.
Sean Dyche's men also posed a threat on the counter-attack, with Andre Gray's clever movement at times allowing him to get the better of Mustafi, but Arsenal responded well by increasing the intensity of their play after the interval. "Maybe it was one I don't know. We won there in the last second and we won again in the last second today", he added, referring to Koscielny's last-gasp victor at Turf Moor in October.
Burnley came back strong and equalised in the 93 minute.
"Every week and every game is an unbelievable fight for everybody", added Wenger.
He told BBC Sport: "We finally got the win but of course it was very hard for us".
Arsenal were dominant and Mesut Ozil had a chance in the 37th minute when he found himself well positioned to latch on to a weak headed clearance from George Boyd, which he could only direct wide of the target.
Burnley spent most of the first half showing why they've played so well of late.