Chelsea's Moises Caicedo was sent off in the 38th minute after a VAR check for landing a crunching tackle on Mikel Merino. But three minutes after the first half, defender Trevor Chalobah flicked a header into the net from a pinpoint corner by James.
Arsenal levelled the score when Merino rose to meet a Bukayo Saka cross in the 59th minute.
"We played 60 minutes with 10 men, having to cope with all of their players – they are a very good team, they’re the league leaders at the moment," James told Chelsea's website.
"We knew the game was going to be tough, but to be honest it was a game we felt we probably could have won. It was difficult to only walk away with a point considering how well we did."
Pedro Neto remained busy down the right and substitute Liam Delap had a fine 78th-minute chance saved by Arsenal goalkeeper David Raya.
James said he was proud of the way Chelsea soaked up the pressure and countered.
"We dominated the whole game in every area throughout. I think that was a big statement on where we are and what we're fighting for," James said.
Chelsea are third in the Premier League table with 24 points, six behind Arsenal, and will next meet Leeds United in an away game on Wednesday.