The forward, who joined the Gunners in a deal worth £60m last month, opened the scoring in the eighth minute when he slotted in from Gabriel Martinelli's pass.
It was his first goal in his sixth appearance for the club and he could have had a second just before the break when he was slipped through by Mikel Merino, but he was denied by Port Vale goalkeeper Joe Gauci.
The hosts, who were promoted to League One at the end of last season, battled well against their Premier League opponents and caused them issues at times from set-pieces.
While the game remained 1-0, they always had a chance of getting a leveller, and that almost came with 15 minutes to go when substitute Devante Cole curled a shot from distance just over.
But Leandro Trossard drove in late on to make certain of Arsenal's place in the next round - where they will host his former club Brighton.
Arsenal have won a record 14 FA Cups but have won the EFL Cup - formerly the League Cup - only twice, in 1987 and 1993.
They came close last season, reaching the semi-finals where they were humbled 4-0 on aggregate by eventual winners Newcastle.
With good strength in depth following their summer strengthening, they will fancy their chances of at least replicating that run to the last four this term.
They had a different look this match, with Arteta making nine changes from Sunday's 1-1 draw with Manchester City.
That perhaps played a part in their struggles to get a second goal after Eze's early opener, and for all their dominance of the ball, they did not create too many chances before substitute Trossard ensured there would be no late drama.
Eze will be delighted to get off the mark after returning to the club, but Mikel Arteta believes the £60m signing has more to give.
"He played very centrally against a team that's super compact inside. It's not easy to find spaces, but he's very capable with the ball," Arteta said.
"He needs to still to play a little more time with those guys to understand the timings, especially the things that he needs to do and then actions like that will be more consistent and better for us.
"In general, very good - there is still a lot more to give, I think."
It was not a memorable display from Arsenal but they got the job done - with the additional positive of Bukayo Saka managing more than 60 minutes as he works back to full fitness after hamstring problems. (BBC)
NEWS EXPRESS is Nigeria’s leading online newspaper. Published by Africa’s international award-winning journalist, Mr. Isaac Umunna, NEWS EXPRESS is Nigeria’s first truly professional online daily newspaper. It is published from Lagos, Nigeria’s economic and media hub, and has a provision for occasional special print editions. Thanks to our vast network of sources and dedicated team of professional journalists and contributors spread across Nigeria and overseas, NEWS EXPRESS has become synonymous with newsbreaks and exclusive stories from around the world.