In an interview with The Athletic published on Wednesday, Schafer said European Leagues, which represents 53 professional men's and women's football leagues, was deeply concerned about Infantino's leadership.
"We question his suitability to continue as president. Here in Switzerland, we've had a close view of FIFA and it seems that the influence of the president has grown from year to year," he added.
"Major decisions are taken without meaningful consultation with stakeholders who will be affected."
Schafer's comments come amid a bitter conflict among governing bodies, sparked by FIFA's now abandoned proposal to sell off a slice of the World Cup's commercial rights.
Infantino is under pressure from three confederations to resign, including Europe's UEFA, Asia's AFC and CONCACAF, which represents North and Central America and the Caribbean.
The sport's regional ruling bodies have also discussed a potential vote of no confidence against the FIFA chief.
Integrity of sport in question
Concerns have been raised over the inaugural FIFA peace prize that was awarded to US President Donald Trump in December and the overturning of a ban for American forward Folarin Balogun during this year's World Cup following a call from Trump.
"FIFA is an association under Swiss law and in this association you have rules. The president and the board members have to perform their duties with care. This is a legal matter," Schafer said.
"If you take such a decision (as the peace prize), without even informing the council, that, in my view, is a breach of law. And then, during the World Cup we all know about the red card issue.
"It was a real scandal. I can't ever recall a political intervention of this kind relating to a sporting matter. Such acts call the integrity of the game into question. That's why there is no future at FIFA for Infantino. There can't be."
Infantino will be seeking re-election for a fourth term next year and will likely face stiff opposition, but Schafer said it was more important to redefine the role of FIFA president than it was to get someone else into office.
"For us, getting these reforms is the most important task and they can only happen with another president," he added.
FIFA did not immediately respond to an emailed request for comment.