The Competition and Markets Authority (CMA) said there were worries regarding competition over the Link ATM network, which is run by Vocalink.
The CMA said Mastercard and Vocalink were two of the three most credible suppliers of services to Link.
As a result, a merger would reduce the number of possible bidders the next time a service provider is chosen.
The two companies have until 11 January to come up with a plan to address the concerns.
If they fail to deliver, then the case could be referred for an in-depth investigation.
CMA acting chief executive Andrea Coscelli said: "The Link ATM network provides an essential service for millions of customers.
"It's important that Link has a good choice of providers when it comes to supplying the necessary infrastructure so it can take advantage of the opening up of payment systems to competition.
"These concerns warrant a closer investigation in the event that Mastercard cannot address them at this stage."
When the deal was announced in July, Chancellor Philip Hammond said the takeover of Vocalink by  America's Mastercard "shows that Britain remains an attractive destination for international investors".
As well as the Link system, Vocalink also supplies Bacs, the automated clearing house which processes direct debits.
Vocalink was owned by several major banks including Barclays, Lloyds and HSBC.
The UK's competition watchdog has raised concerns over Mastercard's takeover of UK payments firm Vocalink.