Emergency patients were waiting too long to be seen and some medicines were out of date at Fosse House in Leicester, its inspectors found.
But, their report said patients were treated "with compassion and dignity".
Central Nottinghamshire Clinical Services, which runs the service, apologised to those affected.
The NHS service in Smith Way, Enderby operates across Leicester, Leicestershire and Rutland.
Under the CQC's new programme of inspections, all of England's GP practices and out-of-hours services are rated according to whether they are safe, effective, caring, responsive and well led.
After the inspection in March, Fosse Way scored "inadequate" in all areas but caring, for which it "requires improvement".
Emergency patients were waiting too long to be seen, the report said, and essential clinical equipment was found to be out-of-date and not available causing "inappropriate delays".
A spokesman for CNCS said it was already making the suggested improvements to waiting times and management of medication and equipment.
Janet Williamson, deputy chief inspector in CQC's central region, said "significant areas of concern" were found despite patients saying they were treated "with dignity and respect".
"We will continue to monitor this service and we will inspect again in six months to check whether improvements have been made," she said.
"I am hopeful that the service will do what is required for the sake of its patients but if we find that the service remains inadequate, we will consider taking further action."
An out-of-hours GP service that serves more than one million people has been put in special measures by the Care Quality Commission.