The Care Quality Commission review said emergency care was one of the poorest-performing parts of the system.
It cited safety as a major weakness, with 22 of 184 units rated inadequate and another 95 requiring improvement.
The government said extra money was being put into services to help.
But the CQC questioned whether this was enough.
The regulator said rationing of council care, including access to home help for daily tasks such as washing and dressing and care homes, was pushing more old and frail people into hospital.
- An A&E unit can be judged to be inadequate on safety if there is overcrowding or delays in resuscitation bays or getting access to key equipment.
- "Requires improvement" is for less serious lapses such as poor hygiene and cleaning practices and equipment not being properly checked and monitored.
- The CQC says any dangerously unsafe units would have immediate action taken, including the suspension of services.
CQC chief executive David Behan said the council care system had reached "tipping point" and was in the worst state he could remember during his 38-year career in the system.
In an unprecedented step for the regulator, he called on ministers to pump more money into the council care system.
He highlighted figures showing the numbers getting care had fallen by a quarter in the past four years, leaving one million older people with care needs getting no help at all.
"That is having an impact on A&E - attendances are rising, emergency admissions are going up and there are delays getting patients out of hospital."
He refused to say how much extra the care system should get. Just under £8bn a year is spent on services - a fall of 9% in the past five years.
he review comes after the BBC revealed on Tuesday that the CQC team that monitors the care market - most care is provided by outside agencies but paid for by councils - was worried about the future of the sector.
It reported growing numbers of care homes were closing and companies handing back contracts to councils.
When the CQC is particularly concerned about safety or the quality of care it can take enforcement action. This includes fines and closing down services.
But it has also been given new powers to prosecute the worst offenders and has had two successful prosecutions to date.
In June, St Anne's Community Services was fined £190,000 following the death of a 62-year-old man who broke his neck in a fall from a shower chair at a nursing home in West Yorkshire in April 2015.
Then in September, Coverage Care Services was fined £50,000 and a former manager fined £685 following the death of a resident at Coton Hill House care home in Shrewsbury in February 2015.
CQC inspectors found the resident had not been given his medication to prevent blood clots for up to 30 days before he died.
But while the CQC expressed concern about the pressures being seen, it also said there were many examples of good care among the 20,000 inspections it had carried out.
Despite the cuts to council care services, the help that was being provided in the home and in care homes was rated as good or outstanding in 72% of cases.
Meanwhile, 87% of GP practices were good and outstanding, as was 42% of hospital care overall. No mental health service was rated as outstanding and just 34% as good.
Not every service has yet been inspected, although the vast majority of hospitals and mental health care facilities have - and over half of GP practices and care services have been.
Katherine Murphy of the Patients Association said the findings were "sobering".
"While there are nuggets of positive examples of trusts successfully swimming against the tide, fundamentally, the tide has turned and the pressures are becoming so great that the health and social care sector is struggling to meet demand whilst delivering excellent quality care."
A year ago, a project - backed by NHS England as part of the vanguard programme - was launched in the London borough of Sutton to get the NHS and the care system to work more closely together.
It has seen GPs employed to carry out regular visits to care homes, with each resident now receiving six-monthly check-ups.
A pharmacist is on hand to visit homes to carry out medicine reviews. And district nurses have been used to train care home staff in dementia, falls and diabetes.
It certainly seems to be working.
Since the project started a year ago, there has been a 10% drop in visits to A&E.
But a Department of Health spokeswoman said the health and care sector was "coping well" given the pressures.
And she said ministers were already providing extra money to the care sector through the Better Care Fund, which is largely funded by the NHS to encourage joint working between councils and the health service.
Local authorities have also been allowed to increase council tax by 2% a year this Parliament to pay for social care - although councils have claimed this is not enough to plug the shortfall in their care budgets because of the wider squeeze on their funding.