The doctors help fill out forms for bus passes, bring people in from the streets and screen for abuse. They treat the patients who cannot, for whatever reason, fit in to mainstream care.
Aaron has always had difficulty keeping appointments.
"I find it hard to understand people and I'm not good with time schedules," he says.
He has Asperger's Syndrome, and struggles with the concept of time.
He was taken off his GP's patient list after he repeatedly missed consultations, leaving him with no access to primary care.
Aaron is now registered at the Whitehouse Centre in Huddersfield. The GP practice was set up in 2001 to serve the local homeless community. Its patient list has expanded to include asylum seekers, refugees and patients deemed as too chaotic for mainstream care.
"I missed a few appointments here too so they let me use the drop-in service instead," Aaron says. "I knew I could come in between 11.30 to 12.30pm and be seen."
With support Aaron now makes and keeps appointments at the centre.
"I feel comfortable here, because if I don't understand things Dr Miller will explain again with easier words. It was hard trying to explain what I needed to doctors before."
Joanne Miller has been a GP at the Whitehouse for 13 years. She says they often have to re-register vulnerable patients who have been moved off other GP lists.
"General practice is under huge pressure from chronic underfunding," she says. "Many GPs do not have the time or resources to go the 'extra mile' for a vulnerable patient."
The Whitehouse Centre specialises in treating complex patients, many of whom are affected by high levels of trauma. As the surgery has few elderly patients with chronic conditions it frees up time to focus on mental health and substance abuse.
Sarah from Pakistan, not her real name, receives mental health support from the Whitehouse Centre. She arrived in the UK with her son in 2012 and is still waiting for a final verdict on her asylum claim. The uncertainty and fear of going back has proved a huge strain.
"It's been a long time of waiting and we don't know what will happen, so that makes us depressed. We cannot work and that is difficult because it doesn't feel natural.
"I cannot trust others but I can talk to the doctors here. They are very kind and it's good to know we have nice people supporting us."
Asylum seekers only have access to primary care and not to mental health services. GPs at the Whitehouse can prescribe anti-depressants and provide a little counselling.
"We can't do as much as a specialist could do. It's about maintaining the patient at a certain level," says Joshua Lumb, the practice manager.
"Mental health services won't see asylum seekers until they have leave to remain. One reason is that it would be hard to get to the heart of the patient's problem while there is this uncertainty about their status hanging over them."
Patients who are new to the country often struggle with basic English, and the centre uses a telephone interpreting service available on the NHS. The system "is not great but is what's available", according to Miller.
Questions are asked slowly and sometimes repeatedly via a translator on loudspeaker. It adds another layer of difficulty to already complex cases. While appointments are set at 15 minutes, they are operated flexibly.
The Whitehouse Centre relies heavily on the voluntary sector for additional support. They have links with the charity Dash and Volunteers Together, which support asylum seekers and the Methodist Mission and Clare House, which help the local homeless.
Alan is a resident at Clare House and a patient at the Whitehouse. He has struggled with drug and alcohol addiction most of his life and spent several months in two rehab programmes. He has had a troubled relationship with his family.
"I've been drinking since I was about 10," he says. "My mum was an alcoholic and committed suicide and I've no father."
Alan has had difficulties getting healthcare in the past because of his unpredictable behaviour.
"The doctors would only see me if there were two of them there. It's been my fault - I'm very poor with trust because I've been let down in the past.
"But Joanne (Dr Miller) seems to know how to deal with me. I can talk to her and she listens to me. If I'm swearing a lot she'll tell me to pack it in. She's helped me with my prescriptions and with my bus pass."
"Increasingly the workload in primary care is about trying to sort out the financial and social problems that make people ill," Miller says.
"We don't try to make one size fit all and we do put in extra effort, such as providing clothing for homeless patients or documenting any injuries asylum seekers have."
These efforts were recently recognised by the Care Quality Commission, which classed the surgery as outstanding - the only one to achieve this result in Huddersfield this year.
The centre has three part-time GPs, one of whom is a volunteer and specialist in torture screening, plus two full-time nurses. The small team meet each morning to discuss their often complex cases, including any concerns such as patients who have gone off the radar.
They are only large enough to deal with about 1,400 regular patients and so can only take them on until they are considered stable enough to return to mainstream care. The aim is to deregister them after a year, but each case is decided separately.
"We've registered 1,000 new people in the past 12 months, because Kirklees is one of the few councils in the area still taking on asylum seekers," Lumb says.
"There is a lot of demand but we're managing it well."
Dr Miller wishes that the could expand their current practice.
"In an ideal world our service would be much bigger but there is no government funding any more for expansion of such schemes.
"But we are fortunate in Huddersfield to have a dedicated practice for some of the area's most vulnerable people."