There is real, responsible value in applying AI to human services: summarizing case notes, drafting routine correspondence, flagging missing documents, helping a resident find the right program. There is also real harm available to anyone careless enough to automate a consequential decision they can't explain.
Start with the stakes
Not every use of AI carries the same risk. We sort use cases by their consequence for a person's life and apply proportionate guardrails. Helping a caseworker draft a letter is low-stakes. Influencing whether a family keeps its benefits is not — and should never be automated away from human judgment.
- Keep a human accountable for every decision that affects a person's benefits or safety.
- Prefer assistive uses — drafting, summarizing, surfacing — over autonomous ones.
- Require explainability: if you can't explain it to the affected person, don't ship it.
- Test for bias against the populations the program serves, before and after launch.
Govern the data first
Responsible AI is impossible without responsible data. That means clear provenance, minimization, retention limits, and privacy protections that meet HIPAA and agency policy. A model is only ever as trustworthy as the data and oversight behind it.
The goal isn't to remove humans from human services. It's to give them back the hours that paperwork steals, so they can spend them on people.
Measure what matters
We instrument every deployment against outcomes that matter to the agency and to residents — accuracy, timeliness, equity of impact — and we keep a human-reviewed feedback loop so the system improves without drifting. When a use case can't clear that bar, the responsible answer is not to ship it.
Modernizing a health or human services program?
We help agencies turn ideas like these into systems that ship. Let's talk about your mission.
Partner with us