Building A Local Clinical AI Note-Taking Tool As A Psychology Grad Student

Yes, I’m a clinical psychology graduate student with two young children, an ongoing dissertation, and I’m still waiting to hear about internship on Match Day. Naturally, I decided that was the perfect time to start building a software product and, somehow, a company, with minimal coding experience.

And yet, here I am.

Bad Timing, Good Timing

The timing helped because AI made the jump feel possible in a way it would not have a few years earlier. It has helped me lean into areas where I’m already strong and compensate in areas where I’m not. Everything in between, like the areas where I'm neither strong nor particularly weak, is another story, and that's probably where AI should stay out of the way.

In early summer 2024, OpenAI released a new model and my brain more or less exploded. The model was better, and the pace of improvement felt different. I realized pretty quickly that this wasn’t something I wanted to watch from the sidelines. I wanted to understand it, which meant using the tools, breaking them, and seeing where they fit into my life.

So I started following releases closely, trying new models as they came out, and getting familiar with what they could and couldn’t do. When vibe coding (AI-assisted coding, even for people who aren’t programmers) started taking off in 2025, I jumped in. At first it was small, disposable projects. One-off tools, like pdf editors or mini psych-oriented games for a class.

The First Real Project

By the summer, I started working on my first substantial project, a PHI detector and report de-identification tool. I learned some hard lessons along the way and eventually pivoted away from that project. I wanted to see whether AI-assisted coding could help me solve a real problem I actually had. I wanted a tool that fit into the way I already worked and put everything in one place.

My documentation workflow had been fairly stable for a while with brief shorthand notes, individual assessment recaps, and then turning those into polished notes and reports. I had already refined that process using different custom GPTs, small projects, and other one-off setups, but all of that lived across separate tools and workarounds. What I didn’t have was a single system that could handle the full workflow while also being HIPAA compliant and allowing PHI, especially for report and intake writing where that information plays a more central role.

Why Local Made Sense

I had seen cloud-based solutions, but for me, the problem was practical. I was already using AI in ways that avoided PHI by design, which is inherently a bit risky and, I suspect, somewhat common among clinicians. I also didn’t have much room for yet another subscription, especially when most of the tools I was using were discounted or free because of my student status. A local note and report-writing assistant started to feel like the obvious thing to try. It addressed privacy concerns, and it also solved my immediate need without adding another service to manage.

So I began working on vibe coding what would become LocalScribe, still unsure whether I could actually pull together something that functioned as a real product. I started with the UI, mostly because that was the most concrete place to begin and gave me something visible to react to. That part was engaging, but it didn’t answer the bigger question of whether the rest would hold.

Testing local AI models changed my expectations about what a private clinical writing tool could actually do. I had assumed, based on small language model benchmarking and general discourse at the time, that local models would be noticeably limited. Instead, I found that even relatively small models running locally were exceptionally good at the specific task I cared about, which was taking existing material and organizing it into coherent clinical notes and reports. That included longer narratives, structured sections, and correctly copying long lists of testing scores.

I built it piece by piece as the core note-writing workflow became more stable. Once that started to work, I became more confident adding features that would make the tool genuinely useful, things like dictation, attachments, and even a report deidentification option. The tools were also changing quickly enough that problems from one month often became easier the next month. Models were improving, access to compute was expanding, and workflows that had felt clunky early on became easier to manage. That was enough to keep going, even before I knew exactly what the product would become.

When It Started Working

Eventually, the project reached the point where I could use it in my own workflow instead of only testing it as a prototype. The core workflow worked reliably, and the surrounding pieces were stable enough that I could focus on refining them instead of constantly fixing them. At that point, I started having conversations with trusted friends and advisors, along with a lot of back-and-forth with AI chat tools, to think through what it would look like to offer something like this responsibly. Those conversations tended to center on similar themes of which safeguards belonged in the product, which risks were real versus theoretical, and whether this was solving a problem that other clinicians would recognize as their own.

What Changed For Me

The part I keep coming back to is how different building feels now. A clinician with a specific workflow problem can make a rough tool, test it, revise it, and learn from it before anyone gives them permission. That does not mean every clinician should ship software, and it does not remove the need for technical help, clinical judgment, privacy review, or restraint. The starting point changes when the first draft is something a clinician can actually make.

For me, LocalScribe began as a way to solve a problem in my own documentation workflow. The surprising part was that a rough first version eventually became something I could actually use and keep building.