Production AI Systems / Build Log

CityWise and Resume Optimizer: AI systems built around trust, translation, and evidence

Two very different AI products taught me the same lesson: the hard part is rarely generation alone. The real work is building systems that can simplify complexity without breaking trust.

Two systems, one design principle

Lately I have been thinking a lot about what makes an AI system actually useful in production. Not impressive in a demo. Not clever in isolation. Useful.

The pattern I keep coming back to is this: people do not really want generated text. They want reliable translation from complexity into clarity. They want answers they can act on. They want outputs that stay grounded in the source material.

That principle shows up in two projects I have been especially proud of: CityWise, a production civic AI system used by LA City Council members, and my AI Resume Optimizer, a structured local-first pipeline for tailoring resumes against specific job descriptions.

On the surface they look unrelated. One translates public policy and council records for residents. The other translates a career history into role-specific evidence. But the engineering challenge underneath is surprisingly similar: take dense source material, retrieve the right evidence, simplify it for a real user, and make sure the simplification never drifts away from the truth.

CityWise: turning council records into answers people actually understand

CityWise, at citywise.app, is a production system I built as the founding AI engineer at Visic. It is now used by LA City Council members, which means the bar is not “interesting output.” The bar is trust, clarity, and repeatability in a civic context where people need to understand what government records actually say.

The problem it solves is not really a retrieval problem. It is a translation problem.

Council records, ordinances, and public documents are written in dense procedural language. They are full of references, legal phrasing, and institutional structure that make sense inside government workflows but feel distant from how a resident would naturally think or ask a question. A person does not ask, “Can you summarize ordinance subsection X in its procedural context?” They ask, “Why is this happening in my neighborhood?” or “What does this vote actually mean for me?”

So the system goal was never just “find a paragraph.” The goal was to make unstructured government data speak the truth in plain language, without losing accuracy along the way.

The two-agent architecture

The core of CityWise is a two-agent architecture.

• Generator agent: retrieves the most relevant context using pgvector ANN retrieval with colocated embeddings, then drafts a plain-language response.
• Judge agent: validates that response against the retrieved source before it ever reaches a user.

That second step matters a lot. If you want an AI system to simplify language, you also need a mechanism that checks whether the simplification preserved the meaning. Otherwise “clearer” can quietly become “wrong.”

Simplifying language should never mean distorting facts.

Concurrency and observability were part of the product, not extras

Both agents run through FastAPI's async support, which lets the pipeline handle concurrent execution cleanly in production. That helped me keep the system responsive without treating the architecture like a toy.

I also instrumented the workflow with Langfuse for production observability. That part is easy to underestimate until a real system is live. Once users are asking meaningful questions, you need to know exactly why an answer was produced, which retrieval path was used, what evidence came back, and where drift might be starting to appear.

Observability is what turns an LLM feature into an operable system. Without traces, you are stuck arguing from intuition. With traces, you can inspect, debug, and improve systematically.

The stack

CityWise is built end to end on:

• LangChain for orchestration.
• Azure OpenAI for model access.
• pgvector for retrieval with colocated embeddings.
• FastAPI for async service architecture.
• Docker for runtime packaging.
• GitHub Actions for CI/CD.
• Langfuse for production tracing and drift visibility.

From ingestion to deployment, I owned the AI platform solo as the only engineer on that system. That meant the design decisions had to work not just conceptually, but operationally.

Resume Optimizer: structured output across providers, without trusting the model's own grading

The AI Resume Optimizer came from a much more personal frustration. Tailoring a resume for every role is tedious, and most “AI resume tools” solve that by asking a model to rewrite everything and then grade how well it did.

I do not trust that pattern very much.

A model grading its own output is noisy and inconsistent between runs. The same resume and same job description can produce different match scores depending on wording, provider behavior, or randomness in the generation path. That is not a strong foundation if you want the score to mean something concrete.

Provider-agnostic, section-wise, and structured

So I built the pipeline differently. The system is provider-agnostic across Groq, OpenAI, and Anthropic, using FastAPI and LangChain as the orchestration layer. The first step converts the resume into structured JSON. From there, each section can be optimized against a specific job description independently rather than as one monolithic rewrite.

That structure matters because different parts of a resume play different roles. A summary should not be optimized the same way as a project bullet. A project section should not be treated like contact metadata. Once the resume becomes a structured artifact, the optimization can be much more controlled.

Historical resume retrieval

The pipeline also indexes historical resume versions and retrieves the most contextually relevant past experience for a given job description. That helps recover strong evidence that may have disappeared from the latest draft even though it is still highly relevant to the target role.

This turned out to be one of the most valuable parts of the system because resumes are lossy documents over time. They change shape to fit past priorities. A role-specific optimizer should be able to recover the strongest supporting evidence across that history, not just accept the latest file as complete.

The deterministic match score

The part I am proudest of is the scoring design.

Instead of asking the model to self-report how well the resume matches a role, I built a weighted evidence retrieval system that computes a deterministic resume-job match score from actual retrieved evidence. In other words, the score is grounded in what the system was able to find and support, not in the model's mood on a given run.

That means the score carries the same meaning regardless of which provider generated the rewritten output. It is not perfect because no scoring system is, but it is much more stable and interpretable than asking a generative model to grade itself.

What these systems have in common

CityWise and Resume Optimizer live in different domains, but the deeper design pattern is the same.

• Start with dense source material that is not naturally usable by the end user.
• Retrieve the right evidence instead of relying on freeform recall.
• Translate into user-facing language that matches how people actually ask questions.
• Add a trust layer so the system can justify or validate what it says.
• Instrument the workflow so production debugging is possible.

That is the kind of AI engineering work I enjoy most. Not just putting a model behind an interface, but designing a system where retrieval, validation, structure, and observability work together to make the output trustworthy.

Why this matters to me

I care a lot about practical AI systems. Systems that do not stop at “the model said something plausible,” but instead make it possible to answer harder questions:

• What evidence did this answer come from?
• What part of the pipeline generated this output?
• How do we know the simplification preserved the source truth?
• How do we make the result stable across providers and over time?

Whether the user is a city resident trying to understand council action or a candidate trying to present relevant experience clearly, the trust question is the same. The model is only one part of the story. The system around it is what makes the output dependable.

If you are building AI products where retrieval, validation, and structured outputs actually matter, I would love to connect through connectwithsajid.github.io.

• AI
• RAG
• LangChain
• FastAPI
• pgvector
• Observability
• Langfuse
• Structured Output