Role
Product strategy, interaction and AI-behavior design, curriculum and data modeling, with product implementation directed in Codex.
Personal project
Sentence Map
A curriculum-grounded AI learning tool that turns Korean subtitles into interactive grammar maps.
Sentence Map turns a photographed Korean subtitle into a labeled map of its grammar. I designed it to show how a sentence fits together before revealing detail, and to check curriculum coverage against published beginner materials rather than asking AI to estimate difficulty.
Applied AIHuman–AI InteractionProduct DesignInformation ArchitectureLearning Systems
Status
Working personal prototype, refined through continued use; not yet evaluated with other learners.
Working outcome
Turns a photographed Korean subtitle into an interactive grammar map and lets me save selected grammar and vocabulary into a Study UI.
Evidence base
Checks mapped terms against 1,243 source-traceable entries extracted from four volumes of the King Sejong Institute’s beginner Korean textbooks, retaining curriculum level and source page.
The problem
How might I turn subtitles into grammar I could learn from?
After several years of self-studying Korean on and off, I could read the letters and follow how a sentence was put together. What stopped me was vocabulary. A subtitle would go by with two or three words I didn't know, and by the time I worked out what they were, the scene had moved on.
I had one advantage: 20 years of studying Japanese. The hardest part of that, as a native English speaker, was adapting to word order that works nothing like English. Korean orders sentences the same way, so I did not have to learn that again. It meant I could see the structure of a sentence clearly and still not know what half of it meant.
I wanted the app to show me how a sentence fits together first, then let me explore the parts I don't understand yet.
From subtitle to sentence map
Designing a path from TV show to grammar study
The streaming sites I use for Korean shows and movies prevent screenshots because of digital-rights protections. I had built a screenshot-based tool for Japanese TV, Jimaku Catcher, but it only works because the Japanese streaming site I use permits screenshots. For Korean, I needed a different flow: notice a subtitle, pause the show, pick up my phone, and photograph the screen.
The first step worked: the app extracted subtitle text from the photos. But because I had not yet told Codex how to present it, the output was a long, unstyled report that ran off the screen.
I gave Codex three presentation requirements:
- Color-code each part of the sentence by its grammatical role.
- Show both a word-for-word translation and a natural English translation beneath the sentence map.
- Let me tap a 'chip' to learn more about it.
A learning reference
I needed to know what was within beginner range
I wanted the sentence map to show me which terms were within beginner range. I could have asked the model to rate each term's difficulty. But based on what? Instead I grounded it in published beginner materials from the King Sejong Institute: four volumes of their beginner Korean textbooks, freely available as PDFs.
One part at a time
The map showed structure; each 'chip' needed context
Each time I used it, new questions surfaced about the term I had tapped:
- What does this word sound like?
- How does this word work in other sentences?
- Is this term within the beginner curriculum?
- How can I save this to come back to later?
Reading the map at a glance
Adding curriculum level and Study status to the sentence map
Two of those questions also belonged in the sentence map itself, answerable at a glance without tapping anything:
- Is this term within the beginner curriculum I am using?
- Have I already saved it to Study?
Making saved material useful
Turning saved terms into a place to study from
At first, clicking 'Save to Study' saved the term to a database… And there it sat. I needed to turn that database into something I could actually study from.
- I linked it to a published curriculum: The first version covered the 44 grammar patterns taught in Sejong Beginner 1A and 1B.
- I built two ways to see the same material: A visual map showed the whole curriculum and my status across it. A conventional list made individual patterns easier to scan.
- Then I added vocabulary: After the grammar views were in place, I added Saved Words as a separate tab for the individual terms I had collected.
Four states separate what the curriculum contains from what I have actually encountered and chosen to study:
- Locked: This grammar pattern is in the curriculum, but I have not encountered it in a subtitle I captured and saved to Study. I can see that it exists, but its explanation stays closed.
- New: I encountered this pattern in a subtitle and chose to save it to Study. Its explanation is now available; New is its initial Study status.
- Learning: Something I am actively working on.
- Known: Familiar enough to consider known.
Design decision
No reading ahead
I deliberately did not make Grammar Mastery a reference book I could read ahead in. A pattern becomes available only after I have met it in a subtitle and decided it was worth studying. That makes progress slower than working through a textbook, but every unlocked pattern comes attached to a sentence I actually heard, in a scene I was actually watching.
Study originally had two subsections, Landscape and List. Those became Map View and List View, grouped together under Grammar Mastery, which sits alongside Saved Words.
Study UI | Grammar Mastery
Study UI | Grammar Details
Study UI | Saved Words
Grammar Mastery organized the curriculum. Saved Words became a practical queue for reviewing individual terms and changing their status.
Future directions
What I'd build next
Add retrieval practice.
Save to Study builds a queue of grammar and vocabulary I chose deliberately. The next step is retrieval practice: prompts that bring selected material back without deciding what counts as progress. Only items I have marked Learning would enter the rotation, and a review pass would not change an item's status on its own.
Extend curriculum coverage to Sejong 2A and 2B.
The extraction pass covered Sejong 1A through 2B, but Grammar Mastery currently maps only the 44 grammar points in Beginner 1. I want to extend it through 2A and 2B so a new subtitle pattern has a place on the same 1A–2B roadmap—and so the Study UI covers the same range the sentence map already marks.
Provide access to detailed grammar explanations.
Some grammar points need more than a short panel note. I want an intentional path from a saved point to supporting Sejong material, with detailed explanations and examples available on demand rather than loaded into the first view.
What this demonstrates
Four decisions that shaped the product
The pipeline is a photo, an API call, and a database. The design work was deciding what the app is allowed to conclude on its own, what it has to ask me, and how much of that reasoning stays visible in the interface.
01
Grounding an AI system in an inspectable source
Every chip's curriculum status traces back to a specific page in a published textbook. I extracted 1,243 entries from four volumes of the King Sejong Institute's beginner Korean series, keeping the level and page number for each one, so the app can check a term against real curriculum instead of asking a model to guess its difficulty.
02
Separating what the system found from what I know
The app tracks four states. Locked and New are set by whether I have encountered a pattern in a subtitle. Learning and Known are mine alone, so the system never concludes I know something just because it appeared on screen.
03
Giving each visual signal one job
I wanted the sentence map to show two things at a glance: whether a word is in the beginner curriculum, and whether I have already saved it to study. My first attempts used dots and check icons, and with both appearing across the sentence, the words themselves became hard to read. I gave each signal its own channel instead: color for grammatical role, type weight for curriculum coverage, and a thin underline for saved terms.
04
Turning AI analysis into a learnable interface
Extracting the subtitle was only the first step. I specified what the analysis had to return for each part of the sentence: grammatical role, meaning, Japanese equivalent, and notes. Then I designed the map to show structure first and reveal the rest when I chose to explore it.
Related work