Mydea
A sketching companion that turns the things you love, a LEGO build, a bracelet, your glasses, into AI inspiration while you draw.
Haeyoung Kim, Hanke Chen, Sahana Sundararajan, Sue Min Hong, Lia Sparingga Purnamasari, Shivani Kannan, Byungsoo Kim
Co-developed Mydea as a Graduate Research Assistant with Haeyoung Kim.
Carnegie Mellon University, School of Design, Pittsburgh
Smart pen, projector, GPT-4.1 and GPT-4.1-mini, OpenAI embeddings, gpt-image-2, Python
The paper
In a nutshell: inspiration made from your own belongings didn't change what ideas looked like. It changed how people got there, often through a memory.
A follow-up to Magic Easel. Instead of objects in a lab, Mydea draws on 15 objects each person chose from their own home. We compared it with generic everyday objects in a study with 24 people, half designers and half not.
On paper, the ideas were equally novel. But personal objects opened a new, memory-based route to ideas, and non-designers came up with 70% more ideas. Standard output scores missed all of this.
Same prompt, same ideas
Good inspiration sits in a sweet spot: close enough to the problem to be useful, far enough from your current thinking to move you. Prompt-based AI tools make that balance hard to hold.
Typing breaks the flow
Time spent writing a prompt is time not spent making.
Too close to copy from
What comes back sits so near your input that it becomes something to copy, not think with.
Everyone gets the same
People who describe similar things receive similar material, so ideas start to look alike.
What about the things you already own?
A vase brings back the flowers it once held; the arrangement suggests a structure; the structure becomes an idea. Personal objects carry memories, need no describing, and are different for every person. But they are also familiar, and familiar things can narrow thinking. Nobody had tested which effect wins when AI uses them as inspiration.
Our question: how does AI inspiration grounded in your own meaningful objects, rather than generic ones, shape the way you come up with ideas? And does it differ for designers and non-designers?
Draw on paper. Get a nudge from your own things.
Before the session, each person photographed 15 objects that mean something to them. While they sketch on paper with a smart pen, Mydea reads the drawing, picks one of their objects that relates to it, and every 25 seconds projects a reimagined version of it beside the page. They can use it or ignore it.
Recognizably theirs, but shifted to a new idea that relates to what they're drawing.
Every participant could trace their personal inspirations back to the object they came from.
Inspiration that feels like yours, without being a copy
The hard part wasn't generating images. It was deciding what the AI should give back: how far to move from the object, how to keep it recognizable, and when to show up. Each choice came from pilot testing.
How it works
Objects are described and stored once, before the session. Then a loop runs every 25 seconds while the person draws.
things that matter to you, at home
GPT-4.1-mini writes what it is
plus its colors, ready to compare
smart pen streams the sketch
AI lists the objects in the sketch
top 2 closest, then a weighted pick
GPT-4.1 shifts it by theme
gpt-image-2 transforms, then styles
steady rhythm, in the corner of the eye
The two outlined steps are where Mydea's design choices live: how far the idea moves, and how it stays recognizable.
Two-step generation: change the idea, keep the look
For personal objects, the photo itself goes into the image model. Step 1 transforms the object toward the new, twisted idea while keeping its colors and materials. Step 2 restyles it as a light line drawing with watercolor touches on white, so it sits gently next to a pencil sketch.
Top to bottom: the participant's photo, step 1 (recreated after the study for illustration), and the final image they saw.
Matching with a little chance
The system scores how related the drawing is to every object, keeps the two closest, and picks between them with a weighted coin flip. Related, but never fully predictable.
A creative twist
GPT-4.1 rewrites the object's description into something thematically related, run at a high creativity setting so the twist feels fresh each time: a bracelet becomes a watch, hair ties become a comb.
Smoothing the timing
Image generation can take a few seconds or much longer. Mydea works on several images in parallel and buffers them, learning the typical wait, so a new image still lands on a steady 25-second beat.
24 people, two kinds of objects
Twelve designers and twelve non-designers each did two 20-minute sketching sessions: one with their own objects, one with the shared generic set. The task was to design how people might communicate, or travel, in the future. Order and task were balanced across participants.
Same-looking ideas, different paths to them
If we had only looked at the sketches, we would have said personal objects did nothing. Novelty and visual distance were the same in both conditions. The interviews told a different story.
How people got from image to idea
Borrowing an object's function stayed the go-to route either way. Personal objects didn't replace it; they added new routes, especially memory.
Reached ideas through memories more often, but produced no more ideas, and more often recognized an inspiration and set it aside. For them, the cost was effort per idea.
Came up with far more ideas, often staying close to what the object visibly was or the kind of thing it is. They also gave the AI more than twice as much credit as designers did.
"It just reminds me of my daily life," unlike "a trip or like shopping where you see something new and unexpected."
P18, on familiar objects
The cost: recognized, but not used
Personal inspirations went unused more often (21 vs 13 times), usually when someone recognized their object but chose not to build on it. Some got pulled into a memory; two disliked how the AI had reimagined something of theirs.
Didn't feel inspiring, still worked
People rated the images as only somewhat inspiring in both conditions, yet personal objects led to more ideas and more routes. Most were daily chatbot users who rarely sketch, and may have expected the AI to answer them directly.
Designing AI that works with what people care about
What's next
Personal objects were often more colorful and patterned than the generic set, so future studies should match visual richness, for example by showing one person's objects to another. We also want to test how the source of an object interacts with how far the AI moves it.
Thank you
To Haeyoung Kim for leading this work, to my collaborators, and to the 24 people who shared their favorite things with us.
Back to all work