Auejin Ham
한국어I completed my compulsory military service as a technical research personnel from February 2023 to 15 March 2026, and I currently work at Cupix. I received Master's degree at School of Computing, KAIST (advisor: Prof. Geehyuk Lee).
GitHub · Google Scholar · Twitter · Email · Full CV (PDF)
Read the full biography
I design interaction paradigms that translate complex human intent into precise machine execution — spanning low-level physical sensors, haptic interfaces and multi-agent AI systems.
My work began with physical constraints where ground-truth answers exist. At 17, I built an eye-tracking phone interface for quadriplegic users that earned a spot on Korea's Intel ISEF delegation. As a researcher at DGIST, KAIST and NVIDIA Research, I quantified the millisecond-level impacts of latency and jitter on human performance, publishing first-author work at top-tier HCI venues like ACM UIST.
I extended this focus to subjective, ungrounded interactions as Technical Research Personnel at bHaptics. There, I engineered real-time tactile software pipelines across iOS, Android and Windows with sub-20 ms latency, turning open-ended human perception into reliable, deployed software products.
Currently, as a Technical PM and Software Engineer at Cupix, I design systems that govern the stochastic nature of LLMs through deterministic engineering. I built Issue Fixer, an autonomous agent system that operated unattended for five months — processing 924 production bugs with a 92.2% root-cause accuracy. I also engineered Isopod, a multi-agent harness that converts natural-language specs into state machines, and led the technical delivery of Beacon Chat, a conversational AI product for construction digital twins.
Across physical sensing, user studies and agentic orchestration, my goal is to lead multi-disciplinary teams in defining the next interface standard for complex AI environments — turning ungrounded, probabilistic interactions into rigorous, real-world systems.
Experience
May 2025 – Present
A company that turns buildings and construction sites into 3D digital twins people can look back through. I work on the frontend and as technical PM.
- What
- A company that turns buildings and construction sites into 3D digital twins people can look back through. I work on the frontend and as technical PM.
- Why
- Sites lose network, yet the desktop app fetched everything from the server, and QA filed bugs faster than the team could absorb them.
- How
- I built an offline mode that stores every entity's metadata and every image locally, and a hybrid layer that switches between the real API and the local database according to connectivity without changing a single call site. Edits made offline synchronise automatically on reconnection.
- New
- I proposed and ran the company's first workflow in which AI agents carry out development work, and served as technical PM for Beacon, its conversational AI product.
Feb 2023 – Apr 2025
A company making wearables that deliver touch through vibration. I worked on the apps and the SDK as technical research personnel.
- What
- A company making wearables that deliver touch through vibration. I worked on the apps and the SDK as technical research personnel.
- Why
- The hardware existed, but something had to carry what happens inside a PC or PSVR game onto the body.
- How
- I built the control apps on iOS (SwiftUI), Android (Jetpack Compose) and Windows (WPF), talked to the hardware over BLE and TCP including firmware update (DFU), and turned live PC-game events into haptic signals.
- New
- I designed the GitHub Actions CI/CD that automated release across all three platforms.
2021 – 2023
Master's course at the KAIST HCI Lab.
- What
- Master's course at the KAIST HCI Lab.
- Why
- My subjects were gaze-based menu selection and touch interaction on the mouse wheel.
- How
- I drove haptic actuators directly, implemented the wireless messaging pipeline, and built the tools used to author haptic interaction.
- New
- The work appeared at ACM CHI 2022 and in the International Journal of Human-Computer Interaction.
Jun 2022 – Nov 2022
Measured how professional gamers aim, at NVIDIA Research.
- What
- Measured how professional gamers aim, at NVIDIA Research.
- Why
- The question was how aiming performance changes as display and input-device conditions change.
- How
- I measured and compared across conditions, and proposed a short-time, wavelet-inspired method for detecting mouse submovements.
- New
- arXiv 2026, first author.
Jun 2020 – Feb 2021
Undergraduate researcher at the DGIST Smart Input Device Lab.
- What
- Undergraduate researcher at the DGIST Smart Input Device Lab.
- Why
- When a mouse's polling rate falls out of step with the display refresh the cursor jitters, and how much that costs a person's aim was not known.
- How
- I built a rig that logs 8000 Hz input without loss and measured the cost experimentally, and analysed the Oculus Touch haptic feedback API.
- New
- ACM UIST 2021, first author.
Jun 2019 – Jul 2020
Undergraduate intern at the KAIST Interactive Media Lab.
- What
- Undergraduate intern at the KAIST Interactive Media Lab.
- Why
- Behavioural-measurement devices drift apart in time when run together, so they stop pointing at the same instant.
- How
- I implemented a UDP protocol that records eight sensor types — eye tracker, motion tracker, webcam, Arduino — against one timestamp, and built a Pupil Labs serialisation library and a CS:GO scene-analysis system.
- New
- Data gathered on that rig went into Secrets of Gosu (ACM CHI 2021).
Aug 2018 – Dec 2018
Software developer at a startup in the KAIST startup incubator.
- What
- Software developer at a startup in the KAIST startup incubator.
- Why
- Computers for blind users cost several thousand dollars, and a phone's voice guidance reads everything aloud to whoever is standing nearby.
- How
- I built a braille input and output device shaped like a phone case, and the iOS application that drives it over Bluetooth.
- New
- Demonstrated to investors at KAIST TECH DAY.
Jun 2018 – Aug 2018
A month at Can Tho University in Vietnam with the World Friends ICT volunteer programme.
2017
Vice manager and head of outreach for a working group under the DGIST Supercomputing Center.
Publications
2026
2023
2022
2022
2022
2021
2021
Projects
2026
An LLM chat server and panel UI over a SaaS that captures construction sites in 3D.
- What
- An LLM chat server and panel UI over a SaaS that captures construction sites in 3D.
- Why
- When someone asks how many safety findings sit on level three, one number stops work and settles invoices.
- How
- Code grades every search result before the model speaks — complete set or sample — and the grade constrains which sentence the model is permitted to write. Where the grade is low the system asks back and narrows the scope.
- New
- The model earns its confidence through conversation rather than imitating it: the path from a sample to a complete set runs through the dialogue itself.
2026
A harness that makes multiple agents carry a natural-language development spec through to the end.
- What
- A harness that makes multiple agents carry a natural-language development spec through to the end.
- Why
- A spec written by a person leaves too much room for interpretation for a machine to judge how far it has got.
- How
- It pins the spec into a checkable state machine, splits it into work cards on a dependency graph, and runs agents with no human intervention until the graph is drained, producing verified commits.
- New
- Progress is measured by what is left in the graph rather than by what an agent reports.
2026
An agent system that owns a production bug queue end to end.
unattended in production 5 months · issues ingested 924 · PRs generated automatically 571 · per issue ~30 min · merge rate 50.0% · root-cause accuracy 92.2% (as of 2026-08)
- What
- An agent system that owns a production bug queue end to end.
- Why
- Most of the time a bug costs goes not into the fix but into working out what broke under which conditions.
- How
- It polls Jira and picks its own issues, compresses the attached recording into keyframes for a VLM to recover the capture conditions, then reproduces in a real browser, fixes, refutes its own fix with a before/after check, and opens the pull request.
- New
- The point is that an LLM saying it is done never counts as the completion signal — verdicts are demoted to typed enums — and a biweekly self-audit files improvement pull requests against its own code.
2024
An unattended kiosk that composites a person in a full-body costume between illustrated character layers and prints the result on the spot. Code
- What
- An unattended kiosk that composites a person in a full-body costume between illustrated character layers and prints the result on the spot.
- Why
- Convention booths have unreliable networks, and a costume's fur and trim make an outline that chroma keying cannot cut cleanly.
- How
- From a single webcam, SAM2 separates the person and Depth-Anything-V2 estimates depth, placing them between the illustration layers. Everything runs on the device.
- New
- No green screen and no server: the booth is self-contained. Run at seven conventions in South Korea between 2024 and 2026.
2024
A desktop bot app for streamers on Chzzk, Naver's live-streaming platform. Web
downloads 56,000+ · broadcasts served 55,600+ (as of 2026-08)
- What
- A desktop bot app for streamers on Chzzk, Naver's live-streaming platform.
- Why
- Reading chat, answering donations and running screen effects while broadcasting is more than one person can hold at once.
- How
- It takes chat and donation events over a socket, runs eleven kinds of bot, and draws the results as a live overlay on the OBS scene — a Tauri v2, Angular and Rust (actix-web) desktop app with a NestJS cloud API.
- New
- Three operating-system builds are published through GitHub Actions, run single-handed without a commercial release service.
2024
An iOS application (SwiftUI) that drives bHaptics vests and gloves. Paper
downloads 8,700+ · active users 400+ (as of 2025-03)
- What
- An iOS application (SwiftUI) that drives bHaptics vests and gloves.
- Why
- To feel what happens in a game on your body, something has to sit between the PC or PSVR title and the hardware in real time.
- How
- It finds supported titles on the same Wi-Fi network by itself, takes their events, and relays them over BLE in under 20 ms, alongside firmware update (DFU) and account flows.
- New
- It ships with a Swift SDK folding device state and account state into the SwiftUI lifecycle, so the app and the SDK share one state machine.
2021
A rig that mimics the physical behaviour of mechanical keys in order to find the right activation method for a touch keyboard. PaperVideo
- What
- A rig that mimics the physical behaviour of mechanical keys in order to find the right activation method for a touch keyboard.
- Why
- A touch keyboard gives no sensation at the moment of contact, so the hand cannot tell when the key registered.
- How
- Press depth, restoring force and other physical parameters are adjustable; capacitive sensing and an RF power detector measure contact, and multiplexing reads the key array. The control side is WPF and Arduino.
- New
- Activation is a variable to sweep rather than a fixed design constant, so the best point can be found by experiment.
More
2022
A Python library that animates values arriving over a serial port in real time.
2021
A real-time web platform for forming teams in KAIST classes.
- What
- A real-time web platform for forming teams in KAIST classes.
- Why
- Remote teaching through the pandemic removed the places where students get to know each other, leaving nothing to judge a prospective teammate by.
- How
- A quiz surfaces candidates with matching interests, an anonymised chat with a moderating bot lets them talk, and a credit record shows how much each person contributed to earlier group projects. Built on React and the Firebase realtime database.
- New
- Anonymity lowers the social cost of choosing while the credit record still exposes a history of free-riding, so a choice can be both comfortable and informed.
2021
A system that lets two people in augmented-reality glasses see where the other is looking in the same room.
- What
- A system that lets two people in augmented-reality glasses see where the other is looking in the same room.
- Why
- Speech and pointing are thin instruments for singling out something far away, and the moment you look at a handheld device you look away from the site.
- How
- Using HoloLens 2 it measures each person's eye and head direction, finds the moment their regions of interest coincide, and outlines that target in both headsets at once. Built in Unity (C#), sharing a coordinate frame through Azure Spatial Anchors and replicating state over Photon Unity Networking.
- New
- Rather than showing gaze continuously it shows it only where two people's attention meets, so the cue itself signals agreement.
2020
A toolkit that records several behavioural-measurement devices against a single timestamp.
- What
- A toolkit that records several behavioural-measurement devices against a single timestamp.
- Why
- Each device runs its own clock, so signals logged separately do not line up on the same instant afterwards.
- How
- It drives the devices together over UDP and serial, and cuts the number of system calls crossing the OS pipe to hold clock drift to 11 microseconds per second. Built with Arduino and Processing (Java).
- New
- Eight kinds of sensor at once — eye tracker, motion tracker, webcam, EMG. The data it collected went into Secrets of Gosu (ACM CHI 2021).
2020
An interaction technique (TwG) that carries emotion and turn-taking to a chatbot through touch.
- What
- An interaction technique (TwG) that carries emotion and turn-taking to a chatbot through touch.
- Why
- Chatbots have been treated in terms of natural language processing for text conversation, leaving little means to convey a user's emotion or manage whose turn it is.
- How
- Users send emotion and hand over the turn by touching a graphic interface; a Wizard of Oz study evaluated the resulting experience.
- New
- Emotional expression improved over conventional text-only chatbot interaction, and turn-taking became more natural.
2020
A library that brings Pupil Labs eye-tracker data into Processing (Java).
2019
A VR interface for operating a crane.
- What
- A VR interface for operating a crane.
- Why
- Crane control carries three problems at once — dizziness, the safety of people nearby, and distance perception. HIAB Korea posed the subject.
- How
- The viewpoint and the control scheme were designed against all three together.
- New
- Winner of the logistics and physical-computing track at Junction X Seoul.
2019
A system that throws a voice assistant's graphical interface onto a wall or desk with a projector.
- What
- A system that throws a voice assistant's graphical interface onto a wall or desk with a projector.
- Why
- A voice-only assistant has no way to hand over a list or a table, and a fixed screen only works if you walk to it.
- How
- Kinect v2 finds the person and the surface, Arduino PWM steers the projector, and the display lands where it can be seen. Built in WPF (C#) and evaluated with a Wizard of Oz study and NASA-TLX.
- New
- The display follows the person, so using the assistant is no longer tied to wherever the speaker sits.
2019
A technique that takes mid-air pinch gestures as input before the hand reaches a vertical mouse.
- What
- A technique that takes mid-air pinch gestures as input before the hand reaches a vertical mouse.
- Why
- A vertical mouse is held with the wrist upright, so nothing can be entered until the hand is already on it.
- How
- It recognises the pinch trajectory in eight directions: left, right, upper left, lower left, upper right, lower right, up and down.
- New
- The seconds before the hand touches the mouse become usable input time.
2018
An infographic of the lunchtime queue at the campus cafeteria.
2016
An interface that operates an Android phone with the eyes alone.
- What
- An interface that operates an Android phone with the eyes alone.
- Why
- Someone with quadriplegia cannot touch the screen, and gaze by itself settles where to look but not when to press.
- How
- It follows the gaze trajectory through a webcam to place the point, takes a wink as the press signal, and injects the touch event over ADB.
- New
- Together they cover the whole design space of single-point touch — tap, long press, drag — using eyes and eyelids only.
Awards
2023
iF International Forum Design · WonderScope: Practical Near-surface AR Device for Museum Exhibits
2022
KAIST · Teaching Assistant @ Wearable Interface (CS492), KAIST
- By
- KAIST
- Year
- 2022
2022
ACM SIGGRAPH · WonderScope: Practical Near-surface AR Device for Museum Exhibits
- By
- ACM SIGGRAPH
- Rank
- Honorable Mention
- Year
- 2022
2021
ACM SIGCHI · Secrets of Gosu: Understanding Physical Combat Skills of Professional Players in First-Person Shooters
- By
- ACM SIGCHI
- Rank
- Honorable Mention (top 5%)
- Year
- 2021
2020
DGIST · Chatbot with Touch and Graphics
- By
- DGIST
- Rank
- 2nd place
- Year
- 2020
2019
2018
Korea Student Aid Foundation
2016
Intel ISEF · Winklick
- By
- Intel ISEF
- Rank
- Finalist
- Year
- 2016
2016
2015
Seoul Metropolitan Science Exhibition Hall · Winklick
- By
- Seoul Metropolitan Science Exhibition Hall
- Rank
- Special Prize (Grade 1)
- Year
- 2015
2015
National Information Society Agency · Winklick
- By
- National Information Society Agency
- Rank
- Silver
- Year
- 2015
2014
Korea Institute of Information Scientists and Engineers
- By
- Korea Institute of Information Scientists and Engineers
- Rank
- Bronze
- Year
- 2014
2014
National Information Society Agency
- By
- National Information Society Agency
- Rank
- Gold
- Year
- 2014
2014
Korea Institute of Information Scientists and Engineers · Winklick
- By
- Korea Institute of Information Scientists and Engineers
- Rank
- Special Award
- Year
- 2014
2013
Ministry of Science, ICT and Future Planning
- By
- Ministry of Science, ICT and Future Planning
- Rank
- Grand Prize
- Year
- 2013
Education
Mar 2021 – Feb 2023
Advisor Geehyuk Lee · GPA 3.81/4.3
- Org
- KAIST
- Degree
- M.S.
- Major
- School of Computing
- Advisor
- Geehyuk Lee
Mar 2017 – Feb 2021
GPA 3.44/4.3
- Org
- DGIST
- Degree
- B.S.
- Major
- Interdisciplinary Engineering
Jun 2017 – Aug 2017
- Org
- University of California, Berkeley
- Degree
- Visiting Student
Teaching & Talks
2024
- Org
- XREAL
- Role
- Invited Talk
2023
- Org
- Hongik University
- Role
- Invited Talk
2022
- Org
- KAIST
- Role
- Teaching Assistant
2021
- Org
- KAIST
- Role
- Teaching Assistant
2019
- Org
- DGIST
- Role
- Undergraduate Tutor
Service
2025
- Org
- ACM UIST
- Role
- Reviewer
- Track
- Full Papers
- Year
- 2025
2024
- Org
- ACM CHI
- Role
- Reviewer
- Track
- Full Papers
- Year
- 2024
2023
- Org
- ACM DIS
- Role
- Reviewer
- Track
- Papers and Pictorials
- Year
- 2023
Skills
Spatial Computing, Gesture-based Interface, Behavioral Sensors
AR/VR (Apple Vision Pro, Oculus Quest, Microsoft Hololens), Body Tracking (Microsoft Kinect, OptiTrack Motive, LEAP Motion) · Eye Gaze (Tobii Eye Tracker, SMI iViewRED, PupilLabs, FOVE VR), EMG (MYO Armband) · Touch & Press (bHaptics TactSuit, Sensel Morph, Wooting Keyboard)
Rapid Prototyping
iOS & Mac & Apple Vision Pro (SwiftUI), Android (Jetpack Compose) · WPF & .NET MAUI & Microsoft Hololens (C#), Unity (C#), React & Electron or Tauri (TypeScript) · Processing 3 (Java), Figma · Firmware Development (ESP32, Arduino)
Signal Processing, Sensor Fusion, Time Series Analysis
Recording, Denoising, and Analyzing High-frequency Time Series Data · Prediction Models, Clock Drift Analysis, Covariance-based Multi-Kalman-Filter Fusion
Real-time Data Collection, Messaging Platform Implementation
Sensing and Recording Low-level Sensor Data (SPI, I2C, Windows HID) · Designing Peer-to-Peer Wireless Networking Protocol (UDP, TCP, RESTful API, BLE UART, Wi-Fi) · Real-time Data Integration System (Photon Unity Networking RPC, ZeroMQ, MessagePack)
User Experiment, Data Analysis & Visualization
Mixed-methods study design and operation (within- and between-subjects), performance measurement (completion time, accuracy, error rate), NASA-TLX, semi-structured interviews, Wizard of Oz evaluation · SPSS, JASP, Matlab, R, pandas, scipy, pingouin, seaborn, matplotlib, jupyter
Agent Systems, Reinforcement Learning
LLM multi-agent orchestration harness design (state-machine formalisation, task dependency graphs, failure taxonomies, checkpoint resume) · Model Context Protocol tool integration, VLM-based screen adjudication, browser automation · Agent Experience (AX) workflow design — AI-then-Human development processes wired through Jira, Slack, GitHub and Azure · Reinforcement learning (OpenAI Gym), computer vision (OpenCV), PyTorch