Columbia PhysTwin: Turning Video into a Virtual Model
First understand what Columbia / UIUC researchers are trying to learn through PhysTwin, why the question matters, how they test it, and what the result cannot prove. Turn cloth or rope motion from a video into a virtual model and see how its shape changes when the material is harder or softer. The student then completes A digital-material experiment tool for overlaying video and simulation and fitting material settings. For a college application or interview, the student separates the source research from their feature, interface, tests, failures, and revisions. Available online or in person in Apgujeong, in one-to-one or small-group formats.
University and lab names show where each public project came from. CIT independently designed these courses; they are not official university courses, partnerships, or endorsements.
8guided sessions
10-20 hoursto the first small project
Advancedrecommended level
Not required with provided dataphysical robot
Primary routes
Choose the student's primary AI education and portfolio route
CIT keeps the international-school route first, followed by the separate science and gifted-school route.
CIT lessons can run online across Korea and overseas or in person in Apgujeong. The student owns the question, code, tests and explanation; classes do not guarantee admission, selection or awards, and online delivery never changes an institution's submission rules.
How do virtual cloth and rope made from video bend and stretch when the material becomes harder or softer?
Build a virtual model that moves like the cloth or rope in a video. Make the material harder or softer, change the pushing force, and compare it with the recording.
Students first use a small working example to understand the researchers' question, method, and evidence. A controlled change helps identify what the student's extension must solve. The student then implements a useful feature, connects an operator interface, and tests the integrated application in normal, boundary, and failure cases.
What will the student complete?
A digital-material experiment tool for overlaying video and simulation and fitting material settings
The final package includes runnable instructions, the feature and interface design, normal and failure tests, one documented revision, and a three-minute explanation in the student's own words.
Feature
A feature that adjusts cloth or rope material values and saves the setting whose motion best matches the video
Operator interface
A comparison screen overlaying source video and simulated material and showing shape difference, stretch, and match score
Integrated result
A digital-material experiment tool for overlaying video and simulation and fitting material settings
Four ideas explained in this course
01reading motion from video
02objects that bend and stretch
03material stiffness
04a virtual model that matches video
Eight introductory sessions
Understand the research, then build and test a working robotics application
The 20 courses are eight-session CIT studios in which students understand a public university or lab project, confirm a working example, and then turn it into a small robotics application with a useful feature and an operator interface. Selected courses add an optional LLM explanation tool that can read run records but cannot control the robot. RoboMaster has 28 sessions, and implementation scope is adjusted to each student's experience and computer access.
Understand the source
01
Review the provided video and how it may be used
02
Read object shape and motion from video
Measure and compare
03
Set the shape and material values
04
Observe forces that push or pull the object
Build a feature
05
Change only the stiffness
06
Implement the student-owned feature: A feature that adjusts cloth or rope material values and saves the setting whose motion best matches the video
Integrate and demonstrate
07
Build the operator interface: A comparison screen overlaying source video and simulated material and showing shape difference, stretch, and match score
08
Integrate, test, and demonstrate: A digital-material experiment tool for overlaying video and simulation and fitting material settings
Public project used in class
See the exact version CIT reviewed
Public projects can change over time. To keep the class example consistent, CIT uses version 54106c6357e3 of Jianghanxiao/PhysTwin. CIT checked it on 2026-08-14; it was created on 2026-07-10. Usage-rights note: MIT.
The student changes the material values of an object rebuilt from video and sees how its bending changes. In class the project is separated into five boxes (input, memory, process, output, control). The lesson opens one of them, changes one value, and leaves the rest closed.
InputWhat comes in?
Video from several cameras, and the masks that keep only the object
MemoryWhat persists?
The object written as points joined by springs, and the material values in use
Processopened hereWhat transforms?
Nudging the material values until the motion on screen matches the video
OutputWhat leaves, and who uses it?
The replayed motion, and the numbers for how far it sits from the real video
ControlWhat decides when anything runs?
What decides how long the fitting runs, and when it is drawn on screen
The files the lesson opens, by name. The course is not a walk through the repository; it opens a chosen few and says which.
One value moves and everything else stays. The right-hand column is the prediction written before the run, not a result; the work is reconciling the two.
The universities and labs named here made the open projects this course reads. CIT designed the course independently; it is not an official, affiliated, or endorsed course.
What the student needs
Computer
GPU and provided assets recommended
Physical robot
Not required with provided data
Programs used
Python, video reconstruction, deformables, digital twins
Clear answers about what students do, what they need, and where the project came from.
Is Columbia PhysTwin: Turning Video into a Virtual Model an official course from Columbia and UIUC researchers?
University and lab names show where each public project came from. CIT independently designed these courses; they are not official university courses, partnerships, or endorsements.
Is hardware required?
Not required with provided data. The course starts with a robot on the computer or a saved recording of a completed run. Computer guidance: GPU and provided assets recommended.
What background should a student have?
Recommended level: Advanced. Students should be able to follow a guided Python example, test one controlled change, and then build and explain a small feature and interface.
What will the student make?
The completed project is A digital-material experiment tool for overlaying video and simulation and fitting material settings. The student implements A feature that adjusts cloth or rope material values and saves the setting whose motion best matches the video and A comparison screen overlaying source video and simulated material and showing shape difference, stretch, and match score.
How can the student use this project in a college application?
Explain the source research question, method, evidence, and limits first. Then separate the student's own feature and interface decisions, normal and failure tests, revisions, and next question. The source institution's name does not imply affiliation or guarantee admission.
Which project version does the course use?
CIT reviewed version 54106c6357e3 of Jianghanxiao/PhysTwin on 2026-08-14. That version was created on 2026-07-10. We keep this version during class so the example does not change unexpectedly, and we check the setup again before teaching.
Can Columbia PhysTwin: Turning Video into a Virtual Model be taken online or in person, one-to-one or in a small group?
Yes. This course is offered online and in person at CIT in Apgujeong, Gangnam-gu, Seoul, with one-to-one and small-group options. Placement and current availability are confirmed after a readiness consultation.
Is this a good first project for this student?
Before placement, we check the student's coding and math experience, available computer, interests, and ability to explain what happened.
Carnegie Mellon SPARK: Human-Shaped Robot SafetyTest a safety feature that stops a simulated human-shaped robot from getting too close to people and obstacles....→
Stanford BEHAVIOR-1K: Teaching Household TasksChoose one household chore and define the object conditions and action order a robot must check. Compare...→
Columbia AI: Predicting What Happens NextPlace an AI-predicted next scene beside one calculated with physics. See which errors build up as the AI...→