When RapAI Video turns two selfies into a lip-synced rap duo, what happens between "upload" and "render"? No formulas here — just the three moving parts, in plain language.
The generator first builds a rough 3D map of each face from the selfies: where the jaw hinges, how wide the mouth opens, where the eyes sit relative to the nose. This is why photo quality dominates results — a shadow across the jaw reads as a wrong hinge position, and every animated frame inherits that error. Front-facing, even light, no sunglasses: the map is only as good as the photo.
Spoken language is a small set of mouth shapes (phonemes) repeated in different orders. The audio track gets broken into these shapes — "oo" is lips rounded, "ee" is lips stretched, consonants are quick closures. Each phoneme maps to a mouth shape on the face map. This is why lyrics with crisp consonants sync better: the anchors are easier to place. Mumbled bars give the anchor system mush to work with.
Frame-by-frame phoneme shapes would look robotic, so a smoothing pass blends between mouth shapes the way a good animator draws in-betweens. Too little smoothing and the mouth flickers; too much and it lags behind fast rap delivery. This is why beat choice matters — trap hi-hat rolls stress the smoothing system harder than boom bap's steady cadence.
Video input of a person rapping would give perfect sync but zero novelty — you already filmed them. Two photos let the generator invent the performance: any two people can become a duo. The trade is sync fidelity, which is why the format works best at social-video resolutions where small imperfections vanish.
The whole pipeline runs in the browser at rapaivideo.net — upload two selfies, paste lyrics, pick a beat, and the free preview shows you all three stages in one render. Once you can spot the three parts in the output, you will also know exactly which lever to adjust next.
Plain-language explainer, November 2026.