analyze_audio_context
ChatGPTHEAR the audio cheaply: classical waveform analysis (no speech model, no AI credits) of a video/audio src — windowed loudness (dBFS), silence regions, clipping/distortion detection, dynamic range. The returned silenceRegions match Clip.silenceRegions exactly, so pass them straight to edit_project updateClip (field silenceRegions) then call trimSilence — no need to detect silence yourself. Cheap and fast — call it freely, it never costs credits. ALSO RETURNS THE TEMPO GRID for music: bpm, beatOffset (seconds to the first beat), beatTimes, barTimes and beatConfidence. Cutting to arbitrary timings instead of to the beat is the loudest tell that a video was assembled rather than edited. Analyze the music you placed, then pass bpm and beatOffset to edit_project snapToBeat to move your cuts onto the grid in one call. bpm is null when nothing periodic is there, and beatReason then says WHY - so you can tell a track with no pulse (swap it) from a file too short to read (use a longer one). TRUST THE NUMBER ACCORDING TO TWO FIELDS. bpmSource: "loop-length" means the file is an exact whole number of bars and the tempo is exact - trust it. "onset" means it was read from the audio alone. beatConfidence: at or above 0.6 treat it as reliable; BELOW 0.6 treat bpm as a hint and verify before retiming a whole film - check it against the file duration (a loop is a whole number of bars) or the filename, which often states the tempo.
analyze_audio_kind
ChatGPTWHAT IS THIS AUDIO, WHO IS TALKING, AND HOW DO THEY SOUND. Per-second labels (speech / music / speech-over-music / ambience / silence / noise) with overall ratios, plus shouldTranscribe, needsAutoDuck and originalAudioReplaceable as direct answers rather than facts you have to interpret. Also speaker turns (SPEAKER_1, SPEAKER_2...) with per-speaker talk time, and per-utterance arousal and valence from prosody. Use it before spending a transcription on a file with no speech in it, and to decide whether music needs ducking under a voice. WHERE SPEECH IS comes from Silero VAD (a small pretrained speech model run on our server) or, when it is unavailable, a classical voiced-run detector; speechDetector says which, and speechRangesForTranscription are the segments that detector found. Music under a voice is judged from the audible pauses between phrases, so a hiss floor is not mistaken for a bed. originalAudioReplaceable is false when the file has no speech but has distinct sound events (soundEventCount). Everything else is classical DSP. Free: it never costs AI credits. IMPORTANT LIMITS, stated in the payload too: diarisation has no neural embedder, so it will not reliably separate two similar voices and errs toward 1 or null — treat speakerCount 2 as a hint to confirm, never as grounds for an automatic split-screen. Valence is a relative lean, not an emotion, and is capped at low confidence.
analyze_dead_air
ChatGPTTHE JUMP-CUT EDIT, PRECOMPUTED. Finds pauses, breaths, filler words, hedges and false starts, and returns removalRanges[] RANKED BY CONFIDENCE with secondsSaved and pctSaved. Take the top of the list and stop when you have tightened enough: a long silence is near-certain, a possible restart is not, and they are not presented as equals. Each range already leaves ~120ms of air on each side, because removing a pause entirely is what makes a jump-cut edit feel breathless. Filler and restart detection use the CACHED transcript of the media automatically (run transcribe on it once first; transcriptSource says which words were used), so never paste the words in: pass clipId (with projectId) or src, and from/to to look at one stretch. Without a cached transcript you get pauses and breaths only and transcriptReason says so rather than quietly returning less. The speech gate adapts to the recording rather than assuming -40dB, so a quiet take is not read as silence, and digital silence (exact zeros) is left out of that adaptation so it cannot drag the gate below every natural pause. WHETHER THERE IS SPEECH AT ALL is decided by a speech detector (Silero VAD, a small model run on our server, or a classical voiced-run detector when it is unavailable; speechDetector names which), never by loudness: a file of clicks or music gets looksLikeSpeech false and no proposals, with the reason. Free: no AI credits.
analyze_faces
ChatGPTWHERE THE FACES ARE, SO TEXT NEVER COVERS ONE. YuNet face tracking (small pretrained model, no AI credits) sampled every 0.25s. Returns faceBoxes (normalised 0-1, so they survive any resize or reframe), facePresencePct, max/median face count, unionFaceBox (every face position over the whole clip, unioned) and captionSafeZones — the bands of the frame no face EVER occupies, which is where a caption or lower third can go without covering someone at some moment you did not check. Also punchInTarget for a reframe or punch-in, lipActivity over time, and isTalkingHead (with a reason when false) to tell the talking-head plate from b-roll that merely has a person in it. Call this before placing captions on footage with people in it.
analyze_highlights
ChatGPTTHE GOOD BITS, FOR AUTO-SHORTS. A per-second interest score combining audio energy, visual motion, lip activity, face presence and transcript keywords, returning the top N moments as ranges — each carrying its COMPONENT scores so you can see why it ranked, and with starts snapped to a nearby shot boundary so a clip looks chosen rather than sliced. Also returns avSync (lip movement against speech energy, in ms — fix an out-of-sync source BEFORE cutting, because every cut inherits the drift) and, when you pass words, transcriptToShots: which shots each sentence spans and whether the speaker is on screen, with a suggestion about staying on the face or cutting away to b-roll. Pass words from transcribe to enable keywords and sentence alignment.
analyze_motion
ChatGPTIS THIS CLIP ALIVE, AND WHERE. Per-second visual activity plus dead-frame detection (no model, no credits). Returns motionCurve, meanMotion, a motionClass (static/gentle/active/frenetic), frozenRanges, blackRanges, flashTimes, duplicateFramePct, and — the useful part — deadHeadSec, deadTailSec and a trimSuggestion [start, end] you can pass straight to a trim. Use it to cut the rolling-before-the-action head and the walk-away tail without watching the file, to pick b-roll that actually moves, and to catch strobes before they ship. A static shot is the one that takes a Ken Burns move well; a frenetic one does not. motionClass is judged on motionSpeed, how many frame widths per second the picture moves (optical flow between neighbouring frames), so it does not change with sampleFps and a hard cut is not counted as motion: static < 0.005, gentle < 0.06 (a seated speaker), active < 0.18, frenetic above. Times are source-file seconds.
analyze_onscreen_text
ChatGPTWHERE NOT TO PUT CAPTIONS. Finds on-screen text regions with their timing (normalised boxes), logos and watermarks, and whether the clip is a screen recording. A caption placed over existing on-screen text is unreadable and you cannot see that you have done it — call this before placing captions on footage you did not create. Returns captionAvoidBands and a recommendedCaptionBand directly. Two honest limits it reports itself: at the 320px working width, text under about 2.8% of frame height is invisible (minDetectableTextHeightPct says so), and the WORDS: each region carries text (the words read by the tesseract OCR engine at full resolution, quoted only when confident and consistent across several frames) plus words[] with per-word confidence, and onScreenText lists the text with its seconds. If the engine is not installed, ocrAvailable is false, text and words are null and ocrReason says so; the boxes and timing still answer the placement question. Logo detection declines on locked-off tripod shots rather than guessing, because a motionless prop and a composited watermark are not separable by temporal variance.
analyze_quality
ChatGPTCAN THIS SHOT TAKE THE MOVE YOU WANT? Per shot: camera motion (static / pan with direction / tilt / zoom / dolly) and a handheld shake level, shot scale (extreme-wide to extreme-close-up) with the subject cell and headroom, focus, noise, and — the one nothing else tells you — EFFECTIVE RESOLUTION, so a 4K file that is really an upscaled 720p is flagged before you crop into it. Returns kenBurnsCandidates (the static shots), stabilisationCandidates, adjacentSameDirectionMoves (two pans the same way back to back, which reads as a mistake) and the top 5 thumbnail frames with their component scores. Shot scale WITHOUT a face is a genuine estimate capped near 0.35 confidence — check the method field before building variety logic on it. No model, no credits.
analyze_reframe
ChatGPTTURN A LANDSCAPE CLIP INTO A VERTICAL ONE WITHOUT LOSING THE SUBJECT. Returns where the crop window CENTRE sits over time as a SMOOTHED curve (cropCentreXCurve, normalised 0-1 across the source width; the window spans centre +/- cropWidthNorm/2), the crop size for the target aspect, movementCount so you can see how busy the result is, and the edit itself: keyframes (always at least two, a still crop is two identical ones, each with cropCentreX plus the clip x/y/scale for canvas) and reframeCommand, one reframeAuto command to pass to edit_project after filling in clipId. Key times are source seconds, right for an untrimmed clip at speed 1; for a clip trimmed to mediaStart m at speed s use (t - m) / s. Pass canvasWidth/canvasHeight to size it for your project, default the target aspect at a 1920 long edge. Smoothing matters more than precision here: a window that chases every micro-movement looks broken, and a still one rarely does: the curve only moves when the subject genuinely leaves a dead zone. Falls back to a saliency centroid when no face is present and marks those samples with a lower confidence and a method field; with two competing subjects it centres between them rather than choosing, which meanConfidence exposes.
analyze_shots
ChatGPTWHERE THE CUTS ALREADY ARE, and what each shot looks like. Classical pixel analysis (no vision model, no AI credits). Returns cuts[] (hard cuts, refined to a real source frame), gradualTransitions[] (dissolves/fades), shots[] with per-shot avgLuma, luma histogram, clipped-highlight and crushed-shadow percentages, approximate colour temperature, dominant colour, saturation and contrast, plus cutDensityPer10s and a pacing verdict (even vs burst-and-hold). Use it BEFORE cutting footage you have not seen: a clip you think is one shot may already contain six, and trimming across an existing cut is the most common way an auto-edit looks broken. crossShotConsistency scores how well the shots match each other and names the worst offender — that is the clip that needs a matching grade before it can sit in the sequence. Also runs on a REFERENCE video the user uploads, to learn its pacing. Every time is in source-file seconds. IT ALSO READS THE SCOPES FOR YOU: colourReadout gives a one-line summary plus specific findings — clipped highlights or crushed shadows with the exact percentage, colour casts with the channel imbalance, exposure and saturation problems, and shots that do not match each other across a cut. Each finding names the control that fixes it. USE IT TO CHECK YOUR OWN GRADE: call this again after setColor or applyFilterPreset and the findings tell you whether what you did helped, which is the only way to grade footage you cannot see. An empty findings list is a real answer meaning nothing is measurably wrong.
analyze_structure
ChatGPTWHERE THE MUSIC PEAKS AND WHERE IT HITS. Energy curve, brightness and onset density over time; sections detected by self-similarity (intro / build / drop or chorus / breakdown / outro) with a relative energy score; the single highest-energy moment; and the biggest energy jump. Plus an IMPACT MAP: hits, whooshes, risers and sub-bass drops with peak dB and decay length. Land your reveal on peakEnergyMoment or biggestEnergyJump, and align a title pop to an IMPACT rather than to the nearest beat — a hit is what the ear actually marks. hasMusicalStructure is conservative and declines on dense mixes; when it declines you still get correct boundaries and energy numbers, just unlabelled sections. No model, no credits.
analyze_visual_context
ChatGPTSEE the video cheaply: classical pixel-math analysis (no vision model, no AI credits) of a video src — shot/cut boundaries, per-shot brightness/saturation/dominant color/motion score/sharpness, face presence + bounding box (YuNet, a small pretrained model — check faceDetectionAvailable if you need to know whether detection actually ran), letterbox detection, and overall "is this static/talking-head" signal. Use this BEFORE editing footage you have not seen — to find where cuts already are, avoid trimming into a blurry shot, know if a shot has heavy motion (steady it or lean into it), or center a reframe/crop on a detected face. Cheap and fast — call it freely, it never costs credits. Cut times are FRAME-ACCURATE: each is the first frame of the new shot, so splitting there leaves no frame of the outgoing shot behind. Per shot, motionClass (static/gentle/active/frenetic) is the same measure analyze_motion classifies on, and isStatic means motionClass static; colours are decoded the way the preview and export show them (colourDecode says which matrix). It does NOT return a tempo grid: for bpm, beatOffset and beatTimes call analyze_audio_context on the music.
analyze_voice_activity
ChatGPTFind real dead air with an actual voice-activity model (Silero VAD — pretrained, no AI credits), not just an amplitude threshold — so quiet speech is never mistaken for silence. Returns speechSegments and deadAir ({start, end, confidence, reason}) gaps between them. Use this instead of analyze_audio_context's silenceRegions when the footage has soft-spoken sections a plain loudness threshold would misflag as pauses (or vice versa). Slightly slower than analyze_audio_context (real inference, not just math) — prefer that one for a quick loudness/clipping check.
bake_clips
ChatGPTFLATTEN clips into one real video file — a pre-comp. Two things need this. (1) Transforming or grading a GROUP as one unit: without a bake, a "camera shake" has to be keyframed onto every layer by hand and they drift apart. (2) Splitting a composite: a live split keeps each half sampling its own patch of the ORIGINAL footage as it travels, so parting halves visibly drain of colour — bake first and each half carries the picture it was cut from. Costs no export quota and is never watermarked; this is an editing step, not a delivery. Returns a jobId — poll get_bake_status, which performs the swap for you when it finishes.
cancel_generation
ChatGPTStop waiting on an AI video generation or living image and free its one-at-a-time slot so a new one can start. Omit jobId to cancel whatever video is currently running. The provider or worker may still finish the original request, and credits already spent are not returned.
cancel_render
ChatGPTCancel a render job from render_project that is still PENDING or PROCESSING, for example an export made obsolete by a later edit. A queued job is removed; a running one stops within a few seconds and produces no file. It then reads CANCELLED in get_render_status and no longer counts against the concurrent-render limit. The monthly export it used is not given back (a failed export keeps its count too). A job that already finished is left alone and the reply says so.
cancel_scheduled_post
ChatGPTStop a post from going out. It stays on the calendar marked cancelled (update_scheduled_post with a new time puts it back). Refuses a post that is being posted right now. Cancelling a post that ALREADY went out does not remove it from the platform; the result says so, and you must tell the owner. Get postId from list_calendar. Needs the Creator Studio subscription.
create_project
ChatGPTCreate a new empty video project and return its id. Pass workspace to file it in a folder as it is created — see list_workspaces.
create_upload_link
ChatGPTGet a one-time upload link for a LOCAL file (video, audio or image) that you can reach from a shell but cannot pass to a tool: the user's footage in your workspace. Send the file with the curl command it returns (curl -T <file> <uploadUrl>); the response is JSON with the durable url to use as an addClip src, exactly like import_media. The link works for 30 minutes, for this account only, within the plan's upload size. In ChatGPT, pass the attachment to import_media as file instead.
create_workspace
ChatGPTCreate a workspace (folder) for grouping projects. Use it when the user is starting a body of work — a client, a series, a campaign — not for every project. Returns its id, which create_project and move_project_to_workspace accept.
edit_project
ChatGPTApply timeline editing commands to a project (add/trim/split/style clips, captions, keyframes, effects, transitions, and more). Commands are { op, ...params } objects — discover them with get_command_schema. Returns per-command results PLUS layout lint warnings (overlapping text, offscreen elements, unreadable sizes) — always fix lint warnings before rendering. Pass previewAt:[times] to get rendered frames back as images in this same response and SEE your edit immediately. Omit projectId to edit the most recent project; set dryRun=true to preview without saving. Pass a requestId you invent to make the batch idempotent — a retry with the same id after a timeout will not double-apply.
edit_readiness
ChatGPTSTART HERE on any file you have not edited before. ONE call that answers what an editor answers in the first ten seconds of opening something: is this a talking head, is there music already, is it noisy, is it multi-shot, does it need reframing for vertical, how many seconds of dead air are removable — each as a fact WITH the evidence beside it, so you can disagree with a conclusion rather than taking it on faith, and null WITH a reason wherever it could not be measured. Then recommendedFirstOps: the first three operations, ordered by how much they change the edit, each naming the tool that performs it. Cheap (it composes the other analysers), no credits. It samples a window rather than the whole file and says so in coversWholeFile.
generate_image
ChatGPTGenerate a photorealistic or general-purpose stylized image from text and optional references using Grok, OpenAI or Google Vertex (Studio-plan feature, expensive in AI credits — much higher fidelity and general-purpose than generate_sticker, which is cheap and limited to transparent die-cut stickers). Synchronous — returns a url immediately, usable as an image clip src via edit_project addClip.
generate_live_image
ChatGPTBring a STILL to life cheaply with vidmoat-image-live-1 (every paid plan, Creator and up; not on Hobby). It moves only what the still already shows, measured: a camera move with real depth parallax (push in, pull out, drift, orbit, pan), ambient motion inside one region with everything else frozen (water, waterfall, waves, clouds, smoke, steam, fire, leaves, wheat, grass, hair, flag, curtain), or a gentle object motion on a cut-out subject (float, bob, sway, tilt). 8 credits a second, 2 to 10 seconds (5 s = 40), video only, no audio. Where the generative tier (G, our own Wan 2.2 model on our GPU) is available, faces breathing or blinking, people moving, vehicles driving, turning objects, and ambient or object motion described in words go to it: realistic motion of THIS picture, 24 credits a second, 4 to 5 seconds, 16:9, 9:16 or 1:1, no loop, about 2 to 5 minutes (the GPU wakes per job); every clip is checked so that nothing new appears (a person, hand, animal or vehicle that is not in the still) and is refunded if it fails. Camera moves, and any request that sets region, motion, box, strength or direction, stay on the cheaper tiers. It REFUSES, before charging, anything appearing, pouring, breaking or transforming (generate_video with imageUrl), and without the generative tier also faces, bodies, vehicles and turning objects. The output is MEASURED before delivery: a clip that does not move, moves outside the region, drifts from the still or gains a subject ends FAILED and the credits are refunded automatically. Asynchronous: returns a jobId; poll get_generation_status. Call with quoteOnly:true first to see the tier, method and price.
generate_music
ChatGPTGenerate original music or a song with Google Lyria (Studio). Describe mood, instruments, tempo and structure; optionally specify instrumental only or desired vocals/lyrics. Never use speech generation to read a musical brief as a substitute. Returns an audio URL and measured duration when available; inspect the sound before claiming musical content. Add the URL as an audio clip and duck only under verified speech.
generate_speech
ChatGPTGenerate spoken audio from text using Auto, Grok or OpenAI text-to-speech (Creator-plan feature, cheap in AI credits relative to generate_image/generate_video). Synchronous — returns a url immediately, usable as an audio clip src via edit_project addClip, or just to hand the user a voiceover/spoken line.
generate_sticker
ChatGPTGenerate a transparent AI sticker from a text prompt (premium feature, uses AI credits). Returns a url to use as an image clip src via edit_project addClip. Styles: cartoon, kawaii, retro, neon, 3d, minimal. Pass rigReady:true to get an ANIMATABLE character instead of a flat sticker — see that parameter.
generate_video
ChatGPTStart generating a short AI video using Grok/xAI or Alibaba HappyHorse (paid Studio only; trials and fully waived promotions excluded;, very expensive in AI credits). From a text prompt, or — by passing imageUrl — BY ANIMATING A STILL THE PROJECT ALREADY HAS, which is how you move a real logo, product shot or frame rather than inventing a lookalike. Asynchronous: returns a jobId; poll get_generation_status until status is COMPLETED. Only one generation can run at a time per account.
get_bake_status
ChatGPTPoll a bake_clips job. While it runs, returns status/progress. When it succeeds it SWAPS THE PROJECT for you: the baked clips are replaced by one video clip pointing at the rendered file, with the originals stashed so unbakeClip can restore them. Takes a few seconds to a minute depending on length.
get_blueprint
ChatGPTThe owner's stored Content Blueprint: their niche, audience, edge, content pillars with a target mix, posting cadence, what to do more and less of, and a month of ideas, with how confident it is and what it was based on. Returns blueprint: null when none has been built, and stale: true when their work has changed since it was written. Never builds one. Read-only. Needs the Creator Studio subscription.
get_challenges
ChatGPTThe owner's Creator Studio challenges (post five days running, ten posts through Studio, a thousand views on one video, use every connected channel) with current progress, target and whether each is done. Progress is computed fresh from their posts every call. Days are counted in the timezone given; UTC when omitted or unrecognised (the result says when it fell back). Read-only. Needs the Creator Studio subscription.
get_command_schema
ChatGPTGet the full timeline command schema — every operation edit_project accepts, with parameters. Call this once before your first edit_project call.
get_creator_overview
ChatGPTThe numbers on the owner's Creator Studio Overview, exactly as their screen shows them: connected channels, finished exports, posts published and views over the last 90 days, the best channel, twelve weeks of posting rhythm, the weekly streak, the top three posts, the next scheduled posts and the blueprint's pillar mix. views is null (never 0) when no post could be measured. Use this for "how am I doing"; use get_post_analytics for per-post detail. Read-only. Needs the Creator Studio subscription.
get_creator_preferences
ChatGPTRead the authenticated creator’s optional onboarding defaults, suggested editing briefs and usual output format. Read before planning; explicit current instructions and existing project settings override these defaults. This grants no permission to spend or edit. Disabled personalization returns no active defaults.
get_generation_status
ChatGPTPoll the status of a generation job started with generate_video or generate_live_image. Returns status PENDING/PROCESSING/COMPLETED/FAILED and, once COMPLETED, outputUrl. A living image (kind "live") also returns what was measured to move (motion.measured) and the credits charged and refunded; FAILED there means the motion check rejected it and the credits came back.
get_post_analytics
ChatGPTHow the owner's posts are actually doing: views, likes, comments, watch time. A metric that could not be read comes back as null with a reason, NEVER as zero — treat null as "nobody counted", not as "nobody watched". The totals say how many posts they cover, because most platforms have not approved our app yet and their posts are genuinely unmeasured.
get_project
ChatGPTGet a project: settings, tracks, and a per-clip summary (id, type, name, start, duration, trackIndex). Pass full=true for the complete editor document including keyframes, effects and text styles.
get_render_status
ChatGPTCheck a render job: status (PENDING/PROCESSING/COMPLETED/FAILED/CANCELLED), progress 0-100, and the download URL when completed. Lost the jobId (a parallel call swallowed it, a retry after a timeout)? Pass projectId instead, or nothing at all, and get that project's (or this account's) most recent render jobs, newest first, with the active ones marked.
get_skill
ChatGPTVidmoat craft skills (the Agent Skills SKILL.md format). With no arguments: the editing manual (the loop that works, the mechanics that cost other agents whole runs such as clips being top-level and setColor being 0-200 absolute, plus pacing, titles, audio and colour). READ IT ONCE at the start of a session. With name: that skill's SKILL.md (craft for one kind of edit or technique, or a specialist brief) and the list of its files; with name and file: one of those files. list_skills returns the catalog of names and what each is for.
get_specialist_brief
ChatGPTGet a professional craft brief from one of the Vidmoat skilled agents: a film editor, colourist, sound designer, motion designer, short-form strategist, retention editor, documentary storyteller, ad creative director, interview editor, accessibility lead, brand guardian, music-video editor, trailer editor, explainer producer, YouTube editor, vlog editor, podcast editor, course editor, gaming editor, video essayist, wedding and event filmmaker, real estate editor, UGC ad creator or food creator. Call this BEFORE planning an edit whose goal falls in one of those domains ("make it cinematic", "cut this for TikTok", "the audio sounds amateur"). Each brief carries the order of operations, the actual numbers professionals work to (loudness targets, safe zones in pixels, hold times, frames per beat, contrast ratios), a DO-NOT list, and an acceptance test you can verify from a preview frame. The brief comes framed exactly as the in-app agent reads it (how to apply it, the editorial direction, and which deterministic verifiers check that craft). Pass projectId and goal to have the specialist read the measured evidence of that project and return a cited beat sheet. Omit id to list every specialist with what it is for.
get_video_object_removal_status
ChatGPTCheck the progress of an object-removal job started with remove_object_from_video, and get the cleaned clip when it is done. Read-only. Poll a remove_object_from_video job. Returns PENDING/PROCESSING/COMPLETED/FAILED plus start, end, region and verification. COMPLETED means the output was measured (target box changed, everything else matches the original) and is ready to review and apply; the project is unchanged. FAILED means the original clip is untouched and credits were returned; when verification.ok is false, error and verification.reasons give the measured reason (for example the object is still there), so change the window or box rather than retrying the same request.
import_animated_sticker
ChatGPTImport a user-provided vector Lottie JSON, GIF or animated WebP as a timeline animated sticker. No AI generation credits. Lottie: animationData object; GIF/WebP: base64 file bytes. Limits: 5 MB file (1 MB Lottie), 6 seconds, 180 frames; compiled at up to 256 px/30 fps, 2 MB embedded output. No external assets, expressions, images or fonts inside Lottie. Requires explicit projectId and requestId. Use search_editor_catalogue kind:elements query:animated for free built-in artwork instead. Playback controls use edit_project setStickerPlayback.
import_media
ChatGPTBring media into the Vidmoat media store: a public media URL (video, audio, or image), or file, a file the user attached in this conversation (their own footage, music or photos). To upload a file from a machine you can run commands on, call create_upload_link instead. Returns a durable url to pass as src to edit_project addClip, plus the probed duration in seconds. Use this to bring footage, music, or images into a project. Not for 3D models (.glb/.fbx): bring those in with upload_media, then render them with the anim3d plugin (import_model, render_scene) and add the resulting video.
inject_html_animation
ChatGPTInject a large, self-contained HTML/CSS element into a Vidmoat project as a new clip — for designed graphics too complex for the standard shape/text commands (lower thirds, title cards, stat blocks, gradient backgrounds). HOW IT RENDERS: the element is wrapped in an SVG <foreignObject>, rasterized ONCE to a still bitmap, and that same bitmap is drawn on every frame. It is never a live DOM node, so it obeys the rules of SVG-as-image, not the rules of a page. ANIMATING IT: two CSS variables are set by the rasterizer on every frame, so an element CAN move without being baked as N duplicated clips. --vm-p is clip progress (0 on the first frame of the clip, 1 on its last) and --vm-t is clip-relative seconds. Both are unitless, so compose them with calc(): a trim-path graph reveal is stroke-dashoffset:calc((1 - var(--vm-p)) 1000); a radial chart wipe or donut is conic-gradient(... calc(var(--vm-p) 360deg) ...); a spinner is transform:rotate(calc(var(--vm-t) 360deg)); a count-up is a row of digit strips translated by calc(var(--vm-p) -100%). Purely opt-in: mention either variable and the element is re-rasterized per frame, ignore them and it keeps its single cached bitmap and costs exactly what it did before. Drive the SHAPE of the motion with a keyframed scale/opacity on the clip if you want easing — --vm-p is linear. DOES NOT WORK (fails silently, no error): CSS animations and transitions — @keyframes never advance, so the graphic renders permanently at its starting state and anything beginning at opacity:0 stays invisible for its whole duration; animate the CLIP with addKeyframe on opacity/scale/x/y instead. backdrop-filter (nothing to sample — frosted glass comes out as a flat translucent panel; bake the tint into the background instead). mix-blend-mode (set the clip's blend mode with updateClip instead). position:fixed (falls back to static and the element reflows out of place — use position:absolute inside a position:relative parent). Web fonts, @font-face, and remote <img>/url() (no network during rasterization — use system font stacks and inline SVG or data: URIs). <script>, event handlers, and iframe/link/@import are stripped outright. WORKS: flexbox and grid, linear/radial/conic gradients, border-radius, box-shadow, text-shadow, filter (blur, drop-shadow), transform, clip-path, -webkit-background-clip:text, CSS custom properties, system fonts, inline SVG. EMOJI: supported, but each machine supplies its own emoji font (Segoe UI Emoji on Windows, Apple Color Emoji on macOS, Noto Color Emoji on the render server) — so the exported video will NOT match what the preview showed. They are also glossy shaded pictographs, which clash beside flat 2D vector shapes and single-weight icons; do not mix the two registers in one graphic. Prefer inline <svg> for iconography — it renders identically everywhere and inherits the design's stroke weight and palette — and reserve emoji for when a colour pictograph is genuinely the subject. Every one of these is checked after the edit and reported back in lint[].
list_calendar
ChatGPTWhat is scheduled to post, and what already went out. Returns each entry with its time, timezone, channels, status and — for anything published — the live URLs. Use this before scheduling so you do not double-book a slot the owner already filled.
list_generations
ChatGPTList this account's AI video generations and living images, running first (each with kind "video" or "live"). Use when generate_video or generate_live_image says one is already running: it returns the jobId to poll with get_generation_status, or to free with cancel_generation.
list_projects
ChatGPTList the 20 most recently edited Vidmoat video projects (id, name, updatedAt, workspace). ALSO RETURNS total, returned and truncated: when truncated is true the list is a WINDOW, not the whole account, so never answer a "how many" question from its length. Pass workspace (a workspace name or id from list_workspaces) to list only what is filed there — on a busy account that is far more useful than the 20 most recent of everything, and it makes total meaningful again because it counts that folder only.
list_recipes
ChatGPTBrowse published Recipes: title, what each makes, the techniques it uses and how many videos have been built from it. Returns previews only — building from one needs a recipe key, which the owner gets from the Recipes page.
list_skills
ChatGPTThe catalog of Vidmoat craft skills: each skill's name, what it covers and when to use it, its category (specialist, craft, reference, manual), version and files. Load one with get_skill and its name before planning the part of an edit it covers. Read-only.
list_workspaces
ChatGPTList the workspaces (folders) projects are organised into, with how many projects are in each. A workspace is NOT a team: a team decides who can see a project, a workspace decides what it is part of. Call this before create_project if the user mentioned a client, a series or a campaign — filing new work in the right place is the difference between a tidy account and 124 loose projects.
move_project_to_workspace
ChatGPTFile a project into a workspace, or (workspace omitted) take it out of the one it is in. Moving a project between folders never changes who can see it and never touches the timeline.
plan_week
ChatGPTA plan for the owner's next week, built from THEIR posting history and numbers — what did well, what times worked, which connected channel they have been ignoring. Each idea carries a prompt you can act on directly and, where one fits, a Recipe slug to build it from. When there is little history it says so; do not present a thin plan as a data-driven one.
preview_frame
ChatGPTSEE the video: renders the project frame at a timestamp exactly as the editor/export pipeline draws it (clips, keyframes, effects, captions) and returns the image. Use it after edits to check composition, overlap, readability and color before rendering. Takes a few seconds.
preview_strip
ChatGPTSEE THE TIMELINE: renders frames as ONE labelled contact sheet. Called with just a projectId it samples at what CHANGES — every clip in and out, markers, and the peaks of keyframed moves — each cell labelled with why it was picked, so you can follow what the edit actually does and tell which frames are worth a closer look with preview_frame. Even sampling across a range misses cuts entirely; this does not. For anything with timing in it — a ramp, a hit, a reveal, a transition — a single still answers almost nothing, and this costs roughly what one preview_frame costs because the browser is launched once and then seeked. Use it to check that an effect ramp actually ramps, that a move lands on the beat, or that a reveal reads. Pass from+to (with optional count), or explicit times[]. Max 12 frames.
quote_expert_edit
ChatGPTPrepare a scope and price for human video editing. Creates an estimate only: never books an editor or charges money. Ask the user for footage length, output duration/count and desired changes. Share any clarification and send the user to the reviewUrl to explicitly choose paid or optional volunteer help.
rebuild_blueprint
ChatGPTBuild a new Content Blueprint from everything the owner has made and REPLACE the current one. Takes about a minute of model time and is rate limited like the Rebuild button. Only call it when the owner has asked for a new blueprint, and pass confirm: true; without it nothing happens. If the build fails, the previous blueprint is left untouched. Needs the Creator Studio subscription.
remove_object_from_video
ChatGPTRemove an unwanted object or person from a video clip and fill the area in (AI inpainting). Starts a job that charges credits; poll with get_video_object_removal_status. Remove an object from an owned clip. Video takes MINUTES and uses paid fal inference: provide removeWhat (object to erase) and backgroundDesc (scene to keep), both from the user, plus a stable requestId. Video without a region is refused. Only the window is processed and priced, only the box is replaced, and everything else stays the original picture. Call quoteOnly first, show the credit price, then maxCredits to start. Reuse requestId after disconnections; retryOf identifies a new paid retry. The result is MEASURED before it is delivered: the box inside the window must actually change and frames outside the box and window must match the original. If that fails the job ends FAILED, nothing is published, the credits are returned automatically, and error/verification say what was measured; do not simply retry the same window and box. Completion means verified and ready for review, not applied. Images retain point-based removal using fractions of the FRAME, not pixels. Inpainting GUESSES the hidden background; inspect the result before applying.
render_project
ChatGPTQueue a video export (render) of a project. Returns a jobId to poll with get_render_status. Plan limits (resolution cap, watermark, export quota) apply.
report_issue
ChatGPTTell the Vidmoat team something about Vidmoat itself. Call this WHENEVER you hit a wall: a tool that does not exist but should, a command that failed in a way you could not work around, a result that did not match what the schema promised, or an instruction that reads one way and behaves another. Also use it for a plain suggestion. You are the only witness to most of these — the user sees a video that came out slightly wrong and never knows which capability was missing, so if you do not report it nobody ever learns. It is free, it costs no credits, it never changes the project, and a report is more useful than a workaround you quietly invented. For a missing capability, put the NAME YOU WANTED TO CALL in tool.
sample_clip_props
ChatGPTREAD BACK WHAT THE RENDERER WILL ACTUALLY DRAW: the computed value of the animated properties on a clip at chosen times, from the same function the compositor calls. get_project returns the keyframes you WROTE — your own input echoed back — so it cannot tell you whether an interpolation is doing what you meant. This can. Every property comes back with values, deltas, and a shape (linear / accelerating / decelerating / accelerating-then-decelerating), plus linearityError: 0.000 means dead linear, a cubic ease-out is around 0.15. For a keyframed property, shape and linearityError are measured INSIDE each ramp between two keyframes (listed in segments[]), never across the held values before or after it, so a linear ramp reads linear however you sample. USE IT TO VERIFY AN EASE INSTEAD OF EYEBALLING FRAMES — at 8 frames across half a second you cannot see the difference between easeOutQuart and linear on a small move, and a silent interpolation bug stays invisible until a person says the video feels wrong. Omit times to sample evenly across the clip. Omit props to get whatever is keyframed on it. Costs nothing and hits no model.
schedule_post
ChatGPTPut a finished export on the calendar. Vidmoat posts it at the time you give, on every connected channel whose platform app is approved; the rest appear as a reminder for the owner to post by hand. Render the project first (render_project, then get_render_status until COMPLETED) and pass that render job id. Channels may be named by platform ("youtube") or by account id; omitting them means every connected channel.
score_cut_alignment
ChatGPTDOES THIS EDIT FEEL MUSICAL OR ASSEMBLED? Give it your cut times and a beat grid; it returns the fraction landing within 60ms of a beat, the mean deviation, a verdict (musical / mostly-on / assembled) and the worst offenders WITH the beat each should move to, so the fix is mechanical. No media is decoded, so it is instant — re-run it after every adjustment. READ systematicOffset FIRST: when nearly every cut misses in the same direction by a similar amount, the beat GRID phase is wrong and one beatOffset correction fixes the whole edit; moving each cut individually would be the wrong action entirely. Also runs on a reference video the user admires — analyse its cuts and its music, score them here, and you have measured how that editor cuts.
search_editor_catalogue
ChatGPTSearch Vidmoat editing capabilities by name, category or purpose. Read-only; no generation, download, edit or credit charge. Returns exact supported names, usage, font weights, access requirements and pagination. Search before choosing unfamiliar fonts/styles; these results never bypass execution permissions. Empty query browses a category. Stock media has its own search_stock_media tool.
search_stock_media
ChatGPTSearch royalty-free stock video/photos (configured Pexels, Pixabay, and optional Wikimedia Commons CC0/public-domain metadata) or stock audio (Freesound, CC0 only — no attribution needed) by keyword. Returns candidates with a downloadUrl and source/license metadata — pass the one you want to import_media with optional provenance {sourceUrl, license, licenseUrl}, then addClip with the url it returns (type: "audio" for music/SFX results). Use when a project needs b-roll, an image, or a music bed/SFX the user did not supply themselves. AUDIO SEARCH IS TAG-MATCHED AND NARROW: the provider ANDs every word, so a full brief like "clean minimal tech corporate loop upbeat" matches nothing on its own. One or two concrete words work far better. If a multi-word query would come back empty it is automatically retried term by term and the results merged, ranked by matchedTerms (how many of your words each one actually matched) — so check that field rather than assuming the top hit answers the whole brief. Audio results also carry bpm when the filename or tags state one, and you can filter by it.
transcribe
ChatGPTGet word-level timings for captions. Two modes: pass src (an audio/video URL, e.g. a clip src) for real speech-to-text (uses AI credits), OR pass script + duration (+ optional start) to distribute known text across a time range for free. Feed the returned words[] into edit_project addCaptions.
update_scheduled_post
ChatGPTChange a post that is on the calendar: move it (scheduledFor, timezone), change where it goes (channels), or change its words (title, description, privacy apply to every channel), or swap the export (renderJobId). Only the fields you pass change. Get postId from list_calendar. Refuses a post that is being posted right now or has already been posted; refuses a time in the past, a channel that is not connected, and a render that has not finished. A FAILED or CANCELLED post given a new future scheduledFor goes back on the schedule. Returns the post as it now stands. Needs the Creator Studio subscription.
upload_media
ChatGPTUpload an image or a 3D model YOU already have (as base64) into the Vidmoat media store: the way to supply your own transparent PNG, hand-authored SVG or .glb/.fbx model when no public URL exists for it. An image returns a durable url usable as an addClip src or as imageUrl for the character-rig plugin. Accepts png, jpeg, webp, gif, svg, glb, fbx; for images with a public URL, prefer import_media.
watch_video
ChatGPTYOUTUBE EXCEPTION: Public YouTube links use Vertex reference analysis and require a project (projectId for MCP; active project in channels). Default first 60 seconds, maximum range 120 seconds, optional query and textHeavy. Returns cached project evidence with ABSOLUTE SOURCE times, sampled visuals and UNVERIFIED audio estimates; no waveform validation, imported footage or exact transcript. For ordinary media files: Returns optional classification with contentType, visible subject/setting/shot tags and model confidence; these are visual estimates limited to the sampled footage, never proof of sounds or personal identity. WATCH AND LISTEN to footage and get back what is actually in it, in words. Costs a vision model call — the analyze_* tools are free and answer different questions, so reach for those first. This one answers "what IS this", not "where are the cuts": a timestamped scene list with a summary of each, the on-screen text, and audioEvents (things HEARD, with a timestamp). Use it on footage you have not seen before deciding what to cut, on a clip a user refers to by content ("the bit where she picks up the cup"), and to find where on-screen text already exists so you do not caption over it. PASS startSec AND endSec TO DESCRIBE ONE CLIP rather than the whole source: the range is cut out before it is watched, so a 6 second clip from a 40 minute file costs 6 seconds of attention, and every timestamp comes back RELATIVE TO THE CLIP, where 0 is its first frame. AUDIO IS VERIFIED, NOT TRUSTED: a vision model asked about sound will describe a soundtrack that suits the picture, so every claimed audio event is checked against the measured waveform and discarded if it lands on silence (source "model"), and distinct sounds the waveform contains but the model did not name (clicks, impacts) are added as source "measured" with a generic label, because an onset proves a sound is there, not what it is. measuredSoundEvents.count is the full measured count. That makes audioEvents safe to cut on — and heardAudio tells you whether anything was listened to at all, so an empty speech field is never confused with a quiet video. For WORDS use transcribe, which is better at them; this gives the gist and the sound, not a transcript.
