account_status
ChatGPTRead the credits currently available in the connected Shorty account. Works when the balance is zero. Does not purchase credits or change the account.
add_background_music
ChatGPTGenerate an original instrumental using the website controlled music vocabulary and pricing. Returns an immutable audio asset; the desktop author applies soundtrack placement and volume locally. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
analysis_capabilities
ChatGPTRead local browser evidence and default backend transcription boundaries. No upload, transcription or rendering is started.
apply_blob_tracker
ChatGPTMeasure connected bright, dark or keyed-color regions in one supplied video shot, associate stable blob IDs across frames, and render actual boxes, center markers, labels and connecting paths into a silent video. This is a pixel-region effect, not semantic object/face recognition. Up to 20 seconds per call. Large backgrounds and tiny noise can be excluded by area. Missing regions draw no overlay; merged/split regions can receive new IDs. Inspect the returned contact sheet before using the derived video with muted Video and trimBefore=0. It already includes the original picture; preserve its aspect/crop, and use the original source for any audio. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
author_edit
ChatGPTDIRECTOR STEP 2. Submit YOUR authored CreativeVideo.tsx. The backend runs the full gauntlet — source policy, strict TypeScript, Remotion bundle, sandboxed half-scale preview — with no AI involved. On failure you get the exact compiler/policy errors to fix; on success you get a persisted planRevisionId plus a preview contact sheet image of your own cut to critique. Iterate until the preview looks right, then call render with the planRevisionId.
authoring_contract
ChatGPTDIRECTOR STEP 0. Read the enforced Remotion authoring guidance, import allowlist and composition contract before author_edit. Use font_catalog, effect_catalog and media_contract for their dedicated catalogs.
backend_features
ChatGPTDiscover this deployment's complete editing capabilities, authoring guidance, effects, typography, segmentation, tracking, spatial text and token pricing. Read before choosing a workflow.
browser_heartbeat
ChatGPTApp-only bounded editor readiness and command delivery. No source, original media or rendering.
browser_result
ChatGPTApp-only metadata acknowledgement. Never send source, media bytes, comments, transcripts or signed URLs.
browser_session
ChatGPTOpen or resume an account-scoped browser editing session. No uploads, paid calls, source compilation or server rendering.
browser_status
ChatGPTRead account-scoped browser command acknowledgements. Pending means the browser has not confirmed completion; this tool never renders or uploads.
cancel_job
ChatGPTCancel an existing durable job when the user requests cancellation. Abandons in-flight work.
cast_voice
ChatGPTCast this edit's narrator from a voice brief of permanent traits. With only the brief, returns the closest members of Shorty's shared voice cast; call again with castVoiceId to reuse one (instant, free) or designNew:true to design a new voice from the brief (about 30 s; a listener reports how it sounds). At most 2 designs per edit. The voice locks once a line is spoken. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
change_voice
ChatGPTReplace the voice of an existing single-speaker speech clip using ElevenLabs on fal. First call list_voices and choose by the returned descriptions. Returns a NEW immutable clip with unchanged picture and validated speech timing, plus an actual audition verdict and swap instructions. Only when the USER asks for another voice; not accent removal, text edits, new narration, cleanup, singing, overlapping speakers or prominent music. Whole clips up to 120s; six variants / 360 submitted seconds per edit, $0.30/min base provider cost. Clean first when needed, convert second, level last. Never convert a levelled or already converted clip. Exact repeats reuse results; rejected words/timing/auditions keep the original. No strength, style or accent controls exist. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
choose_reference
ChatGPTOPTIONAL STEP 1. Upload the reference video whose editing style Shorty will copy: a TikTok, Instagram, YouTube, Facebook, or X post link, a direct media URL, or a local file path. Post links are resolved to the underlying video for callers with credits. Hosted callers passing a path get back a one-line signed upload command to run (with a shell tool, or pasted by the user); stdio reads the path directly. Creates the edit project and publishes the reference as an immutable asset. Skip it to edit without a reference — start with add_footage instead.
choose_soundtrack
ChatGPTChoose requested reference audio, a Shorty-library or generated instrumental track, or no soundtrack; returns the selected track and beat grid. This browser host does not expose get_audio or native audio evidence. Base the reason on the user brief and actual host-supported evidence; never claim to have heard audio. Selected reference media must already belong to the backend project. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
compose_spatial_text
ChatGPTCreate transparent video of authorized text inside or behind measured scene geometry over an exact source interval. Choose font, styling, anchor, masking and motion controls. Returns a derived alpha-video asset. Uses immutable assetId in projectId. Read media_contract for worker limits and pricing. Starts a paid durable job; use job to inspect progress. Preserves source assets.
create_project
ChatGPTCreate a named project in the connected account, or return the existing project with that name. Adds a project without replacing existing work.
editor_session
ChatGPTRead a completed render's editable composition. For client-authored renders, returns the accepted source file map, original footage/audio bindings and cues, direction and duration needed to author a revision. Legacy editor compositions use their existing session format. Does not save or start work.
effect_catalog
ChatGPTRead or search the installed text animation, treatment, infographic, graphic loop and sticker catalogs. Optional category and query filter the returned entries. Does not start work.
execute_tool
ChatGPTCompatibility dispatcher for the fixed backend operations individually exposed by this connection. Prefer each named tool, including get_transcript. Rejects any operation outside that upfront set and enforces its strict parameter schema, current availability, account ownership and website billing. Never authors or renders a composition.
font_catalog
ChatGPTRead the installed font catalog available for authored Remotion compositions. Does not start work.
generate_shot
ChatGPTGenerate a missing video shot using the backend's configured generator. Returns an immutable footage asset after completion. Billed per requested second (backend_features pricing.shotGeneration.creditsPerSecond), so request only the length the edit needs; use job to resume.
generate_sound_effect
ChatGPTGenerate an original MP3 sound effect using the configured external provider and store an immutable asset. Reserves credits; accepts a 0.5–22 second duration. Omit projectId to create or reuse the Sound effects project. Same inputs/runKey resume one generation; change runKey for a new variation. Retrieve existing audio with get_sound_effect. Does not place audio into an edit.
generate_sound_effect_job
ChatGPTGenerate a requested original sound effect through the same durable, billed provider job as website revisions. Place the returned audio asset in the local composition. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
generate_video_shot
ChatGPTGenerate one requested original video shot using the website generation service. Returns an immutable project video asset for local composition. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
generate_voiceover
ChatGPTSpeak narration lines in the cast voice. Each line id (vo1..vo20) becomes an audio asset with that logicalAssetId for place_audio_cues; re-sending an id records a new take. text is verbatim (inline tags like <short pause> allowed), style is that line's sustained delivery. Returns each line's exact duration in seconds and frames and its word timings. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
get_frames
ChatGPTAsk the browser to extract bounded local media evidence directly into your AI host context. You, the calling agent, perform semantic analysis. No Shorty analysis model, upload or Shorty credit charge. Sampled evidence is not a complete semantic analysis.
get_project
ChatGPTRead one project's assets, analyses, plan revisions, edit runs, renders and jobs. Use returned editRunId or renderJobId with status. Does not create a project or start editing work.
get_sound_effect
ChatGPTInspect or wait for an existing sound-effect job in the connected project and return verified inline audio. Does not generate audio, reserve credits, cancel work or persist delivery files.
get_source
ChatGPTAsk the open browser editor to share its actual source, local media metadata and revision directly with your AI host. It does not upload source or original media to Shorty.
get_transcript
ChatGPTWord-level transcript of one clip's own audio with source-second timings, optionally limited to a span. Empty when no speech was recognised. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
ground_visual_entities
ChatGPTMeasure semantic boxes, centers, landmarks or planar surface corners in a still image or up to eight exact video frames. Supply named entities, landmarks and a question. Returns immutable normalized geometry for composition authoring. Uses immutable assetId in projectId. Read media_contract for worker limits and pricing. Starts a paid durable job; use job to inspect progress. Preserves source assets.
import_stock
ChatGPTImport stock candidates from search_stock results into this edit. Each becomes an asset with a logicalAssetId (stock-v1, stock-i1, ...) you mount with getAsset(props, id) like a supplied clip. Returns each asset's size, length, frame rate and fillMagnification, and shows four frames across every video so you can check the whole clip, not just its thumbnail. Up to 4 per call and 6 per edit; importing a stockId twice returns the same asset. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
inspect
ChatGPTDIRECTOR STEP 1. Watch the project's media: returns contact-sheet images sampled at an exact cadence (default 10fps, the backend analyzer's cadence) plus duration, dimensions, and audio facts. Adaptive workflow: first pass the whole clip at 10fps; if adjacent tiles show fast cuts, whip transitions, or beat flashes you cannot place precisely, re-inspect JUST that section with fps up to 24 and a startSec/endSec window. Each sheet is labeled with its exact time range; tiles read left-to-right, top-to-bottom at 1/fps seconds per tile.
inspect_color_grade
ChatGPTAsk the browser to extract bounded local media evidence directly into your AI host context. You, the calling agent, perform semantic analysis. No Shorty analysis model, upload or Shorty credit charge. Sampled evidence is not a complete semantic analysis.
inspect_edit_references
ChatGPTAsk the browser to extract bounded local media evidence directly into your AI host context. You, the calling agent, perform semantic analysis. No Shorty analysis model, upload or Shorty credit charge. Sampled evidence is not a complete semantic analysis.
inspect_reference
ChatGPTAsk the browser to extract bounded local media evidence directly into your AI host context. You, the calling agent, perform semantic analysis. No Shorty analysis model, upload or Shorty credit charge. Sampled evidence is not a complete semantic analysis.
job
ChatGPTInspect or wait for an existing durable job. Never submits new work or reserves credits.
list_projects
ChatGPTList projects in the connected Shorty account. Does not create projects or start editing work.
list_voices
ChatGPTDiscover target voices for changing an EXISTING recording with ElevenLabs on fal. Returns stable targetVoiceId, vocal description, apparent age, presentation, demoAccent and suitable uses; compare these to the user brief, not names alone. Optional brief ranks voices; languageCode filters advertised model support, not an audition guarantee. The source performance and accent survive conversion, so a British demo does not remove an American accent. Free catalog lookup; no audio generation. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
lock_camera_to_subject
ChatGPTTrack one distinctive feature through a single continuous source shot and transform the footage frame by frame so that feature stays at its seed-frame position or the canvas center. The target must remain visible throughout the interval, which cannot cross a cut. Measured checkpoints and a sub-pixel tracker constrain drift. canvasWidth and canvasHeight define the output framing. fitCanvas uses the smallest constant zoom that hides exposed edges, limited by maxZoom. lockObjectSize compensates for subject scale changes; lockRotation compensates for tilt and is intended only for a rigid, textured feature, since a turning face or head can produce false rotation estimates. smoothing selects locked tracking or a smoother trajectory. Returns measured transforms, residual-motion/zoom/uncovered-frame QA, an inspection sheet and a source block for applying those transforms. Long shots use a lower internal tracking resolution to bound memory; there is no fixed frame-count cap. This operation moves the footage rather than attaching an overlay or stabilizing the whole scene. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
lookup_catalog
ChatGPTRead a capability's guide (its usage rules) before you plan with it. kind: grounding — coordinates for people, hands and objects, tracking routes, camera lock; planar — content inside a screen, sign or other flat surface; spatial-text — depth-space text that passes behind real foreground; masks — cutouts, mattes, background replacement, subject-only effects; speed — slow motion, speed curves, reverse playback; effects — @shorty/remotion-picture-effects (CatalogueVideo looks, reference looks, blob tracking) and @remotion/effects shaders; text — TextFx plates, editorial infographics, graphic loops, stickers, text treatments, curved or extruded 3D type. Pass ids (catalogue ids from the capability index) to get those entries' descriptions. Each guide comes back once per session; it also arrives by itself with the first result of the capability's own tool. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
measure_motion
ChatGPTMeasure non-semantic pixel displacement of an already identified rectangle over a bounded source interval. Returns numeric patch-motion measurements. Does not recognize or locate semantic entities. Uses immutable assetId in projectId. Read media_contract for worker limits and pricing. Starts a paid durable job; use job to inspect progress. Preserves source assets.
media_contract
ChatGPTRead the supported media processing parameter schemas, worker limits and deployment pricing. Does not start processing.
normalize_loudness
ChatGPTMeasure recorded audio using ITU-R BS.1770 integrated loudness and apply a constant gain toward a shared target, default -16 LUFS, with a -1.5 dBTP true-peak ceiling. targetLufs accepts -24 to -12; assetIds can select up to 12 clips in the same project. Returns a derived clip for each changed input with the picture stream, frame count and duration preserved. Inputs within 1 LU of the target and nearly silent inputs are skipped without a new asset. Loud transients can prevent reaching the target; the result reports that condition. This changes gain rather than removing noise or setting the final composition mix level, and it can also raise background noise. When noise reduction is requested for the same clip, normalize the cleaned result. Dialogue, music, ambience and deliberately quiet moments require an appropriate requested target rather than automatic uniform gain. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
open_shorty
ChatGPTOpen the complete Shorty editor inside ChatGPT: browse reference/effect templates, upload footage, review edits and revise selected objects. This only opens a view; it never starts paid work. Use the current shortySelection context for project/edit identifiers. The website uses its own Shorty sign-in; connect the same account for private context sharing. Studio submits requests to ChatGPT: you must inspect and author the composition yourself with authoring_contract and author_edit, then render its planRevisionId. For revisions read editor_session and preserve the existing source/media bindings. Never use backend autopilot or revise_edit for Studio requests.
optical_flow_slow_motion
ChatGPTCreate a silent video asset for a slowed interval using neural frame interpolation when the requested playbackRate is below the source clip's minPlaybackRate. Each source interval is limited to 8 seconds, the slowed result to 20 seconds, and an edit to four operations. playbackRate can reach 0.05 for footage at the composition frame rate. The result already plays at the requested speed and composition fps: output frame n corresponds to sourceStartSeconds + n * playbackRate / fps. trimBefore is measured in output frames; applying the same slow playbackRate again would slow it twice. Repeated source frames are removed before interpolation, and hard cuts are held rather than interpolated across. Fast occlusions, rotating text and flashing light can produce artifacts; returned contact sheets expose the result for inspection. Keep the original clip for recorded audio. Quality tiers are standard and premium, with different backend quotes; processing typically takes about a minute per operation. Credits are charged only through the current website pricing model. Returned asset IDs identify the source, interval, rate and engine. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
plan_speed_curve
ChatGPTPlan a speed ramp for one video clip and get the exact <SpeedRamp> JSX ('@shorty/remotion-speedramp'). Planning is instant and free; smoothSlowMotion also runs billed optical flow (minutes). Curves, laid along the source span and fitted to the slot: montage: Real time, a smooth surge to fast, then a sharp drop into a brief slow beat at 70% of the source, back to real time. hero: Real time, fast, a brief deep slow-motion dip on the moment at 50% of the source, fast again, settling back to real time — put the action peak at the middle. bullet: Fast, one deep slow-motion dip through the middle of the source (it plays long: bullet time), fast again. jumpCut: Slow, steady speed with one sudden fast skip through the middle of the source — a jump cut that keeps the motion continuous. flashIn: Enters at a 20x rush and settles to 0.5x slow motion — the incoming side of a ramp transition. flashOut: 0.5x slow motion that accelerates into a 20x exit — the outgoing side of a ramp transition. whipThrough: Enters fast, steadies through the middle of the source, leaves fast — a whole short shot between two whip cuts, the way speed-ramped house tours cut every shot. valley: The whole shot in about a second, laid along the slot so it flows: the first quarter eases (in log speed) from a rush at the cut down to minRate, the middle half holds minRate on the subject (optical flow; at most 0.5 s in a 1 s slot), the last quarter eases back up into the next cut. The floor is minRate; the peak is whatever reading the given source takes, up to maxRate — the longer the camera travel read, the faster the rush. The curve speed-ramped house tours put on every pass and generated shot, read from the start of the shot to its end. (These points only sketch it; speedCurve builds the valley along the slot.) hill: Slow, a smooth climb to a peak of speed in the middle of the source, a smooth descent back to slow — the J curve or hill: dreamy at both ends, a rush between them. landing: Fast approach that decelerates into slow motion for the last quarter of the source — land on a hit, pose, or reveal. launch: Starts in slow motion for the first quarter of the source and accelerates away — leave a pose or reveal at speed into the next cut. anchors: put exact source moments on exact output frames, slow on each anchor and fast between; hold 0..0.8 makes slow holds and quick whips. custom: your own points, fitted like a preset. Rates are multiples of real time. Slow motion always uses optical flow: smoothSlowMotion is on unless false and synthesizes any stretch slower than the clip's frames in this call, reusing an existing retime or charging like optical_flow_slow_motion (counts toward its per-edit limit). Check the retime's contact sheets; if fast occluding hands or blur warp, re-plan with smoothSlowMotion: false. The result reports rates, a rate profile, onScreen visibility and whether it is a clear ramp (fastest at least 10x slowest, needed when ramps are requested). If the brief gives ramp speeds and frame counts, plan exactly those. Plan each shot once, twice at most: re-plan only if refused or the slow stretch misses the action. Never ramp a clip whose speech you keep. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
prepare_local_assets
ChatGPTApp-only asset transfer for authenticated desktop Shorty. Returns short-lived project-bound URLs and SHA-256 metadata. It never sends, writes or renders composition source.
prepare_local_upload
ChatGPTApp-only upload ticket for a user-selected file and an existing project. Returns a structured PUT URL. Uploads publish media for backend processing; composition code and rendering stay local.
read_asset
ChatGPTRead an immutable project asset. Returns verified PNG/JPEG images, MP3 audio, or tracking JSON inline (up to 8 MiB). Use inspect for video contact sheets.
reduce_noise
ChatGPTReduce steady background noise in recorded speech while copying the picture stream and preserving video frames and duration. The whole source clip is processed, limited to 20 minutes and six operations per edit. Strength limits background attenuation to 6 dB for light, 12 dB for medium or 20 dB for strong. The operation is intended for audible speech with steady hiss, hum, fans, wind or similar noise; music, singing, intentional ambience, whispering, distant speech and changing crowds can be damaged or suppressed. A result with detected voice damage produces no new clip and reports that the original should be retained. Returns a derived clip with cleaned audio when processing succeeds. Noise reduction precedes loudness normalization; already-levelled input is refused. Processing is charged through the current backend website-credit quote, rather than a separate MCP price. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
remove_filler_words
ChatGPTRemove actual filler words using ElevenLabs Scribe v2, matching language MFA acoustic alignment and protected sample-accurate cuts. First call mode=analyze with the clip and its speech language code: it returns the fresh transcript plus candidate IDs and context. A provisioned MFA model and dictionary are required for that language; unsupported languages fail before upload. Decide which candidates are truly dispensable in that sentence; discourse markers are NOT always fillers. Candidates marked untranscribed-voicing flag sustained sound in a gap that ASR omitted; inspect that audio before selecting it, since it may be a missed real word. Then call mode=apply with its analysisId and selected candidateIds. Never supply timestamps or select a name, negation, meaningful response or required correction. Unsafe candidates are refused. The returned video contains original picture and recorded voice with approved fillers removed and word timings remapped. Replace the original at trimBefore=0 and rebuild timing/captions. For an unwanted opening, optionally set keepFromWordId to the first word of the clean take; earlier speech and silence are trimmed. Only do this when the user wants that opening removed. Internal retakes, all pauses and undetected fillers are not automatically removed; ambiguous speech stays. Up to 180 seconds per clip and four analyses per edit. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
render
ChatGPTRender an accepted immutable planRevisionId from author_edit at full scale. Reserves credits and creates a video artifact. A failed matching render may be retried automatically. Resume or collect the video with status.
retry_job
ChatGPTRetry an existing durable job when the user requests a retry. May reserve credits and repeat external-provider work for generation or analysis jobs.
search_stock
ChatGPTSearch free Pixabay stock for a video clip or photo to fill a beat the user's footage cannot show. Matches tags, so the query is 1–4 concrete English words for what is visible ("ocean waves aerial", "coffee pour closeup"), never a sentence or a mood. Returns numbered candidates (stockId, length, size, orientation, tags) and one image of their thumbnails; then call import_stock with the stockIds you will use. Clips in the canvas orientation come first. At most 12 searches per edit. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
segment_frame
ChatGPTRemove the background from one exact source-video frame at timestampSeconds. Returns a new transparent foreground PNG. Does not accept prompts or geometry controls. Uses immutable assetId in projectId. Read media_contract for worker limits and pricing. Starts a paid durable job; use job to inspect progress. Preserves source assets.
segment_image_objects
ChatGPTSAM 3.1 per-object masks for a supplied STILL IMAGE. Use for individual windows, lamps, sky, products or other selected regions; unlike segment_frame this accepts images and text/positive-negative pixel points/boxes. Returns separately addressable transparent PNG assets plus numbered source-overlay evidence. Optional fillLogicalAssetId takes pixels from a second already aligned image (e.g. night windows over day); inspect alignment first. outputMode=mask returns white RGB with true alpha for CSS maskImage, not an opaque grayscale PNG. Coordinates refer to the original source pixels. Returned object indices belong to this request, not stable identities across retries. Reaching maxMasks means detection may be incomplete. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
segment_video
ChatGPTRemove the background from an exact source-video interval while preserving foreground motion, up to 512 source frames. Returns a new transparent WebM whose frame zero maps to sourceStartSeconds. Uses immutable assetId in projectId. Read media_contract for worker limits and pricing. Starts a paid durable job; use job to inspect progress. Preserves source assets.
source_bind_backend
ChatGPTDownload and verify one immutable result of a requested backend tool into the local browser project. Checks asset ownership, byte size and SHA-256; this never uploads local source or media.
source_project
ChatGPTCreate or open a local browser project. Import originals in the widget; never upload them merely for analysis or rendering.
source_render
ChatGPTRender a saved browser revision to MP4 or a still using local WebCodecs. Requires a supported active widget and all local media. No server fallback. Inspect browser_status for actual completion.
source_render_status
ChatGPTRead a browser rendering acknowledgement. No render is complete until the browser reports completed.
status
ChatGPTOpen one durable edit run or authored render job in the render monitor: inspect it, wait for it, and deliver its finished video. Requires exactly one of editRunId or renderJobId; to find runs, inspect the project with the get_project tool instead. Status never starts or retries a job.
sync_takes_to_track
ChatGPTSync performance takes to the soundtrack: for a music video, lip-sync, rap or dance shot as several takes over the same song, find where in the track each take begins from the rhythm its camera audio picked up off the playback, and start the soundtrack where the synced takes do. Call it before writing any cut whenever the brief asks for sync (synced to the track, lip-sync, in time with the song, music video takes) and the takes carry their own recorded sound. It returns each take's offset (track second at the take's second 0) with a confidence, sets the soundtrack to start at the first moment synced takes cover (on a downbeat), and gives the beat grid again from that start. After it, place every synced take with trimBefore = round((soundtrackStart + compositionFrame/fps - offset) * fps), playbackRate 1 and muted, so mouth and music stay locked; the submit check holds you to that within 2 frames. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
tool_catalog
ChatGPTRead exact browser command and backend processor schemas, execution location, availability and configured pricing. Media evidence, agent analysis, composition source and rendering use the browser editor; requested backend transcription/processors/generation use account credits.
track_object
ChatGPTTrack one rigid moving object named by a literal prompt over an exact shot-bounded source interval, up to 512 frames. Returns motion keyframes and a source block for attaching composition layers. Use semantic grounding for deformable subjects. Uses immutable assetId in projectId. Read media_contract for worker limits and pricing. Starts a paid durable job; use job to inspect progress. Preserves source assets.
upload_for_processing
ChatGPTUpload exactly one selected local browser asset for requested backend transcription or another requested specialized backend tool. Uses the existing account and website credit model for the subsequent operation. Never upload merely to understand footage or render locally.
upscale_video
ChatGPTRestore or upscale one interval of a shot: Topaz Starlight, a generative video restorer, rebuilds every frame with real-looking detail, keeping the frame count, frame rate and motion. Factor 1 restores at the same size: footage that has the pixels but looks soft, blocky from compression or noisy gets back skin, eyes, hair, foliage and texture. A larger factor also adds pixels to a clip that is short of them. It returns a NEW clip that takes that interval's place, like a stabilized one. Call it only when the user asked for it in their own words: sharper, clearer or HD footage, better picture quality, a blurry, soft or pixelated shot fixed, a clip upscaled (in any language). Never on your own initiative, however soft a shot looks or however far a clip is over the 1.2 resolution budget: without the user's words, keep the shot as it is, frame it wider, or use another one. Pass those words, copied exactly from the brief or request, as userRequest. When the user asked, restore (factor 1) the shots they mean, or that the edit leans on, a face above all, that look soft or smeared; upscale a clip smaller than the canvas (each video asset reports fillMagnification; it is the default factor above 1.1) or a punch-in on one that cannot afford it (fillMagnification times the extra scale). Leave sharp, clean footage alone. The detail is generated: a soft face comes back convincingly, but a face or a word only a few dozen pixels tall is invented, and heavy motion blur stays. The result attaches frames before and after: if a face or a word changed, use the original instead. Limits: 0.5s to 10s per call, at most 640 output megapixels per call (width x height x frames: 10.3s of 1080x1920 at 30fps), factor up to 4, output long side at most 3840, 3 per edit. It takes minutes (up to a minute to start, then about 40 seconds per second of 1080p output) and costs $0.02 per second of output up to 1080p ($0.08 above, double above 30fps), so restore or upscale only the intervals the cut uses. Upscale last: stabilize first, never slow an upscaled clip, and never make masks, tracks or text plates from it. Runs the named backend operation on the connected account using authoritative website credit pricing. Uses immutable backend project/asset IDs; never authors or renders a composition. Poll job and use source_bind_backend for selected results.
write_source
ChatGPTSave a complete source revision in the browser. Same contract and revision protection as author_edit.