Support Cinematic mode videos in your app

Written by Rony Fadel

Description: Discover how the Cinematic Camera API helps your app work with Cinematic mode videos captured in the Camera app. We'll share the fundamentals — including Decision layers — that make up Cinematic mode video, show you how to access and update Decisions in your app, and help you save and load those changes.

What is cinematic mode?

  • a "tiny film crew right in your pocket".
  • brings you a camera with shallow depth of field and natural focus falloff.
  • a director, who directs the attention and narrative by changing the focus.
  • a focus puller who anticipates keyframes ahead of time and creates smooth transitions between focus points.
  • It's captured in the Camera app on iPhone 13 and newer.

Cinematic API

Available on macOS 14, iOS 17, iPadOS 17 and tvOS 17.


  • Two assets: Rendered asset and Cinematic mode asset
  • Rendered asset can be shared and played as a regular Quicktime movie.
  • Cinematic mode asset: has all the information needed to create the Rendered asset. Allow non-destructive post-capture edits.

Cinematic mode asset

  • has multiple tracks
    • the original QuickTime movie: can be HDR/SDR, 1080p @ 30 fps on iPhone 13 and up to 4k @ 24/25/30 fps on iPhone 14.
    • Disparity used for focus and rendering the shallow depth of field.
    • Metadata:
      • rendering attributes which hold focus disparity (chosen by the cinematic engine) and aperture as an f number (chosen by the user)
      • Cinematic script which holds all the automatic scene detections, and the focus decisions deciding which detection to keep in focus.


  • To pick a Cinematic mode asset, use PhotosPicker and filter for .cinematicVideos.
  • Set the request options to get the .original version.
  • Can be played back as a regular movie using AVPlayer and AVPlayerItem.
  • To exploit to the max, we'll need to add a custom video compositor.
  • Create a session usingCNRenderingSession, and an AVMutableComposition.
  • In the custom compositor you can set the f number by setting a fNumber on the frameAttributes.


  • each cinematic frame holds all detections for a given point in time.
  • Base focus decisions are decided by the system. They can be overriden by a Weak user decision (overriden by the following Base decision), or a Strong user decision that will keep focus on a subject until the next user decision.
  • to draw detection boxes, you can get the current script frame using cinematicScript.frame(at: scriptFrameTime, tolerance: tolerance).
  • To add a new decision, create a CNDecision and add it to the cinematicScript.
  • The cinematicScript changes have a dataRepresentation property that can be written to disk.

Missing anything? Corrections? Contributions are welcome 😃

Written by

Rony Fadel

Rony Fadel

Former Apple engineer turned indie developer. Currently interested in working remotely, on shorter term (<6 months) macOS/iOS/ReactJS projects.