WWDC Notes

Introducing ARKit 3

Show Apple's description.
ARKit is the groundbreaking augmented reality (AR) platform for iOS that can transform how people connect with the world around them. Explore the state-of-the-art capabilities of ARKit 3 and discover the innovative foundation it provides for RealityKit. Learn how ARKit makes AR even more immersive through understanding of body position and movement for motion capture and people occlusion. Check out additions for multiple face tracking, collaborative session building, a coaching UI for on-boarding, and much more.

New Features in ARKit 3

  • Visual Coherence
  • Positional Tracking
  • Simultaneous Front and Back Camera
  • Record and Replay of Sequences
  • More Robust 3D Object Detection
  • Multiple-face Tracking
  • HDR Environment Textures
  • Faster Reference Image Loading
  • Motion Capture
  • Detect up to 100 Images
  • Face Tracking Enhancements
  • People Occlusion
  • Raycasting
  • Collaborative Session
  • ML Based Plane Detection
  • New Plane Classes
  • RealityKit Integration
  • AR QuickLook Additions
  • Auto-detect Image Size
  • AR Coaching UI

What’s New

People Occlusion

  • Available on A12 and later
  • Two ways:
    • people fragmentation (AR experience will always be occluded by people, regardless of the distance of AR objects and people)
    • people fragmentation + depth (ARKit estimate the people distance and occluded objects/people based on that)

Motion Capture

  • Tracks Human Body
  • Two ways: 2D and 3D

Collaborative Session

  • Multiple devices can be used to merge POI together and create a big AR world
  • Uses the MultipeerConnectivity framework for sharing AR World data or other wireless way to do so.

AR Coaching UI

  • To be used when telling the user to move the phone in the horizontal/vertical plane to start the AR experience
  • Automatically hides/updates itself based on the AR state

New Plane Classes

  • Wall floor ceiling table seat door window (last two are new)

Visual Coherence

  • Motion blur
  • Bokeh

Missing anything? Corrections? Contributions are welcome 😃

Related

Written by

zntfdr

Federico Zanetello

Federico Zanetello is an iOS Engineer with strong passion for Swift, minimalism, and design. When he’s not busy automating things, he can be found writing at fivestars.blog and/or playing with the latest shiny toys.