Autoshot Unreal Round Trip

Introduction

In this tutorial, we'll go through the process of creating an Unreal Jetset Autoshot round trip using a free Fantasy Castle asset. We'll cover everything from setting up a new project in Unreal Engine to exporting assets and shooting with a green screen, ultimately bringing everything back into Unreal for rendering and compositing. This guide is perfect for anyone looking to integrate live-action footage with 3D environments seamlessly.

Chapter 1: Creating Jetset Project

1. Open Unreal Engine and navigate to the main menu.
2. Create a new project:
   • Name it "Fantasy Castle."
   • Change the project prefix to "FantasyCastle."
3. Set this project as your current project.

Chapter 2: Installing Autoshot Tools and Autoshot

1. Go to the Lightcraft downloads page (link provided in the video).
2. Download both Autoshot and Autoshot tools.
3. Install both tools in any order:
   • Autoshot tools contain necessary DLLs and AI model weights.
   • If prompted with a warning during installation, select "OK."

Chapter 3: Create Matching Autoshot Project Folder

1. Launch Autoshot and create a new project folder:
   • Navigate to your desired drive (e.g., D drive).
   • Create a new folder named "Fantasy Castle."
2. Open the folder, which will automatically create three subfolders: assets, footage, and sequences.

Chapter 4: Organizing Project Folder

• Use the created subfolders as follows:
  • Assets: For storing USD and USDZ files.
  • Footage: For synchronized takes from Jetset.
  • Sequences: For processed sequences.
• Copy the assets folder path for later use.

Chapter 5: Setting Up NVIDIA Omniverse

1. Download the NVIDIA Omniverse Launcher from the provided link.
2. Install the launcher and create an NVIDIA account if necessary.
3. Open the Omniverse Launcher and go to the Exchange section to install:
   • USDView: For viewing USD files.
   • Unreal 5.3 Connector: Install the connector for Unreal Engine 5.3.

Chapter 6: Unreal Plugin Setup

1. In Unreal, navigate to the Plugins menu.
2. Ensure the following plugins are enabled:
   • NVIDIA Omniverse
   • USD Importer/Exporter
   • Image Plate
   • Composure
   • Python Editor and Sequencer scripting
   • Movie Render Queue and additional render passes

Chapter 7: Installing the Castle Scene

1. Open the Epic Games Launcher.
2. Navigate to the Unreal Marketplace and search for the free "Infinity Blade Castle" asset.
3. Add the asset to your current project.

Chapter 8: Adding Scene Locators

1. Create a new folder in Unreal named "Scene Locators."
2. Add empty actors as scene locators:
   • Name each locator using the prefix "sceneloc" followed by a descriptive name (e.g., "Castle Approach").
3. Ensure locators are positioned accurately on the ground.

Chapter 9: Export Layer

1. Open the Layers window in Unreal.
2. Select all desired objects for export.
3. Right-click and add the selected actors to a new layer named "USD Export."

Chapter 10: USD Export Directory

1. Copy the project folder path from Autoshot.
2. Back in Unreal, export selected items:
   • Go to File > Export Selected > USD.
   • Navigate to the assets folder and create a new folder named "USD" to save the export.

Chapter 11: USD Export Settings

• Choose export options:
  • Uncheck "MDL" and "Include Materials" for faster export.
  • Set the export axis to Y-up.
  • Click OK to export.

Chapter 12: Inspecting Export with USDView

1. Launch USDView from the Omniverse Launcher.
2. Open the exported USD file to verify successful export and view the contents.

Chapter 13: Fixing Export Issues

• If you missed exporting any items, select them in Unreal and add them to the layer, then repeat the export process.

Chapter 14: Converting from USD to USDZ

1. In Autoshot, navigate to the Models tab.
2. Set the USD file path to the exported USD file.
3. Create a new folder named "USDZ" for processed assets.
4. Pick the USD file and click "Make USDZ" to process and downsample textures.

Chapter 15: Verifying Export and Pushing to Jetset

1. Push the generated USDZ file to Jetset, ensuring a successful transfer.
2. Optionally, use a wired connection for larger models to improve transfer speed.

Chapter 16: Model Setup in Jetset

1. Open Jetset and select your Fantasy Castle project.
2. Set the live action tracking origin using the scene locator.
3. Configure the green screen settings, including garbage matting.

Chapter 17: Recording Takes

• Ensure optimal performance before shooting:
  • Check tracking and GPU usage indicators.
• Start recording takes with the green screen setup.

Chapter 18: Reviewing Takes

• After recording, review the takes for any dropped frames and assess tracking quality.

Chapter 19: Picking Sync Date

1. Sync the recorded takes with the appropriate project folder.
2. Select the desired shooting date for synchronization.

Chapter 20: Viewing Takes in Resolve

1. Import the synchronized footage into DaVinci Resolve.
2. Create a new project and set the frame rate to match your footage.

Chapter 21: Sorting Files with Smart Bins

• Create smart bins in Resolve to organize and filter your clips for easier access.

Chapter 22: Setting In and Out Points

• Use the timeline to set in and out points around your desired segment for editing.

Chapter 23: Finding In and Out Frames

• Retrieve in-frame and out-frame numbers for processing in Autoshot.

Chapter 24: Take Selection in Autoshot

1. Select the relevant take in Autoshot for processing.
2. Review the metadata for the selected take and adjust settings as necessary.

Chapter 25: Setting Run Values and In/Out Frames

• Configure your processing settings, including output format and color space.

Chapter 26: Creating Unreal Level Sequence

1. Paste the command from Autoshot into Unreal's command console to create a level sequence.
2. Open the generated sequence to verify camera placement and animation.

Chapter 27: Level Sequence Organization

• Review the level sequence structure, including the attached camera and image plate.

Chapter 28: Green Screen Material Keyer

1. Apply the material keyer to the image plate to remove the green screen.
2. Adjust material properties to improve keying results.

Chapter 29: Rendering Background EXR Frames

• Set up and render EXR frames from Unreal for compositing in Resolve.

Chapter 30: Making Sequences Bin in Resolve

1. Create a new bin in Resolve to store image sequences and rendered frames.

Chapter 31: Dragging in Sequences

• Import the rendered sequences into Resolve for compositing.

Chapter 32: Creating Fusion Clip

1. Create a Fusion clip to combine live action and CG elements.
2. Use viewers to assess the composite.

Chapter 33: Ultra Keyer Setup

• Utilize the Ultra Keyer tool for effective green screen removal.

Chapter 34: Planar Tracker as Garbage Matte

1. Use a planar tracker for garbage matting to isolate the subject.
2. Ensure the mask tracks effectively with the subject.

Chapter 35: Blurring Matte Transition

• Add blurring to the matte edges for a smoother composite.

Chapter 36: Rendering in Place for Timeline Playback

• Render the composite to ensure smooth playback in the timeline.

Conclusion

In this tutorial, we covered the complete process of integrating live-action footage with a 3D environment using Unreal Engine and Jetset. From project creation to final compositing in Resolve, each step is crucial for achieving high-quality results. By following these steps, you can effectively bring your virtual and real worlds together, opening the door to endless creative possibilities. Consider experimenting with different scenes and settings to further enhance your skills in this area.