Streamline - DLSS
=======================

>The focus of this guide is on using Streamline to integrate DLSS into an application.  For more information about DLSS itself, please visit the [NVIDIA Developer DLSS Page](https://developer.nvidia.com/rtx/dlss)
>For information on user interface considerations when using the DLSS plugin, please see the ["RTX UI Developer Guidelines.pdf"](<RTX UI Developer Guidelines.pdf>) document included with this SDK.

Version 2.4.0
=======

### 1.0 INITIALIZE AND SHUTDOWN

Call `slInit` as early as possible (before any dxgi/d3d11/d3d12 APIs are invoked)

```cpp
#include <sl.h>
#include <sl_consts.h>
#include <sl_dlss.h>

sl::Preferences pref{};
pref.showConsole = true; // for debugging, set to false in production
pref.logLevel = sl::eLogLevelDefault;
pref.pathsToPlugins = {}; // change this if Streamline plugins are not located next to the executable
pref.numPathsToPlugins = 0; // change this if Streamline plugins are not located next to the executable
pref.pathToLogsAndData = {}; // change this to enable logging to a file
pref.logMessageCallback = myLogMessageCallback; // highly recommended to track warning/error messages in your callback
pref.applicationId = myId; // Provided by NVDA, required if using NGX components (DLSS 2/3)
pref.engineType = myEngine; // If using UE or Unity
pref.engineVersion = myEngineVersion; // Optional version
pref.projectId = myProjectId; // Optional project id
if(SL_FAILED(res, slInit(pref)))
{
    // Handle error, check the logs
    if(res == sl::Result::eErrorDriverOutOfDate) { /* inform user */}
    // and so on ...
}
```

For more details please see [preferences](ProgrammingGuide.md#221-preferences)

Call `slShutdown()` before destroying dxgi/d3d11/d3d12/vk instances, devices and other components in your engine.

```cpp
if(SL_FAILED(res, slShutdown()))
{
    // Handle error, check the logs
}
```

#### 1.1 SET THE CORRECT DEVICE

Once the main device is created call `slSetD3DDevice` or `slSetVulkanInfo`:

```cpp
if(SL_FAILED(res, slSetD3DDevice(nativeD3DDevice)))
{
    // Handle error, check the logs
}
```

### 2.0 CHECK IF DLSS IS SUPPORTED

As soon as SL is initialized, you can check if DLSS is available for the specific adapter you want to use:

```cpp
Microsoft::WRL::ComPtr<IDXGIFactory> factory;
if (SUCCEEDED(CreateDXGIFactory(__uuidof(IDXGIFactory), (void**)&factory)))
{
    Microsoft::WRL::ComPtr<IDXGIAdapter> adapter{};
    uint32_t i = 0;
    while (factory->EnumAdapters(i, &adapter) != DXGI_ERROR_NOT_FOUND)
    {
        DXGI_ADAPTER_DESC desc{};
        if (SUCCEEDED(adapter->GetDesc(&desc)))
        {
            sl::AdapterInfo adapterInfo{};
            adapterInfo.deviceLUID = (uint8_t*)&desc.AdapterLuid;
            adapterInfo.deviceLUIDSizeInBytes = sizeof(LUID);
            if (SL_FAILED(result, slIsFeatureSupported(sl::kFeatureDLSS, adapterInfo)))
            {
                // Requested feature is not supported on the system, fallback to the default method
                switch (result)
                {
                    case sl::Result::eErrorOSOutOfDate:         // inform user to update OS
                    case sl::Result::eErrorDriverOutOfDate:     // inform user to update driver
                    case sl::Result::eErrorNoSupportedAdapter:  // cannot use this adapter (older or non-NVDA GPU etc)
                    // and so on ...
                };
            }
            else
            {
                // Feature is supported on this adapter!
            }
        }
        i++;
    }
}
```

### 3.0 CHECK DLSS SETTINGS AND SETUP VIEWPORT RENDERING SIZE

Next, we need to find out the rendering resolution and the optimal sharpness level based on DLSS settings:

```cpp

// Using helpers from sl_dlss.h

sl::DLSSOptimalSettings dlssSettings;
sl::DLSSOptions dlssOptions;
// These are populated based on user selection in the UI
dlssOptions.mode = myUI->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
dlssOptions.outputWidth = myUI->getOutputWidth();    // e.g 1920;
dlssOptions.outputHeight = myUI->getOutputHeight(); // e.g. 1080;
// Now let's check what should our rendering resolution be
if(SL_FAILED(result, slDLSSGetOptimalSettings(dlssOptions, dlssSettings))
{
    // Handle error here
}
// Setup rendering based on the provided values in the sl::DLSSSettings structure
myViewport->setSize(dlssSettings.renderWidth, dlssSettings.renderHeight);
```

Note that the structure `sl::DLSSOptimalSettings` will upon return from `slDLSSGetOptimalSettings` contain information pertinent to DLSS dynamic resolution min and max source image sizes (if dynamic resolution is supported).

### 4.0 TAG ALL REQUIRED RESOURCES

DLSS requires depth, motion vectors, render-res input color and final-res output color buffers.

```cpp

// IMPORTANT: Make sure to mark resources which can be deleted or reused for other purposes within a frame as volatile

// Prepare resources (assuming d3d11/d3d12 integration so leaving Vulkan view and device memory as null pointers)
sl::Resource colorIn = {sl::ResourceType::Tex2d, myTAAUInput, nullptr, nullptr, nullptr};
sl::Resource colorOut = {sl::ResourceType::Tex2d, myTAAUOutput, nullptr, nullptr, nullptr};
sl::Resource depth = {sl::ResourceType::Tex2d, myDepthBuffer, nullptr, nullptr, nullptr};
sl::Resource mvec = {sl::ResourceType::Tex2d, myMotionVectorsBuffer, nullptr, nullptr, nullptr};
sl::Resource exposure = {sl::ResourceType::Tex2d, myExposureBuffer, nullptr, nullptr, nullptr};

sl::ResourceTag colorInTag = sl::ResourceTag {&colorIn, sl::kBufferTypeScalingInputColor, sl::ResourceLifecycle::eOnlyValidNow, &myExtent };
sl::ResourceTag colorOutTag = sl::ResourceTag {&colorOut, sl::kBufferTypeScalingOutputColor, sl::ResourceLifecycle::eOnlyValidNow, &myExtent };
sl::ResourceTag depthTag = sl::ResourceTag {&depth, sl::kBufferTypeDepth, sl::ResourceLifecycle::eValidUntilPresent, &fullExtent };
sl::ResourceTag mvecTag = sl::ResourceTag {&mvec, sl::kBufferTypeMvec, sl::ResourceLifecycle::eOnlyValidNow, &fullExtent };
sl::ResourceTag exposureTag = sl::ResourceTag {&exposure, sl::kBufferTypeExposure, sl::ResourceLifecycle::eOnlyValidNow, &my1x1Extent};

// Tag in group
sl::Resource inputs[] = {colorInTag, colorOutTag, depthTag, mvecTag};
slSetTag(viewport, inputs, _countof(inputs), cmdList);
```

> **NOTE:**
> If dynamic resolution is used then please specify the extent for each tagged resource. Please note that SL **manages resource states so there is no need to transition tagged resources**.

> **NOTE:**
> If `sl::kBufferTypeExposure` is NOT provided or `dlssOptions.useAutoExposure` is set to be true then DLSS will be in auto-exposure mode (`NVSDK_NGX_DLSS_Feature_Flags_AutoExposure` will be set automatically)

### 5.0 PROVIDE DLSS OPTIONS

DLSS options must be set so that the DLSS plugin can track any changes made by the user:

```cpp
sl::DLSSOptions dlssOptions = {};
// Set preferred Render Presets per Perf Quality Mode. These are typically set one time
// and established while evaluating DLSS SR Image Quality for your Application.
// It will be set to DSSPreset::eDefault if unspecified.
// Please Refer to section 3.12 of the DLSS Programming Guide for details.
dlssOptions.dlaaPreset = sl::DLSSPreset::ePresetA;
dlssOptions.qualityPreset = sl::DLSSPreset::ePresetD;
dlssOptions.balancedPreset = sl::DLSSPreset::ePresetD;
dlssOptions.performancePreset = sl::DLSSPreset::ePresetD;
dlssOptions.ultraPerformancePreset = sl::DLSSPreset::ePresetA;
// These are populated based on user selection in the UI
dlssOptions.mode = myUI->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
dlssOptions.outputWidth = myUI->getOutputWidth();    // e.g 1920;
dlssOptions.outputHeight = myUI->getOutputHeight(); // e.g. 1080;
dlssOptions.sharpness = dlssSettings.sharpness; // optimal sharpness
dlssOptions.colorBuffersHDR = sl::Boolean::eTrue; // assuming HDR pipeline
dlssOptions.useAutoExposure = sl::Boolean::eFalse; // autoexposure is not to be used if a proper exposure texture is available
dlssOptions.alphaUpscalingEnabled = sl::Boolean::eFalse; // experimental alpha upscaling, enable to upscale alpha channel of color texture
if(SL_FAILED(result, slDLSSSetOptions(viewport, dlssOptions)))
{
    // Handle error here, check the logs
}
```

> **NOTE:**
> To turn off DLSS set `sl::DLSSOptions.mode` to `sl::DLSSMode::eOff`, note that this does NOT release any resources, for that please use `slFreeResources`

> **NOTE:**
> Set the DLSSOptions.useAutoExposure boolean to be true only if you want DLSS to be in in auto-exposure mode. Also, it is strongly advised to provide exposure if a proper exposure texture is available. 

> **NOTE:**
> Alpha upscaling (`DLSSOptions::alphaUpscalingEnabled`) is experimental, and will impact performace. This feature should be used only if the alpha channel of the color texture needs to be upscaled (if `eFalse`, only RGB channels will be upscaled).

### 6.0 PROVIDE COMMON CONSTANTS

Various per frame camera related constants are required by all Streamline features and must be provided ***if any SL feature is active and as early in the frame as possible***. Please keep in mind the following: 

* All SL matrices are row-major and should not contain any jitter offsets
* If motion vector values in your buffer are in {-1,1} range then motion vector scale factor in common constants should be {1,1}
* If motion vector values in your buffer are NOT in {-1,1} range then motion vector scale factor in common constants must be adjusted so that values end up in {-1,1} range

```cpp
sl::Constants consts = {};
// Set motion vector scaling based on your setup
consts.mvecScale = {1,1}; // Values in eMotionVectors are in [-1,1] range
consts.mvecScale = {1.0f / renderWidth,1.0f / renderHeight}; // Values in eMotionVectors are in pixel space
consts.mvecScale = myCustomScaling; // Custom scaling to ensure values end up in [-1,1] range
// Set all other constants here
if(SL_FAILED(result, slSetConstants(consts, *frameToken, myViewport))) // constants are changing per frame so frame index is required
{
    // Handle error, check logs
}
```
For more details please see [common constants](ProgrammingGuide.md#251-common-constants)

### 7.0 ADD DLSS TO THE RENDERING PIPELINE

On your rendering thread, call `slEvaluateFeature` at the appropriate location where up-scaling is happening. Please note that when using `slSetTag`, `slSetConstants` and `slDLSSSetOptions` the `frameToken` and `myViewport` used in `slEvaluateFeature` **must match across all API calls**.

```cpp
// Make sure DLSS is available and user selected this option in the UI
if(useDLSS) 
{
    // NOTE: We can provide all inputs here or separately using slSetTag, slSetConstants or slDLSSSetOptions

    // Inform SL that DLSS should be injected at this point for the specific viewport
    const sl::BaseStructure* inputs[] = {&myViewport};
    if(SL_FAILED(result, slEvaluateFeature(sl::kFeatureDLSS, *frameToken, inputs, _countof(inputs), myCmdList)))
    {
        // Handle error
    }
    else
    {
        // IMPORTANT: Host is responsible for restoring state on the command list used
        restoreState(myCmdList);
    }
}
else
{
    // Default up-scaling pass like for example TAAU goes here
}
```

> **IMPORTANT:**
> Plase note that **host is responsible for restoring the command buffer(list) state** after calling `slEvaluate`. For more details on which states are affected please see [restore pipeline section](./ProgrammingGuideManualHooking.md#80-restoring-command-listbuffer-state)

### 8.0 MULTIPLE VIEWPORTS

Here is a code snippet showing one way of handling two viewports with explicit resource allocation and de-allocation:

```cpp
// Viewport1
{
    // We need to setup our constants first so sl.dlss plugin has enough information
    sl::DLSSOptions dlssOptions = {};
    dlssOptions.mode = viewport1->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
    dlssOptions.outputWidth = viewport1->getOutputWidth();    // e.g 1920;
    dlssOptions.outputHeight = viewport1->getOutputHeight(); // e.g. 1080;
    // Note that we are passing viewport id 1
    slDLSSSetOptions(viewport1->id, dlssOptions);
    
    // Set our tags, note that we are passing viewport id
    setTag(viewport1->id, &tags2, numTags2);
    // and so on ...

    // Now we can allocate our feature explicitly, again passing viewport id
    slAllocateResources(sl::kFeatureDLSS, viewport1->id);

    // Evaluate DLSS on viewport1, again passing viewport id so we can map tags, constants correctly
    //
    // NOTE: If slAllocateResources is not called DLSS resources would be initialized at this point
    slEvaluateFeature(sl::kFeatureDLSS, myFrameIndex, viewport1->id, nullptr, 0, myCmdList);

    // Assuming the above evaluate call is still pending on the CL, make sure to flush it before releasing resources
    flush(myCmdList);

    // When we no longer need this viewport
    slFreeResources(sl::kFeatureDLSS, viewport1->id);
}

// Viewport2
{
    // We need to setup our constants first so sl.dlss plugin has enough information
    sl::DLSSOptions dlssOptions = {};
    dlssOptions.mode = viewport2->getDLSSMode(); // e.g. sl::eDLSSModeBalanced;
    dlssOptions.outputWidth = viewport2->getOutputWidth();    // e.g 1920;
    dlssOptions.outputHeight = viewport2->getOutputHeight(); // e.g. 1080;
    // Note that we are passing viewport id 2
    slDLSSSetOptions(viewport2->id, dlssOptions);

    // Set our tags, note that we are passing viewport id
    setTag(viewport2->id, &tags2, numTags2);
    // and so on ...

    // Now we can allocate our feature explicitly, again passing viewport id
    slAllocateResources(sl::kFeatureDLSS, viewport2->id);
    
    // Evaluate DLSS on viewport2, again passing viewport id so we can map tags, constants correctly
    //
    // NOTE: If slAllocateResources is not called DLSS resources would be initialized at this point
    slEvaluateFeature(sl::kFeatureDLSS, myFrameIndex, viewport2->id, nullptr, 0, myCmdList);

    // Assuming the above evaluate call is still pending on the CL, make sure to flush it before releasing resources
    flush(myCmdList);

    // When we no longer need this viewport
    slFreeResources(sl::kFeatureDLSS, viewport2->id);
}

```

### 9.0 CHECK STATE AND VRAM USAGE

To obtain current state for a given viewport the following API can be used:

```cpp
sl::DLSSState dlssState{};
if(SL_FAILED(result, slDLSSGetState(viewport, dlssState))
{
    // Handle error here
}
// Check how much memory DLSS is using for this viewport
dlssState.estimatedVRAMUsageInBytes
```

### 10.0 TROUBLESHOOTING

If the DLSS output does not look right please check the following:

* If your motion vectors are in pixel space then scaling factors `sl::Constants::mvecScale` should be {1 / render width, 1 / render height}
* If your motion vectors are in normalized -1,1 space then scaling factors `sl::Constants::mvecScale` should be {1, 1}
* Make sure that jitter offset values are in pixel space
* `NVSDK_NGX_Parameter_FreeMemOnRelease` is replaced with `slFreeResources`
* `NVSDK_NGX_DLSS_Feature_Flags_MVLowRes` is handled automatically based on tagged motion vector buffer's size and extent.