Using SwapChainPanelRenderer to improve real-time rendering in Lumia Imaging SDK 3

Using SwapChainPanelRenderer to improve real-time rendering in Lumia Imaging SDK 3

In my previous blog post about PropertyDescriptions in Lumia Imaging SDK 3 I wrote some code for rendering effect results in real-time that wasn't really that good and resulted in performance issues. To be fair, the blog post was about using PropertyDescriptions so rendering was used only for a more complete demo, but as I was reading the documentation I noticed a couple of ways to improve the code I wrote.

In that code, whenever the slider was moved to change the strength of the blur effect, I would create a new WriteableBitmapRenderer to render the result.

using (var renderer = new WriteableBitmapRenderer(this.viewModel.blur, this.resultBitmap))  
{
    await renderer.RenderAsync();
}

This is bad resource management - creating a new renderer has a performance cost so it should actually be kept alive and only disposed when it's no longer needed.

Doing some performance analysis proved that this code was bad and what happened when I loaded an image (3072x1728 pixels) and moved the blur effect slider around.

Awful memory usage when recreating renderers

Yikes! There are spikes going up to 503MB and I was being gentle with the slider because otherwise the app was literally crashing.

Luckily, there's a smarter way to write this code.

Reusing WriteableBitmapRenderer

Yeah, just by reusing the WriteableBitmapRenderer I was able to bring down memory usage to somewhere around 200MB. I used a single local WriteableBitmapRenderer.

if (this.renderer == null)  
{
    this.renderer = new WriteableBitmapRenderer(this.viewModel.blur, this.resultBitmap);
}

await this.renderer.RenderAsync();  

This commit has all the changes that were needed for this. And guess what - memory usage was down.

Better memory usage with a single, reusable renderer

However, using WriteableBitmapRenderer and WriteableBitmaps should be avoided as much as possible. Switching to SwapChainPanelRenderer will improve the performance drastically.

SwapChainPanelRenderer

Instead of using the XAML Image object to render the image, I added a SwapChainPanel.

...
    <RowDefinition Height="Auto" />
</Grid.RowDefinitions>  
<SwapChainPanel x:Name="ImageSwapChainPanel" />  
<RelativePanel Grid.Row="1" Margin="24">  
...

After SwapChainPanel gets loaded, local SwapChainPanelRenderer is initialized. Another event handler is used to react to SwapChainPanel size changes to re-render the image in proper size - this is also very cool now that we have resizable windows in UWP.

private SwapChainPanelRenderer renderer;  
private EffectViewModel viewModel;

public MainPage()  
{
    this.InitializeComponent();

    this.viewModel = new EffectViewModel();
    this.ImageSwapChainPanel.Loaded += ImageSwapChainPanel_Loaded;
}

When the ImageSwapChainPanel is loaded, renderer is created if null and SizeChanged event handler attached.

this.renderer = new SwapChainPanelRenderer(this.viewModel.blur, this.ImageSwapChainPanel);

this.ImageSwapChainPanel.SizeChanged += async (s, args) =>  
{
    await this.renderer.RenderAsync();
};

Once the file gets opened, the image gets rendered.

IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.Read);  
this.viewModel.blur.Source = new RandomAccessStreamImageSource(fileStream);  
await this.renderer.RenderAsync();  

And when the slider is moved, the blur effect kernel value gets updated and image rendered again.

this.viewModel.blur.KernelSize = (int)e.NewValue;  
await this.renderer.RenderAsync();  

This results in quite a performance improvement!

Using SwapChainPanelRenderer to drastically improve performance

Further performance improvements can be done by reducing the size of the SwapChainPanel. Do you need the full-size real-time effect rendering? If not, just cut down the size of the panel.

Conclusion

Using WriteableBitmapRenderers and WriteableBitmaps should be avoided as much as possible and renderers should not really be recreated all the time for real-time rendering as that causes serious performance issues. Instead, renderers should be kept alive and reused by the app if needed again. When it comes to interactive/real-time rendering, using a SwapChainPanelRenderer is recommended, and it should live as long as SwapChainPanel lives.

Complete project available on GitHub.

Igor Ralic

igor ralic

View Comments
Microsoft Certified Solutions Developer: Windows Store Apps in C#