Charles Petzold: August DirectX Factor Converted to Windows 8.1

August DirectX Factor Converted to Windows 8.1

August 5, 2013
Roscoe, N.Y.

I hope Windows 8 developers have been enjoying my DirectX Factor column for MSDN Magazine. This column has been one of the most challenging jobs I’ve ever had. DirectX is big and complex, and very often I feel like I’m wrestling an enormous multi-headed monster in a futile attempt to slice out a little meat that I can assemble into a coherent 2000-word sandwich suitable for public consumption.

The first six installments of DirectX Factor were about using XAudio2 to generate sound, culminating with a software simulation of a classic analog electronic music synthesizer. I would have liked to explore sound much longer, but the reader response to those columns demanded I move on to something else.

Consequently, beginning with my latest DirectX Factor column just published in the August issue of MSDN Magazine, “Finger Painting with Direct2D Geometries”, I’ll be focusing on Direct2D and DirectWrite, and eventually the areas where they overlap (for example, converting text characters to geometries). I’m also planning on exploring the areas where Direct2D begins to make incursions into topics normally associated with Direct3D, such as shader processing. I have a long-term "arc" in mind for these columns, but like a season of Breaking Bad the subject matter may seem to wander for a while before it all comes together in an stunning dramatic climax.

If there is reader interest, I’d love to continue the DirectX Factor column into Direct3D, and particularly how 3D can be used in more mainstream (non-game) applications.

My deadline for that August column was in May, so of course the code in that column is for Windows 8 rather than Windows 8.1. Visual Studio 2013 Preview includes a facility to convert a Windows 8 project to Windows 8.1, but the new project remains pretty much the same and doesn't use any new features in Windows 8.1. As I began exploring Windows 8.1, there seemed to be several new features that directly affected the programs in the August column, so I decided to do an extensive rewrite.

You can download the Windows 8.1 version of BasicFingerPaint. I decided not to perform a similar conversion on the GradientFingerPaint because most of the code is quite similar.

This converted program also incorporates some recommendations made by James McNellis of Microsoft, who graciously reviewed the earlier code and had lots of helpful suggestions about more efficient ways to use std::vector and std::map, and avoid making numerous copies of the objects stored in these collections. Like many people, I've been away from C++ for a number of years, and I am very thankful for suggestions such as these.

Here are the major new features in Windows 8.1 that affected this conversion:

The New Template

I discussed the new DirectX project templates in Visual Studio 2013 Preview in a recent blog entry.

When I created a new project named BasicFingerPaint based on the new “DirectX App (XAML)” template, Visual Studio 2013 Preview created (among other files) classes named DirectXPage and BasicFingerPaintMain. For the rendering, I decided to use a naming convention consistent with BasicFingerPaintMain and went with BasicFingerPaintRenderer.

DirectXPage handles all the Windows Runtime input — from pointer devices, from the application bar, from other controls, and from the window — and BasicFingerPaintRenderer handles all the DirectX output. BasicFingerPaintMain functions as an intermediary between those two classes, much like the View Model in classic MVVM architecture.

If you like to layer program responsibilities very strictly, it’s possible for DirectXPage to know nothing about DirectX (despite its name), and for the Renderer class to know nothing about the Windows Runtime. (Actually, that’s not entirely possible because the GetOutputBounds method in DeviceResources that the Renderer often needs to call returns a Windows::Foundation::Size object.) In such an architecture, the Main class can perform conversions between Windows Runtime types and DirectX types as the information is passing between DirectXPage and the Renderer classes.

The earlier version of BasicFingerPaint defined two types of objects to store strokes — a plain old C++ structure named StrokeInfo, and a ref class named StrokeInformation, the idea being that StrokeInfo stores information using std::vector and DirectX types (including the ID2D1PathGeometry object used to render strokes) while StrokeInformation stores information using Vector and other Windows Runtime types for saving and loading finger-painted drawings. I kept that distinction in the new program, letting BasicFingerPaintMain perform the conversions between the two structures. This conversion only takes place when drawings are saved or loaded.

However, I think if I were to do this all over again, I’d use only one structure for storing stroke information. I’d use std::vector for the coordinate points, DirectX types for the points and colors, and I’d make conversions between Windows Runtime types and DirectX types in the constructors and other methods of this structure.

The Application Bar

I suppose we might have guessed that application bar buttons were simply too fundamental to the Windows 8 user interface to be relegated to a control template defined on the application level, particularly when that template had a bug that made it unusable for toggles.

With Windows 8.1, new AppBarButton and AppBarToggleButton classes are available that derive from Button and ToggleButton. The bulk of the StandardStyles.xaml file generated by Visual Studio 2012 project templates for Windows 8.0 supported application bar buttons, and that file is no longer part of the Visual Studio 2013 projects for Windows 8.1.

AppBarButton and AppBarToggleButton are simple but exceptionally versatile. They have a Label property of type String for the text label that appears under the button; an IsCompact property that suppresses the label and incorporates smaller padding (for example, for a series of media player transport buttons); and an Icon property of type IconElement for the image that appears inside the circle.

IconElement is non-instantiable but four classes derive from it:

BitmapIcon, which has a UriSource property to reference a bitmap file.

PathIcon, which has a Data property of type Geometry for specifying a vector drawing.

FontIcon, which lets you specify a font family (and font properties) along with a character code.

SymbolIcon, which has a Symbol property of type Symbol, a 190-member enumeration.

This is certainly versatile! All that's missing is an ElementIcon that lets you put a whole UIElement-based visual tree in that tiny circle. (Wouldn't you really like application bar buttons that incorporated animations or an Image element with a SurfaceImageSource to render a 3D rotating cube?)

The SymbolIcon is probably what you’ll use to be most consistent with the application bar styles in StandardStyles.xaml, and it's the easiest to use because you can just set the Icon property of AppBarButton to a Symbol enumeration member, such as Save, Clear, Undo, Mail, ZoomIn, and many more. The numeric values of these members are character codes ranging from 0xE100 through 0xE1E9 referencing characters in the Segoe UI Symbol font, and they correspond with the 192 separate app bar buttons style in the old StandardStyles.xaml. (These 192 buttons are displayed in the LookAtAppBarButtonStyles program in Chapter 8 of Programming Windows, 6th edition.)

You may have noticed a discrepancy: 190 enumeration members vs. 192 button styles. The two that are missing from the Symbol enumeration are 0xE179 (“All Apps”) and 0xE183 (“Sky Drive”).

Here’s how the three application bar buttons appear in the new DirectXPage.xaml file:

<Page.BottomAppBar>
    <AppBar Padding="10,0,10,0">
        <Grid>
            <StackPanel Orientation="Horizontal" HorizontalAlignment="Left">
                ...
            </StackPanel>

            <StackPanel Orientation="Horizontal" HorizontalAlignment="Right">
                <AppBarButton Icon="Clear"
                              Label="Clear All"
                              Click="OnClearAllAppBarButtonClick" />

                <AppBarButton Name="appBarOpenButton"
                              IsEnabled="False"
                              Icon="OpenFile"
                              Label="Open File">
                    ...
                </AppBarButton>

                <AppBarButton Name="appBarSaveAsButton"
                              Icon="Save"
                              Label="Save As">
                    ...
                </AppBarButton>
            </StackPanel>
        </Grid>
    </AppBar>
</Page.BottomAppBar>

But wait! There's more!

Flyouts

It is often the case that an application bar button (or some other button) invokes a little dialog box. Although Programming Windows, 6th edition demonstrated how to make a Windows 8 dialog box from a Popup and a UserControl derivative, in Windows 8.1 there is a better way: Flyout and the related MenuFlyout both derive from FlyoutBase.

FlyoutBase has some events for initialization and cleanup, and some methods and properties for placement. Most importantly, Flyout defines a Content property of type UIElement, which allows you to define a visual tree that appears in the Flyout.

What’s more, Button has a new property named Flyout of type FlyoutBase, which automatically displays the Flyout dialog or FlyoutMenu when the button is pressed. This means that you can define both the button and the visual tree of the associated flyout entirely in markup.

In the Windows 8.1 version of BasicFingerPaint I was able to eliminate the OpenDialog and SaveDialog classes in the earlier program, and move everything into DirectXPage.xaml and the code-behind file. Here’s how both the application bar buttons and flyouts appear in the 8.1 version of DirectXPage.xaml:

I like this! It's a very nice way to consolidate related user interface.

Saving the Current Drawing

As I mentioned in the earlier blog entry about the new templates, the SaveInternalState call from App.xaml.cpp to DirectXPage.xaml.cpp does not include the SuspendingOperation object, which effectively limits the method to synchronous calls. This is easy to fix (which I did in the new BasicFingerPaint project as I had to do in the original BasicFingerPaint), but it shouldn’t be broken.

Can you imagine somebody not knowing (or forgetting) that asynchronous operations can't be performed when the app is being suspended unless this deferral is obtained? That's a hard bug to catch — as I know from experience — and there's no reason the template should obscure that. Indeed, the template should generate code showing this deferral actually being used.

The Rendering Surface

As I also mentioned in that earlier blog entry, DirectXPage now uses the new SwapChainPanel rather than the earlier SwapChainBackgroundPanel as a rendering surface for DirectX output. In the BasicFingerPaint program, the rendering surface is always the size of the program’s window, so it really doesn’t make a difference which is used, but the DirectXPage.xaml.cpp file created by the template installs handlers for the SizeChanged and CompositionScaleChanged events of SwapChainPanel, and this latter event is not supported by SwapChainBackgroundPanel.

Input and Rendering in a Secondary Thread

The most significant archictural change to BasicFingerPaint in the transition from Windows 8.0 to Windows 8.1 occured as a result of the new CoreIndependentInputSource.

In the Windows 8.0 version of BasicFingerPaint, everything happens in the user interface thread. The Pointer events run in the UI thread, and the rendering loop — a handler for the CompositionTarget::Rendering event — also runs in the UI thread. To avoid unnecessary calls to Render and Present when no new Pointer input has been processed, I defined an IsRenderNeeded property in the Renderer class.

In Window 8.1, you can call the CreateCoreIndependentInputSource method of SwapChainPanel (or SwapChainBackgroundPanel) in a secondary thread to object a CoreIndependentInputSource. This class supports all the Pointer events, except that the handlers for these events also run in that secondary thread. Don't worry if this sounds messy: The “DirectX App (XAML)” template automatically generates this code for you.

This means that in a program such as BasicFingerPaint, it makes much more sense for the Pointer event handlers to make Render and Present calls directly to update the screen entirely in this secondary thread, leaving the user interface thread for more mundane tasks.

Therefore, I disabled the rendering loop entirely in the new BasicFingerPaint. The various Pointer event handlers in DirectXPage call into methods in BasicFingerPaintMain to pass the information to BasicFingerPaintRenderer. The various Pointer event handlers also call the Render method in BasicFingerPaintMain, which calls the Render method in BasicFingerPaintRenderer to render all the finger-painted strokes, followed by a call to the Present method in DeviceResources. All of this occurs in the secondary thread.

Of course, some other changes had to be made to accomodate this secondary thread. In the original version of the program, the PointerPressed handler accessed the two ComboBox collections to obtain the current color and stroke width selected by the user. Because this handler is now running in a secondary thread, it can't access Windows Runtime user interface objects. So I installed SelectionChanged handlers for the two ComboBox controls and saved the current selections as private members, which are then accessed by the PointerPressed handler. Simple enough.

Synchronization

All the Pointer event handlers in DirectXPage call Render and Present but some other event handlers in DirectXPage also need to call Render and Present, for example, when a drawing is loaded from disk, or when the size of the SwapChainPanel changes. These calls occur in the user interface thread.

What this means is that the program might be making Direct2D calls simultaneously from two separate threads. If you read "Multithreaded Direct2D Apps" you’ll discover that this isn’t a problem. All you need to do is call D2D1CreateFactory with the D2D1_FACTORY_TYPE_MULTI_THREADED flag.

The only catch: The DeviceResources class created as part of the “DirectX App” and “DirectX App (XAML)” templates calls D2D1CreateFactory with the D2D1_FACTORY_TYPE_SINGLE_THREADED flag, and I’d rather not take the responsibility for altering something in DeviceResources.

This means that I need to do my own synchronization. Fortunately there’s an exceptionally easy way to do it. In DirectXPage.xaml.h, you can define an object of type critical_section from the Concurrency namespace:

Concurrency::critical_section m_criticalSection;

To use this to obtain exclusive access to a particular resource (such as DirectX), you call:

critical_section::scoped_lock lock(m_criticalSection);

The first time I saw this code I said “Huh?” The syntax didn’t look right. Turns out that scoped_lock is a member class of the critical_section class. The code is instantiating an object named lock of type scoped_lock, passing to the scoped_lock constructor the critical_section object.

Instantiating an object of type scoped_lock either (1) takes ownership of the critical_section object and continues executing code, or (2) blocks execution if the critical_section object is currently owned by another thread. What’s nice about scoped_lock is the definition of its destructor, which causes ownership of the critical_section object to be released when the scoped_lock object goes out of scope. This means that you can create the scoped_lock object in a method (or block), and it will release ownership of the critical_section object at the end of the method (or block).

This means that you can liberally scatter this single line of code wherever a thread needs exclusive access to some resources. In DirectXPage.xaml.cpp, I put it in every method that calls into DeviceResources or BasicFingerPaintMain.

Geometry Realizations

Geometries in Direct2D are device-independent objects. This makes sense because they're basically collections of coordinate points, and coordinate points are the same regardless of the output device. (I explore geometries in more detail in the installment of DirectX Factor scheduled for the September issue of MSDN Magazine.)

In Windows 8.1, Direct2D has a new feature called "geometry realizations" that are device dependent objects and undoubtedly render faster than normal geometries.

To create and use geometry realizations, you'll first need to use D2D1CreateFactory to create a factory object of type ID2D1Factory2. (The DeviceResources class in the new templates does this). If you're familiar with the numbering scheme for enhancements in DirectX, you'll know that ID2D1Factory2 derives from ID2D1Factory1, which derives from ID2D1Factory. ID2D1Factory2 overrides the CreateDevice method to return an object of type ID2D1DeviceContext1, which of course derives from ID2D1DeviceContext but defines three new methods:

CreateStrokedGeometryRealization

CreateFilledGeometryRealization

DrawGeometryRealization

The first two methods take an object of type ID2D1Geometry and create an object of type ID2D1GeometryRealization. The CreateStrokedGeometryRealization call requires a stroke width, a stroke style (ID2D1StrokeStyle), and a flattening tolerance so curves can be converted into series of tiny straight lines. The CreateFilledGeometryRealization just requires the flattening tolerance.

DrawGeometryRealization requires only the ID2D1GeometryRealization object and a brush to render the geometry.

It's quite likely that the geometry realization is actually an ID2D1Mesh object, which is basically a collection of triangles that describes the area that must be filled with the brush when the geometry is rendered. It might be interesting to compare the performance of DrawGeometryRealization with the performance of FillMesh to render an ID2D1Mesh object created from the same geometry.

Anyway, because the ID2D1GeometryRealization is a device-dependent object, it must be recreated if the device is recreated.

To take advantage of this new facility, I added a new data member of type ID2D1GeometryRealization to the StrokeInfo structure. However, the program does not create this object as a stroke is in progress, but only when the stroke is completed. (I strongly suspect there would be no overall performance improvement if the geometry realization is rendered only once.) The ID2D1GeometryRealization is also created when strokes are loaded into the renderer, and when the device context is recreated.

In the Windows 8.0 version of the BasicFingerPaint, the Render method contains two loops — one for the completed strokes and the other for strokes in progress — and both loops called a second method named RenderStroke. In the Windows 8.1 version of the program, the Render method contains those same two loops, but for the completed strokes it calls DrawGeometryRealization, and for the strokes in progress, it calls DrawGeometry. Hence, the RenderStroke method no longer serves a purpose and has been removed.

Although Windows 8.1 API does not represent a really big upgrade over Windows 8.0, I was amused to discover how many new features I could incorporate into a simple finger-painting program.