I recently decided that it might be nice to provide some Windows shell customizations for handling a new file format that I am working on. Making some fields available to Windows Search might be nice and customizing the default view when a user examines a directory of our files. Also customizing the icon shown so that it reflects the file content. All these are possible using Shell plugins and there is even a nice wizard for ATL in Visual Studion 2010 that will get things started.

So I began with the default ATL filehandler extension that is provided by the ATL wizard and started to add some code to the preview window and implemented the thumbnail handler. For this we can see if the file has some data we could draw and then generate an image. In our case, sometimes the file contains an image - if so, we can draw this in the preview and also use it as the thumbnail.

Now the preview view was working fine but the images fail to paint properly in the thumbnail. I reduced the code there to just draw some lines and it started to work in that the line was present but the color was always black.

So here is the code used by ATL to prepare the drawing context. It creates a memory display context and selects a bitmap into that and when we later draw on this memory DC the result ends up in this bitmap.

BOOL GetThumbnail(
		_In_ UINT cx,
		_Out_ HBITMAP* phbmp,
		_In_opt_ WTS_ALPHATYPE* /* pdwAlpha */)
	{
		HDC hdc = ::GetDC(NULL);
		RECT rcBounds;

		SetRect(&rcBounds, 0, 0, cx, cx);

		HDC hDrawDC = CreateCompatibleDC(hdc);
		if (hDrawDC == NULL)
		{
			ReleaseDC(NULL, hdc);
			return FALSE;
		}

		HBITMAP hBmp = CreateCompatibleBitmap(hDrawDC, cx, cx);
		if (hBmp == NULL)
		{
			ReleaseDC(NULL, hdc);
			DeleteDC(hDrawDC);
			return FALSE;
		}

		HBITMAP hOldBitmap = (HBITMAP) SelectObject(hDrawDC, hBmp);

		// Here you need to draw the document's data
		OnDrawThumbnail(hDrawDC, &rcBounds);

		SelectObject(hDrawDC, hOldBitmap);

		DeleteDC(hDrawDC);
		ReleaseDC(NULL, hdc);

		*phbmp = hBmp;
		return TRUE;
	}

There are two problems here. When a memory DC is created it has by default a monochrome bitmap. Here is what MSDN has to say:

A memory DC exists only in memory. When the memory DC is created, its display surface is exactly one monochrome pixel wide and one monochrome pixel high. Before an application can use a memory DC for drawing operations, it must select a bitmap of the correct width and height into the DC.

So when we then create a compatible bitmap from this, we get a monochrome bitmap. So the first fix is to use the window display context so that we can support a color bitmap: CreateCompatibleBitmap(hdc, cx, cx);.

The second problem shows up when reading the documentation for IThumbnailProvider::GetThumbnail.

phbmp
[out] When this method returns, contains a pointer to the thumbnail image handle. The image must be a device-independent bitmap (DIB) section and 32 bits per pixel.

Oops. CreateCompatibleBitmap creates device dependent bitmaps. We need to be using CreateDIBSection to get a device independent bitmap. If we create a DIB and select that into the memory display context then all should be well. So to fix this the default GetThumbnail() function must be overridden to prepare a proper surface for drawing.

BOOL CDemoDocument::
GetThumbnail(_In_ UINT cx, _Out_ HBITMAP* phbmp, _In_opt_ WTS_ALPHATYPE* /* pdwAlpha */)
{
    BOOL br = FALSE;
    HDC hdc = ::GetDC(NULL);
    HDC hDrawDC = CreateCompatibleDC(hdc);
    if (hDrawDC != NULL)
    {
        void *bits = 0;
        RECT rcBounds;
        SetRect(&rcBounds, 0, 0, cx, cx);

        BITMAPINFO bi = {0};
        bi.bmiHeader.biWidth = cx;
        bi.bmiHeader.biHeight = cx;
        bi.bmiHeader.biPlanes = 1;
        bi.bmiHeader.biBitCount = 32;
        bi.bmiHeader.biSizeImage = 0;
        bi.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
        bi.bmiHeader.biClrUsed = 0;
        bi.bmiHeader.biClrImportant = 0;

        HBITMAP hBmp = CreateDIBSection(hdc, &bi, DIB_RGB_COLORS, &bits, NULL, 0);
        if (hBmp != NULL)
        {
            HBITMAP hOldBitmap = (HBITMAP)SelectObject(hDrawDC, hBmp);
            OnDrawThumbnail(hDrawDC, &rcBounds);
            SelectObject(hDrawDC, hOldBitmap);
            *phbmp = hBmp;
            br = TRUE;
        }
        DeleteDC(hDrawDC);
    }
    ReleaseDC(NULL, hdc);
    return br;
}

Now it works!