The Enterprise Eightfold Path

Deploying per-user file type associations (FTAs) on XenApp 7.x, Server 2012 R2, Windows 8.1, Server 2016 and Windows 10

The Enterprise Eightfold Path

Deploying per-user file type associations (FTAs) on XenApp 7.x, Server 2012 R2, Windows 8.1, Server 2016 and Windows 10

By James Rankin   |     Friday 27 May 2016

Tags:

We’ve been down the file type associations route before. I’ve had quite a lot of emails and questions as a result of my previous articles, so I think it’s ripe for a bit more clarification and a bit of a (second!) revamp. They’re a particular bugbear to those of us using XenApp 7.x or RDSH systems, because we’re used to doing them the old easy way, and now the rules have unceremoniously changed.

Note – the background to this article has simply been lifted-and-shifted, for the most part, from the previous article referenced above. However, once we’re down in the detail sections, there will be a lot more detail than before. Also, I’m going to remove the previous article in a couple of weeks, as I feel this one is more complete.

FTAs?

We’re all familiar with FTAs – they are the program that opens a particular file type extension. Files with the .docx extension commonly open in Microsoft Word. Files with the .txt extension commonly open in Notepad, but you can use many other things, WordPad, Notepad++, even Microsoft Excel. Each user normally has a default FTA that is the program that will open it when you double-click a particular file type. If you wanted to open the file type in a different program (for instance, open a file with the .xml extension in Notepad instead of the default of IE), you can right-click on the file and choose Open With.

Why deploy per-user file type associations?

Imagine you have a XenApp server hosting sessions for a multitude of users. Some users want to use, for instance, Adobe Reader to open PDF files. But some may want, for job reasons, to use something like Adobe Acrobat Pro. There may well be licensing issues to take into account that mean you can’t let everyone use the full version, so you may want to enforce the PDF file type association differently for different groups of users. Having to silo XenApp servers based around a file type association doesn’t make an awful lot of sense, so I know this is fairly common practice (provided the licensing from the vendor allows this form of control).

There are many other applications that you may want to define this way. Some users may have to open Visio drawings in Visio Viewer and others using the full version. Maybe they might want to open text files in Notepad or Notepad++ dependent on function. Picture files, video files, etc; there are a huge amount of file formats that this could apply to.

And finally, you may want to simply set a specific default FTA for an application, but also then give the users the option to change this to a different application they prefer, and then persist that setting from session to session (potentially across different devices), a particular consideration in XenApp or RDSH environments. (I can hear Microsoft protesting as we speak, you want to do what?)

But the main clincher is – it used to be so straightforward to enable this, either as an enforced policy, or a pre-defined preference that user could change and expect to persist. And in my humble experience, it’s not just confined to XenApp or RDSH or other multi-user systems – I’ve worked with plenty of enterprises who wanted to use these techniques on VDI or physical.

The old ways

In Windows 7/2008 R2/XenApp 6.x, life was so simple. Group Policy Preference Folder Options could be used to enforce a particular FTA, or you could set it to “Apply once” to give them a specific default and then control it themselves, simply by roaming the FileExts area of the user Registry. This was a great improvement over previous Windows versions, where you were reduced to using the ftype and assoc commands which were a bit of a mystery.

The new ways

Of course, as soon as we’re on Windows 8.1, Windows Server 2012 R2, Windows 10 or Windows Server 2016 (and along with that, anything on XenApp 7.x), the whole thing has been changed. Why? Your guess is as good as mine, but I believe it’s part of a wider play from Microsoft aimed at reducing the complexity of multi-user systems, and henceforth making them easier (read: cheaper to support!) targets for Azure migration. The news from Synergy about Windows 10 XenDesktops being available in Azure makes me more confident that this is the end goal. Seeing Citrix and Microsoft allying in this way kind of makes me think of SHIELD and HYDRA (I’m very fond of a comic-book analogy), but that’s an opinion for another article.

Once we’re on these later platforms, the Group Policy Preferences method I wrote about previously simply flat-out doesn’t work any more. Normally, user FTA settings are written to a specific Registry area – HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts, with subkeys for each filetype (e.g. .PDF). But Microsoft appear to have changed something. Now, when you write a new filetype using the Default Programs part of Control Panel (or the Open With context menu), the Registry settings are changed in a very different way.

To show you what I mean, I’ve logged on to a Server 2012 XenApp 7.8 system with three different PDF readers installed locally (Adobe Reader, Foxit Reader and SumatraPDF), and I’m going to select FoxIT Reader as the application to use. The way I trigger this is by right-clicking a PDF file and choosing Open With | Choose Default Program, but you can also access this through the Control Panel.

Once I select FoxIT Reader as the default program, we can see the Registry changes we’d expect – but there are some additions we haven’t seen previously

Note the Hash value. Apparently, this is a hash generated from the application, the computer name and the username, and which needs to be unique in order for this key to be used. So if I export these values into the user profile and roam them to another XenApp server, the FTA will be lost. If you export, roam, save or persist this Registry key, when you move to another server system, you are prompted to choose the default program again, because the hash doesn’t match. Apparently this was done to reduce tampering by malware, but it’s very short-sighted because it doesn’t take into account multi-session systems like XenApp and RDS.

Incidentally, here’s Microsoft’s specific quote on the issue (which naturally avoids telling us they’re probably dumbing down your solutions to make them easier to absorb into Azure :-))

“Pre-Win 8, apps could set the default handler for a file type/protocol by manipulating the registry, this means you could easily have a script or a group policy manipulating the registry.

However in Win 8 onwards, the registry changes are verified by a hash (unique per user and app) that detects tampering by apps. In the absence of a valid hash, we ignore the default in the registry.”

A result of this, and even more annoying, is the fact that the action of setting a default program in this way also writes a permissions change to the UserChoice key. See below:-

Note the Deny set for the user account that made the change (in this case I was logged on as Administrator). This means that you can’t set the values in this key through a script or such like without altering the Registry permissions first. But even if you do alter the permissions, you need the Hash value to match the server system – and without knowledge of the algorithm in use, you’re not going to get very far.

What do Microsoft recommend that we do?

Well, they have actually given us a method of dealing with this. Let it not be said that they haven’t made a function available for setting the file associations! It is somewhat long-winded, and involves setting the file association(s) for a user, and then exporting it (them) out to an XML file using the following command

Once you’ve exported the XML file with your configured file associations (you can edit it by hand to just get the FTAs you need), you can then deploy this at logon by using a Group Policy Object. This all sounds good so far…

…until you realize that the GPO that sets the FTA is Computer Configuration only. That’s right, it writes to a Registry key in HKEY_LOCAL_MACHINE. Making it a royal pain in the proverbials for deploying to multi-session environments such as XenApp. If you don’t believe me, check it out – it’s to be found only in Computer Config | Policies | Admin Templates | Windows Components | File Explorer

This is really annoying. As I said before, I don’t think this move is in any way truly security-related. If it was, they could simply write up a GPO that does this on a user basis. It’s not the only GPO that has moved quite annoyingly to being specifically device-based – try setting a home page for Microsoft Edge for different sets of users. Anyway – Microsoft’s plans for world domination aside, let’s address this in the here and now. How can we manage it?

Cracking the problem

I did some work with Duncan Murdoch on this, and our testing (and that of other guys I’ve spoken to, such as Freek Berson, Shaun Ritchie and Ryan Gallier) indicates that the basic formula we came up with works. However, the last article I did wasn’t particularly clear about the requirements, so we’ve laid them out much more simply in this one, as well as adding some gotchas that apply specifically to Server 2016 and Windows 10.

We’ve approached this from two perspectives:-

1. You just want to enforce a specific FTA for certain user groups (a policy setting, if you like) – every time the user logs on the FTA is set

2. You want to enforce a specific FTA at first logon, and then allow the user to change this if they wish (a preference setting) – this selected setting is then saved and roams with the user

Naturally, #2 is a lot more detailed than #1 🙂

Technical details

Firstly, a bit of technical background.

I looked at three FTAs originally – PDF, VSD and JPEG/JPG. These are fairly common and are all FTAs I’ve been asked about in comments and emails. A lot of people also talked about the browser and office documents – but those are specifically addressed in the Limitations section. However, although all of these are in the attached AppSense config, I’ve again specifically used the PDF file type as the main example.

There are three main areas of interest in the Registry when it comes to FTAs (where xxx is the file type extension concerned)

Key #1 – HKLM\Software\Classes\.xxx
Key #2 – HKCU\Software\Classes\.xxx
Key #3 – HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.xxx

The interplay between these is unclear, but each is queried when it is looking for particular file type actions to perform. There are many other areas in the Registry that are also queried, but these three appear to be the most pertinent.

When a user changes an FTA manually (using Open With or Default Programs), this appears to write a UserChoice subkey to Key #3 (along with the Hash value, and the Registry permissions change).

Keys #1 and #2 are used mainly as “guides” where the final key doesn’t exist (that’s about all I can tell – I’m no developer!) And this is the trick we’re going to utilize to allow us to both set and roam the FTAs.

The final thing we need to know, from a technical perspective, is the file type handler. This redirects to another Registry key in HKCU/HKLM\Software\Classes\filetypehandler to get the list of commands associated with it (normally applications with switches). You can usually dig these up by installing the software in question, opening the file type extension with it, and looking at the UserChoice key in FileExts (the last Registry path above). Here are some common ones we’ve come across in our testing to get you started:-

AcroExch.Document.xx – Adobe Reader version xx (for PDF)
Applications\SumatraPDF.exe – Sumatra PDF (for PDF)
FoxitReader.Document – FoxIT Reader (for PDF)
AcroExch.Document.DC – Adobe Document Cloud (for PDF)
Word.Document.xx – Word version xx (version number, not year of release) (for DOCx)
Applications\Swriter.exe – OpenOffice (for DOCx)
docxfile – WordPad (for DOCx)
Visio.Drawing.xx -Visio version xx (version number, not year of release) (for VSD)
VisioViewer.Viewer – Visio Viewer (for VSD)
GIMP-2.8-jpeg – GIMP (for JPG)
PBrush – Microsoft Paint (for JPG)
jpegfile – Windows Photo Viewer (for JPG) – not available on Windows 10 or Server 2016
txtfile – Notepad (for TXT)

Note that any file type handler starting with Applications indicates it isn’t a registered file type, and just points directly to an executable.

Limitations

Now, if you want to do things like apply browser and Office document FTAs you’re going to find it a little more difficult. Office-wise, most people are Microsoft customers anyway and unlikely to change (in my experience, anyway!) It’s also quite tricky to get the likes of OpenOffice, for instance, to register properly as a file type handler when changing this around. For this reason (and to try and keep this article down in size!), I’m going to avoid visiting the subject of Office FTAs. If anyone’s interested in co-operating on a way to do this, please get in touch.Internet browsers are also very difficult. There are different handlers for each protocol type, and also various file types to be considered (XML, HTM*, MHTML, etc.) There are also a huge range of Registry values that deal with this, and the fact that many of them also have the infamous Hash value now applied to them as well, meaning that moving between devices just prompts the user to choose an association again (or resets it back to Edge, on Windows 10 or Server 2016). Here’s an example of some places you would need to look to get a browser FTA set (and this is without getting started on individual protocols or filetypes!):-

HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts
HKCU\Software\Microsoft\Windows\Shell\Associations\UrlAssociations
HKLM\Software\Classes
HKCU\Software\Classes
HKCU\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\MuiCache
HKCU\Software\Microsoft\Internet Explorer

So again, browser-wise, I’m going to have to leave this one for the moment, but it’s something I’d be very interested in collaborating on, if anyone has any time and/or insight to share!Windows 10 and Server 2016

And finally, a quick note around Windows 10 and Server 2016 with Modern Apps. Using the tricks we’re going to describe sends these operating systems into a twist, and they will aggressively reset your FTAs to their own favourites, Modern Apps, with even the slightest hint of encouragement. So, for instance, if you’re putting a custom user FTA on PDFs, for instance, you need to get rid of Microsoft Edge. If you’re doing it for JPGs, you need to get rid of the Microsoft Pictures app. Use my post about disabling Modern Apps to get this done, otherwise they will constantly reset your custom file type associations in that nannying, intrusive, we-know-what’s-best-for-you-and-all-your-users way that Microsoft seem ever so keen to impose on everyone.

Enforcing a particular FTA

Firstly, we will just look at enforcing a particular FTA. So when a user logs on, they are provided with an FTA that is decided by the administrator based around a specific Condition (security group, OU, you choose).

The main thrust is, that if neither Key #1 or Key #3 mentioned above exist when the user logs on, then the (Default) value from Key #2 will be used to apply the FTA to the user. So, to summarize:-

  • Ensure that Key #1 is removed (this is HKLM)
  • Ensure that Key #3 is removed (this is HKCU)
  • Ensure that Key #2 specifies the correct file type handler for the application, and that this is restricted to a specific group, OU, user, whatever you desire

You can do this quite easily with Group Policy Preferences. Here are some examples:-

This one sets the default PDF FTA to either Adobe Reader, FoxIt Reader, or Sumatra PDF based around a group membership

This one sets the default VSD FTA to either Visio Standard 2007 or Visio Viewer 2013, based around an OU membership (quick confession – I could only test this one on Windows 10 and Windows 8.1 because my copy of Visio doesn’t run on RDSH)

These should work fine once configured for your environment. The only problem with these is if an administrator installs a new application it may rewrite the HKLM value in Key #1 until the Group Policy refresh cycle takes place, but a controlled process for installation of new software should take care of this potential issue.

Setting and then roaming a user-defined FTA

This is a little trickier, but we’ve done it in test for each of PDF, VSD and JPG files here. Extending it may prove tricky, dependent on the application (I’m working on others, but in reality it sometimes depends on how the app writes to the Registry).

This is a little complicated, tooling-wise. Surprisingly enough, I’m using AppSense Environment Manager to handle this. Here is an example workflow which you can adapt to a script or other tool. In the example, we’re using PDF, but we have successfully tested this with VSD and JPG as well – hopefully the technique is easily portable to other file type formats.

In this example shown, when a user logs on, I am setting the default PDF handler to Adobe Reader 11, but then overwriting it with a custom user preference if this exists in a network share. At logoff, if the default handler has been changed, it is written to a Registry key via PowerShell and then saved out to the network share mentioned above. So the user will always get Adobe Reader 11 as the PDF handler – unless they have changed this, in which case it will apply their customized setting from the network.

Logon

(Filetype PDF)

  • Set default FTA for PDF – write (AcroExch.Document.11) to REG_SZ HKCU\Software\Classes\.pdf\(Default)
  • Delete Registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.pdf
  • Delete Registry key HKLM\Software\Classes\.pdf (elevated rights required)
  • If it exists, hive in the value for HKCU\Software\Classes\.pdf\ from \\SERVER\SHARE\file (overwriting the value from the first step with the user customization)

Post-logon (Windows 10 or Server 2016 only)

  • Delete key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.pdf\UserChoice (this sometimes gets re-populated by those pesky Modern Apps!)

Logoff

(Filetype PDF)

IF user has changed FTA (value ProgId exists in HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.pdf\UserChoice)

  • Run PowerShell to copy HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.pdf\UserChoice\ProgId to HKCU:\Software\Classes\.pdf\(Default)
  • Hive out the value from HKCU\Software\Classes\.pdf to \\SERVER\SHARE\FILE
  • Delete Registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.pdf
  • Delete Registry key HKLM\Software\Classes\.pdf (elevated rights required)

ELSE

  • Hive out the value from HKCU\Software\Classes\.pdf to \\SERVER\SHARE\FILE
  • Delete Registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.pdf
  • Delete Registry key HKLM\Software\Classes\.pdf (elevated rights required)

This may seem a bit convoluted, so here it is in AppSense EM to maybe make things seem a bit more straightforward. Here are the Actions you need to perform at logon:-

The details of the Action “Set Adobe Reader as default PDF FTA” are also here:-

As for the logoff section, this is put together as below:-

If the user has selected a new FTA for the PDF filetype during their session, then a value will have been written to the UserChoice key. This next check sees if that value exists…

…and if it does, this PowerShell Action that copies the ProgId key from UserChoice to the Classes\.pdf key is shown here, which means it should now be imported at next logon (because the UserChoice values will not be saved)

And the Hiving Actions (both in Logon and Logoff) are copying of Registry entries from HKEY_CURRENT_USER\Software\Classes\.pdf either from, or to, a network location. You could do this in AppSense, using regedit.exe, using reg.exe, PowerShell, VBScript – loads of ways to crack this particular nut.

The trickiest bit is elevation of privilege for the HKLM removals, which is where AppSense EM makes everything so much easier (SYSTEM elevation as a tick-box). I believe Ryan Gallier did some work around this in batch on CitrixIRC

Yes, this feels massively more complicated to write down than it was to put together! The AppSense EM config is available here if that helps anyone out (it actually contains entries for PDF, JPG and VSD, for the record).

Summary

This feels quite hideously complicated. It’s a right headache compared to how easy it used to be in the past. Microsoft, ten out of ten for making our lives harder. Hope all those Azure dollars are worth it!

If you’re just enforcing then Group Policy cuts it nice and easily (well, fairly easy, nowhere near as easy as it used to be). If you want to roam them from device to device, you’re into some heavy scripting or UEM tools, mainly to get the timings right and use elevated permissions to delete from HKLM.

Bear in mind that this is heavily application-dependent. I found with GIMP, that if you didn’t allow the application to register itself as the handler for the filetype you’re playing with, this won’t work. So make sure before you start doing this that you’ve allowed the target application to “hog” all available filetypes.

And Modern Apps that have FTAs (like Edge, Photos, Videos, etc.) really screw the pooch. If you want to use this method, get rid of them! They also have an aggressive tendency to overwrite the Registry settings you pushed down at logon. Frankly, Modern Apps are better off out of the equation. See my previous blog article on how to kill the damn things – almost as hard as killing vampires. There is, however, an XML file in %systemroot%\system32 called DefaultOEMAssociations.xml that may be the source of the apps for these resets – will do a follow-up article on how to get rid of this if it turns out that this is where they come from.

Mainly, I see people using this method a) because they did before, and b) to avoid licensing issues with things like Adobe Acrobat Pro. If this helps you in any way get around these issues, then it’s been worth it.

Phew! Enough already. I’m off to watch the football 🙂

Comments

18 responses to “Deploying per-user file type associations (FTAs) on XenApp 7.x, Server 2012 R2, Windows 8.1, Server 2016 and Windows 10”

  1. Vanilla Bean Vanilla Bean says:

    Can't wait for EM V10 which comes with file type association policy actions.

  2. Vanilla Bean Vanilla Bean says:

    Wow….does it? I must have missed that when I've been testing (spins up AppSense v10 VM)

  3. Vanilla Bean Vanilla Bean says:

    Indeed there is a File Type Association policy action in version 10. However, I hope when it comes to the release version, that it actually works, because I've just tested it and it simply doesn't do anything.

  4. Vanilla Bean Vanilla Bean says:

    Hi James,

    As I'm looking into FTA's for IE for 2012R2 I stumbled on this post.

    Basically, how the HKLM and HKCUSoftwareClasses keys work is that the FTA in HKLM is only used when it doesn't exist in HKCU. So if you were to create and FTA for vsd to Visio.Drawing.xx in HKLM and an FTA for vsd to Visio.Viewer in HKCU, your vsd's would open in Visio.Viewer. At least, that's how it works for me on 2012R2, haven't tested this on 2016.
    And now that we're talking about Visio (2010, not sure about other versions), it has the incredible annoying habbit of repairing its FTA's when they're not set to Visio.Drawing.xx in HKLM. So in the scenario where you want only a limited set of users to have access to Visio for vsd's you'd have to leave the Visio FTA in HKLM and create an FTA for Viewer in HKCU for the other users (or everyone and denying the Visio group users)

    Not sure where the FileExts key comes in in my scenario, I don't use it.

    Wout

  5. Vanilla Bean Vanilla Bean says:

    I can confirm that the release version of AppSense v10 has a File Associations Action that does actually now work 🙂

  6. Vanilla Bean Vanilla Bean says:

    Just want to say thank you for a succinct article on the topic and wish I had found it a week (and a few dozen grey hairs) earlier. Definitely a bookmark!

    Do you happen to know of a way to disable the "Choose Default Program" option that some files have (i.e. pdf)? In our instance we don't want users modifying the option for set associations. The policy is set and applies next time they login but I don't want them to even be able to change it for a session nor to be able to browse around the system vol from the context menu option.

  7. Vanilla Bean Vanilla Bean says:

    In a nutshell, not sure. I can't see Microsoft disabling that entry point with a policy because it needs to be there to allow the user to change it, and MS these days are big on user freedom (God knows why).

    I would dare say AppSense Environment Manager could disable it with the Lockdown feature, but it's not worth it just for that feature alone.

  8. Florian says:

    Hello everybody,

    I’ve got a big Problem concerning File Type Association with tif and mspview.exe. Our OS is W2k12R2 and we Need to open tif and tiff only with mspview.exe from SharepointDesigner.
    Can you please help?
    Thanks a lot

  9. James Rankin James Rankin says:

    You can drop me an email at james [at] htguk [dot] com if you are having trouble and the techniques in this article don’t seem to help…

  10. Matthew Lonsdale says:

    This is currently a huge problem for us a business and customer of AppSense. We’ve recently upgraded our infrastructure from 8.0 to 8.9. Obviously this has been a huge leap forward, but now we are in position where we cannot set FTA for users on our Citrix estate. does anyone know if this issued has been resolved with the latest release of AppSense Desktop NOW. If so we maybe looking at an additional upgrade.

  11. James Rankin James Rankin says:

    Sorry I missed this comment because of a spam storm… there should be a new article out today that may help you with this…stand by…

  12. xxx says:

    Have you ever thought about creating an ebook or guest authoring on other websites?
    I have a blog based upon on the same topics you discuss
    and would really like to have you share some stories/information. I know my readers would appreciate your
    work. If you are even remotely interested, feel free to send me
    an e mail.

  13. James Rankin James Rankin says:

    Indeed they are, because the idiots at DropBox decided to force the Public folder to be private and therefore have broken every download link in my entire blog history. Thanks for that, DropBox!!!

    I will reinstate these ones tonight when I get a chance. Any more people find please let me know in the comments and I can recreate the links on a case by case basis.

    Cheers,

    JR

  14. Rajen says:

    Thanks, James. Looking forward to seeing the configuration. I’ve been battling the same issue.

  15. James Rankin James Rankin says:

    The links for this article should now be fixed. Thanks again, DropBox.

  16. […] a great article by James Rankin with a detailed explanation of the changesand technical details (http://35.176.47.236/deploying-per-user-file-type/) which is a great […]

  17. […] a great article by James Rankin with a detailed explanation of the changesand technical details (http://35.176.47.236/deploying-per-user-file-type/) which is a great […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Join our mailing list

Sign up for our newsletter today and we'll send you exclusive content including free guides and articles. We promise not to send you spam and we don't share your details with anybody else.

Contact us

Howell Technology Group
One Trinity Green
Eldon Street
South Shields
NE33 1SA

T. 0191 4813446

Email us

Cookies policy

The HTG website uses cookies to store information on your computer. By continuing to browse this website you are agreeing to our use of cookies. Learn more

Accept

Thank you - you've accepted our cookies policy.