Problems with OAuth Access Token encryption and decryption using Microsoft.OWIN hosted in IIS.

If you want to secure access to your WebAPI calls, a mechanism you can use is OAuth2 Bearer tokens. These tokens are generated via a login call for instance, and the website or mobile app can hold on to this token to authenticate with the server. These tokens can be generated using Microsoft’s OWIN implementation (also known as Katana).

These tokens have an expiration date. After that date you won’t accept the token obviously. However there are also some situations that can occur where the token can’t even be decrypted.

First of all, the default way of encrypting the token when you host the Owin/Katana in your own process (HttpListener or otherwise) is different from when it is being hosted in IIS using the SystemWeb host (which is a separate NuGet package btw). The former uses the DPAPI to protect the tokens, while the latter uses ASP.NET’s machine key data protection. There is also the option of providing your own provider/format.

I am currently only familiar with the SystemWeb host under IIS, and we recently ran into some problems after updating our software and moving it to another machine. See, we had these mobile devices who registered with our WebAPI service and stored a token which should not expire. However, after the update we found the tokens would not decrypt anymore and our users were presented with a security error, which meant they had to reregister the device with our software.

We quickly found out that we forgot to set the machine key in our web.config so encryption on the new server was different than the old one. However after configuring our web.config to use the same machine key as the old server tokens were still not being decrypted.

After a lot of searching it turned out that Microsoft.Owin 3.0.1 will not decrypt tokens created by Microsoft.Owin 3.0.0. As soon as we downgraded all our Microsoft.Owin packages back to 3.0.0 version it worked again.

To make a long story short:

Make sure both machine key and Microsoft.Owin versions stay the same if you want your tokens to keep working after an update of your software. Otherwise you find out the hard way why you should probably have used your own token encryption/decryption scheme in the first place. Our next order of business is finding a way to update our Microsoft.Owin version in the future without breaking our current user’s device registrations.

NuGet package UI always indicates an update for some packages.

Or, why you don’t get a nice reassuring green checkmark after an update of a package.

I recently noticed in a rather large solution with around 70 installed NuGet packages (don’t ask) that an update of some packages did not result in a green checkmark. Also when you open the update screen again, it again indicates there is an update for that package. Also when you try to update again it will not allow you to select projects to update since it thinks (knows) all your solution projects already have the update:

I recreated the situation in a simple solution with the following two packages. I have rigged the Json.NET package to display the above behaviour. When I update both packages one gets the green checkmark and the other doesn’t.

This is usually caused by a rogue copy of an old version of the package still in the packages folder under your solution folder:

As you can see above, there is a Newtonsoft.Json.6.0.2 folder, And also the newer installation. There are probably many reasons why this can happen (for instance the EntityFramework package asks for a restart of Visual Studio, and if you don’t do that and just continue on working on your solution it might leave the old version behind). The solution is simple, just delete the folder.

Afterwards it won’t show up in the packages that need updates anymore.

Please be aware that if you have a project under the solution root folder that is not in your solution, but does contain nuget package references, these can still reference the folder you just deleted (and prompty restore it again if you open the project).

Improving the 2nd way in knockout-winjs’s two way data binding

In the original version of the knockout-winjs sample I wrote a couple of days ago (see this post)there is a basic implementation of writing back to an observable when a control changes value. The part that does this can be found in the init method of the generic binding handler and it looks like this:

// Add event handler that will kick off changes to the observable values
// For most controls this is the "change" event
if (eventName) {
    ko.utils.registerEventHandler(element, eventName, function changed(e) {
        // Iterate over the observable properties
        for (var property in value) {
            // Check to see if they exist
            if (value.hasOwnProperty(property)) {
                // Determine if that value is a writableObservable property
                if (ko.isWriteableObservable(value[property])) {
                    // Kickoff updates 
                    value[property](control[property]);
                }
            }
        }
    });
}

You might notice that if a control has a change event (the name is control dependent and not all controls need one but in this case that name is set in the eventName variable) then we register an event handler with the element and update ALL writable observables bound to a property of the control.

This is of course not how we want to update our observables. I would prefer to update just the one that needs updating. So I introduced another field in the definition for each control’s binding handler (I named it the changedProperty). Also I won’t bind to the element’s event but to the control’s event directly. This has one issue however if you also want to be able to bind to this event explicitly.

To accomplish this I changed the above code to the following:

// Add event handler that will kick off changes to the observable values
// For most controls this is the "change" event
if (eventName) {
    // If the change event is already bound we wrap the current handler with our update routine.
    var currentEventHandler = null;
    if (control[eventName]) {
        currentEventHandler = control[eventName];
    }

    control[eventName] = (eventInfo) => {
        if (value.hasOwnProperty(changedProperty)) {
            // Determine if that value is a writableObservable property
            if (ko.isWriteableObservable(value[changedProperty])) {
                // Kickoff updates 
                value[changedProperty](control[changedProperty]);
            }
        }

        if (currentEventHandler) {
            currentEventHandler(eventInfo);
        }
    };
}

I also found out that if we want to bind two events on a single control the current implementation of binding event has a bug. It read as follows:

// After the control is created we can bind the event handlers.
for (var property in value) {
    if (value.hasOwnProperty(property) && (property.toString().substr(0, 2) === "on")) {
        control[property] = (eventInfo) => {
            value[property].bind(viewModel, viewModel, eventInfo)();
        };
    }
}

It turns out however that the ‘property’ variable is changed by the time the actual event is called. So we can’t really use it in the event. I fixed it in the following fashion:

// After the control is created we can bind the event handlers.
for (var property in value) {
    if (value.hasOwnProperty(property) && (property.toString().substr(0, 2) === "on")) {
        control[property] = (eventInfo) => {
            // Must use eventInfo.type here because 'property' will
            // be changed by the time the actual event is fired.
            value["on" + eventInfo.type].bind(viewModel, viewModel, eventInfo)();
        };
    }
}

To watch all this in action download this expanded example: ToDoApp4

WInJS Navigation and KnockoutJS, expanding the to-do sample

I was planning to expand my to-do sample with some more WinJS controls to weed out any bugs that might still be present in the knockout-winjs integration library. After thinking of some controls I would like to test out I figured I would need more than just the single page currently in the sample app. And that of course means some way to navigate between them.

WinJS comes with a navigation stack built-in, and it would be ridiculous not to use that. However a “simple” sample I found online (this one) turned out less simple than I expected. After some digging through the code however I found that for my simple purpose I could strip most of the plumbing and get something actually simple. So first we will set up some plumbing and then add a to-do item edit page.

I’m building on the outcome of my last blog post: ToDoApp2

Setting up the navigation plumbing

A few changes to the way we initialize the application:

function onDeviceReady() {
  // Handle the Cordova pause and resume events
  document.addEventListener('pause', onPause, false);
  document.addEventListener('resume', onResume, false);

  ConcreteCoding.KnockoutToWinJS.addBindings(ConcreteCoding.KnockoutToWinJS.controls);

  // When we navigate to a page, this event listener will render it, and apply our knockoutjs bindings.
  WinJS.Navigation.addEventListener("navigated", (eventObject) => {
    var url = eventObject.detail.location;
    var host = document.getElementById("pageContent");

    // Unload the previous page if needed.
    if (host.winControl && host.winControl.unload) {
      host.winControl.unload();
    }
    WinJS.Utilities.empty(host);

    eventObject.detail.setPromise(WinJS.UI.Pages.render(url, host, eventObject.detail.state).then(function () {
      ko.applyBindings(eventObject.detail.state, host);
    }));
  });

  WinJS.UI.processAll().then(() => {
    // Navigate to the initial page and supply the original view model as the state parameter.
    return WinJS.Navigation.navigate("/html/ToDoView.html", new ToDoViewModel());
  });            
}

And we also have to extract the page from the index.html into its own ToDoView.html.

Index.html:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <title>ToDoApp</title>

    <link href="WinJS/css/ui-light.css" rel="stylesheet" />
    <script src="WinJS/js/WinJS.js"></script>

    <link href="css/index.css" rel="stylesheet" />

    <script src="scripts/platformOverrides.js"></script>
    <script src="scripts/knockout-3.3.0.debug.js"></script>
    <script src="scripts/knockout-winjs.js"></script>
    <script src="scripts/index.js"></script>

    <script src="cordova.js"></script>    
</head>
<body>
    <div id="pageContent"></div>
</body>
</html>

ToDoView.html:

<!DOCTYPE html>
<html>
<head>
    <title></title>
</head>
<body>
    <input type="text" data-bind="textInput: newDescription" />
    <br />
    <div data-bind="foreach: toDoItems">
        <input type="checkbox" data-bind="checked: done" />
        <span data-bind="text: description"></span>
        <a href="#" data-bind="click: $parent.removeItem.bind($parent, $data)">x</a>
        <br />
    </div>

    <div data-bind="winAppBar: {placement: 'bottom'}">
        <button data-bind="winAppBarCommand: {label: 'Add', type: 'button', icon:'add', onclick: addNewItem }"></button>
    </div>    
</body>
</html>

Adding the new page with the navigation button

So ofcourse we create a new file containing the HTML for the new page. Very simple but using the WinJS BackButton control, because we are using the WinJS navigation stack. I also changed some properties in the ToDoItem class so we have a bit more to fill the page with.

The new HTML page:

<!DOCTYPE html>
<html>
<head>
    <title></title>
</head>
<body>
    <button data-bind="winBackButton"></button>
    <h1 class="win-type-xx-large">Edit item</h1>
    <br />
    <label>Title</label>
    <br />
    <input type="text" data-bind="textInput: toDoItem().title" />
    <br />
    <label>Done?</label>
    <input type="checkbox" data-bind="checked: toDoItem().done" />
    <br />
    <label>Description</label>
    <br />
    <textarea data-bind="textInput: toDoItem().description"></textarea>
</body>
</html>

The new view model class for the above view, and the new ToDoItem class:

class EditToDoItemViewModel {
    toDoItem: KnockoutObservable<ToDoItem>;

    constructor(item: ToDoItem) {
        this.toDoItem = ko.observable(item);
    }
}

class ToDoItem {
    title: KnockoutObservable<string>;
    description: KnockoutObservable<string>;
    done: KnockoutObservable<boolean>;

    constructor(title: string) {
        this.title = ko.observable(title);
        this.description = ko.observable("");
        this.done = ko.observable(false);
    }
}

Finally we add an editItem method that handles navigating to our new page, we place it in the ToDoViewModel class. This method should also be data bound on a button or link for each to-do item in the main page, but I’ll leave that as an exercise for the reader (or you can just download the complete project at the end of the post):

editItem(item: ToDoItem) {
    WinJS.Navigation.navigate("/html/EditToDoItemView.html", new EditToDoItemViewModel(item));
}

Now just run it, and you should be able to add items, edit them in a separate edit page, and navigate back to the original page. The complete project can be found here: ToDoApp3

Cordova + KnockoutJS + WinJS example

After my post yesterday about making KnockoutJS and WinJS work together I figured it would be a good idea to make a nice sample application to show everything works. And why not user Cordova in the process.

So here is a very simple ToDo sample I created for a knowledge sharing session at the office. Which I will ‘enhance’ with WinJS stuff: ToDoApp1

It is very simple, and it doesn’t even save your to-do items but it shows off some knockout JS stuff, and that was the point.

What I did was change the Add button to a button on a WinJS AppBar and bind the onclick handler to the exact same method that was used before. Apart from adding the WinJS nuget package, the WinJS typescript definition files and the knockout-winjs library from my previous post to the project (and referenced in the HTML file), I only changed a few simple things:

Added this piece of HTML at the end of the body:

<div data-bind="winAppBar: {placement: 'bottom'}">
    <button data-bind="winAppBarCommand: {label: 'Add', type: 'button', icon:'add', onclick: addNewItem }"></button>
</div>

Changed the onDeviceReady method to register the WinJS control binding handlers and add the WinJS.UI.processAll method to start the WinJS magic:

function onDeviceReady() {
  // Handle the Cordova pause and resume events
  document.addEventListener('pause', onPause, false);
  document.addEventListener('resume', onResume, false);

  ConcreteCoding.KnockoutToWinJS.addBindings(ConcreteCoding.KnockoutToWinJS.controls);

  WinJS.UI.processAll().then(() => {
    ko.applyBindings(new ToDoViewModel());
  });            
}

I also added a little padding to the body in CSS because the WinJS style sheets put everything frighteningly close to the edges of the screen. But I’ll leave that to your imagination. Or you can download the entire project from here: ToDoApp2

I’ll try to add some more WinJS controls to this little sample to test them out (and if needed fix the knockout-winjs library in the process). So keep an eye out.

Making KnockoutJS and WinJS work together

We are building some Cordova apps for iOS, Android and Windows Phone. And we had everything figured out except for the UI part. We had already chosen Knockout in many of our other projects to facilitate data binding between UI and ViewModel. So when it was time to look into some UI solutions we came across WinJS. I am a huge fan of the Windows Phone UI (whatever it is called nowadays, Metro or Modern… who knows) so WinJS was my prefered choice.

Even more so when I saw a Cordova tutorial video on Microsoft where they used WinJS, and where they did not show but casually mention WinJS and KnockoutJS worked well together using a community library. The library turned out to be this one.

It sounded great, it looked like it could work, and if you would try some simple stuff, it really worked. There are a few snags though:

  • Events don’t work well…. or at all… for many of the controls.
  • No updates since November 2013, so no hope of those bugs getting fixed.
  • No NuGet package (ok, thats a minor one).

So after we found out we only needed 2 controls from WinJS and just the CSS for the rest of the UI we decided to make some dedicated WinJS binding handlers for these controls. However it kept nagging me and I decided to do something about it. The rest of this post will document my findings.

First of all, I converted the original JS file to TypeScript, this makes my life so much easier since I am not really a javascript developer and I am used to C#’s type safety.

How does the original library work?

The original library dynamically creates a list of binding handlers for each WinJS control in a definition list:

function addBindings(controls) {
  Object.keys(controls).forEach(function (name) {
    var controlConfiguration = controls[name];
    var ctor = WinJS.Utilities.getMember(name);
    var eventName = controlConfiguration.event
    var propertyProcessor = controlConfiguration.propertyProcessor;
    var bindDescendants = controlConfiguration.bindDescendants || false;
    var bindingName = "win" + name.substr(name.lastIndexOf(".") + 1);

    ko.bindingHandlers[bindingName] = {
      init: function (element, valueAccessor, allBindings, viewModel, bindingContext) { 
        // generic init stuff here
      },
      update: function (element, valueAccessor, allBindings, viewModel, bindingContext) {
        // generic update stuff here
      }
    }
  });
}

var controls = {
  "WinJS.UI.AppBar": {
    bindDescendants: true
  },
  "WinJS.UI.AppBarCommand": {},
  "WinJS.UI.BackButton": {},
  "WinJS.UI.DatePicker": {
    event: "change"
  }
  // etc. for more controls.
}

addBindings(controls);

This code works great for most simple controls. And the author took some good steps to get the more complicated controls (like the ListView control) working. However binding the events for these somewhat more complicated controls is where things went buggy.

The main problem being that the current viewmodel is not passed to the event handler. When you click an AppBarCommand you would expect it to work as the knockout click handler and pass in the current viewmodel (more info on event binding in knockout here).

The improved way of handling events

The following updated version of the init and update methods of the generic binding handler implementation will give a better event handling experience. I have not yet thoroughly tested it though, and the solution for detecting events is a bit hacky (to put it mildly), so any suggestions there are always welcome.

The init method:

init: function (element, valueAccessor, allBindings, viewModel, bindingContext) {
  // The options for the control
  var value = valueAccessor();

  // Options record for the WinJS Control
  var options = {};

  // Iterate over the observable properties to get their value
  for (var property in value) {
    // Don't parse properties starting with "on" since they are event handlers and should be treated differently.
    if (value.hasOwnProperty(property) && (property.toString().substr(0, 2) != "on")) {
      if (propertyProcessor && propertyProcessor[property] !== undefined) {
        options[property] = propertyProcessor[property](value[property], function () { return element });
      } else {
        options[property] = ko.unwrap(value[property]);
      }
    }
  }

  // If the WinJS control depends on having child elements 
  if (element.children.length > 0 && bindDescendants) {
    // This is done synchronously
    // @TODO: Determine if this could be done async
    ko.applyBindingsToDescendants(bindingContext, element);
  }
                     
  // Create a new instance of the control with the element and options
  var control = new ctor(element, options);

  // After the control is created we can bind the event handlers.
  for (var property in value) {
    if (value.hasOwnProperty(property) && (property.toString().substr(0, 2) === "on")) {
      control[property] = (eventInfo) => {
        value[property].bind(viewModel, viewModel, eventInfo)();
      };
    }
  }

  // Add event handler that will kick off changes to the observable values
  // For most controls this is the "change" event
  if (eventName) {
    ko.utils.registerEventHandler(element, eventName, function changed(e) {
      // Iterate over the observable properties
      for (var property in value) {
        // Check to see if they exist
        if (value.hasOwnProperty(property)) {
          // Determine if that value is a writableObservable property
          if (ko.isWriteableObservable(value[property])) {
            // Kickoff updates 
            value[property](control[property]);
          }
        }
      }
    });
  }

  // Add disposal callback to dispose the WinJS control when it's not needed anymore
  ko.utils.domNodeDisposal.addDisposeCallback(element, function (e) {
    if (element.winControl) {
      element.winControl.dispose();
    }
  });

  return { controlsDescendantBindings: bindDescendants };
}

The update method:

update: function (element, valueAccessor, allBindings, viewModel, bindingContext) {
  // Get the WinJS control 
  var control = element.winControl;
  var value = valueAccessor();

  // Only update the control properties that are different with the unpacked value
  for (var property in value) {
    if (value.hasOwnProperty(property)) {
      if (property.toString().substr(0, 2) != "on") {
        var unwrappedValue = ko.unwrap(value[property]);
        if (control[property] !== unwrappedValue) {
          if (propertyProcessor && propertyProcessor[property] !== undefined) {
            var returnValue = propertyProcessor[property](value[property], 
              function () { return element }, control[property]);
            if (returnValue !== null) {
              control[property] = returnValue;
            }
          } else {
            control[property] = unwrappedValue;
          }
        }
      } else {
        // I think we are fine here if we just override the
        // event handler even if it may not have changed at all.
        control[property] = (eventInfo) => {
          value[property].bind(viewModel, viewModel, eventInfo)();
        };
      }
    }
  }
}

The complete file can be found knockout-winjs.

I will try to fix more stuff and add more features to the knockout-winjs “connector”. Maybe even update it on a fork on github and create a nuget package for it. Who knows….

Find all string literals in C# code files, but not the ones in comments

When trying to find string literals to translate in our rather large 59 project solution I found out it is not that easy to search for string literals in your code when you use the default code comments format.

You see, there is a default regular expression in Visual Studio which lets you search for string literals (eg. search for all strings that are between quotes). This regular expression is:

((".+?")|('.+?'))

This regular expression however will also find entries in code comments like this:

/// <summary>
/// Begins to invoke the action in the current controller context.
/// </summary>
/// <param name="callback">The callback.</param>
/// <param name="state">The state.</param>
/// <returns>
/// Returns an IAsyncController instance.
/// </returns>

It took me and a co-worker some time to come up with a regular expression to filter these out. Our result was:

(?=(^((?!///).)*$)).*((".+?")|('.+?')).*

I’m not entirely sure myself how it works. But it is based on the fact that we only want to know the entire line which contains the string literal (and not specifically only the string literal). So we match all lines that don’t start with whitespace followed by ///, then in those lines match the original regex.

OAuth configuration URL’s and general information

Last couple of weeks I have been working on a new implementation of the login and security stuff in our new MVC application. We wanted to use 3 external OAuth Authentication providers (LinkedIn, Microsoft and Google). I noticed I had to look up application configuration links from samples and tutorials on the internet, since they were hard to find when actually logged on as a user for each of these services. So for easy reference, here are the URLs:

Microsoft: https://account.live.com/developers/applications/index
Google: https://console.developers.google.com/project
LinkedIn: https://www.linkedin.com/secure/developer

Another thing of note are the strange things you request from a user’s Google account when using OAuth. See for instance these stackoverflow questions: http://stackoverflow.com/questions/18329629/scope-to-get-email-address-alone
http://stackoverflow.com/questions/24410179/is-there-a-way-to-only-get-a-users-email-address-with-googles-oauth2-impleme

Enabling TypeScript builds for library projects in Visual Studio.

If you are creating a modular web application where all content files and script files are embedded in library projects, migrating from JavaScript to TypeScript will need some extra work. As I already discussed in a previous post, debugging TypeScript might be a bit tricky, and I already mentioned there that we have our TypeScript in library assemblies.

I did not mention how we enabled TypeScript build support for these assemblies (since it will not work out of the box for this project type). So here is a small instruction manual:

– Unload the project you want to add TypeScript support to.
– Open the project file in the XML editor.
– Add the following at the end of the file (but before the Project end tag):

<Import Project="$(MSBuildExtensionsPath32)MicrosoftVisualStudiov12.0TypeScriptMicrosoft.TypeScript.targets" />

– Save and close the project file, then load the project again.

The build will now work. However it will likely have problems with overwriting the generated .js files. You probably have those file in source control because you want them added as an Embedded Resource to your output assembly. And if they are in source control they will be read-only by default.

To fix this you can add a pre-build event to your project that clears the read-only attribute (and since we also have the .js.map files are Embedded Resources we fix these up too):

attrib -r /s "$(ProjectDir)*.js"
attrib -r /s "$(ProjectDir)*.js.map"

It is probably possible to use the tf command line tool to actually check these files out instead of changing just the read-only stamps, but the above works fine for us.

Debugging typescript when they are built into a class library.

We have created a modular MVC website where each module is built in a class library. There are plenty of examples on the web on how to do this. However when we decided to convert our Javascript files to TypeScript we ran into a few issues regarding debugging.

If you put a breakpoint into a TypeScript file in a regular project, Visual Studio will automatically pick it up (using the running javascript, and the mapping file generated by typescript). If you are using a custom VirtualPathProvider to retrieve the js files from resources in a class library things get a bit harder to debug.

To accomodate this scenario there are a few things we must do:

– Just as the javascript files we need to mark the js.map files as embedded resource too.
– Just as the javascript files we need to mark the TypeScript files as embedded resource too.
– The above file types must be retrieved via the custom VirtualPathProvider.
– Because we need to mark the TypeScript files as embedded resources, and because file can only have one build action, we also need a good way to compile all typescript files.

The first two points are obviously the easy parts, just select all .ts and .js.map files in your class library project, and set the build action to Embedded Resource.

|To have the custom VirtualPathProvider pick up those file types, you will have to change the web.config file of your MVC host project. Something like the following line should already be present:

<add verb="GET" path="*.js" name="Static for js" type="System.Web.StaticFileHandler" />

Add the following two lines just below that:

<add verb="GET" path="*.ts" name="Static for ts" type="System.Web.StaticFileHandler" />
<add verb="GET" path="*.js.map" name="Static for js.map" type="System.Web.StaticFileHandler" />

Now for the final bit. Since we set the TypeScript files to EmbeddedResource they won’t compile to a javascript file anymore when you build the project. To counteract this, add a typescript file to your project that just references all your other typescript files like so:

/// <reference path="My1stFile.ts" />
/// <reference path="My2ndFile.ts" />

Keep this file set to the regular build action. And on build, this file, and all files it references will get built.