By Ugo Lattanzi on July 9th , 2014 in nodejs | comments

Yesterday I spent some time to understand a problem with a Node Js application on Microsoft Azure. To be quick, my code was a simple web application built on top of Express and the code was something like this:

var express = require("express");
var app = express();

.... configure express

var port = Number(process.env.port || 5000);

app.listen(port, function() {
    logger.info("Listening on " + port);
});

Running locally the code works very well both if you run it on your dev environment or a server. Unfortunately it doesn't work if you try to run it on Microsoft Azure Website. But Why?

Adding some log I identified the problem on the port environment, basically process.env.port returns a string instead of a number (to be precise it was \\.\pipe\e289ed7e-b57b-46bb-8bba-ad8cd1f1529c) f The solution is easy, do not try to convert it to a number but pass it as is to node:

var port = process.env.port || 5000;

app.listen(port, function() {
    logger.info("Listening on " + port);
});

The reason is that Node is not running on its process like on local machine (node app.js to be clear), but it's mapped under IIS using IISNode with a wildcard on an HTTP Handler using named pipe

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <system.webServer>         
      <handlers>
           <add name="iisnode" path="app.js" verb="*" modules="iisnode"/>
     </handlers>
      <rewrite>
           <rules>
                <rule name="StaticContent">
                     <action type="Rewrite" url="public{REQUEST_URI}"/>
                </rule>
                <rule name="DynamicContent">
                     <conditions>
                          <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="True"/>
                     </conditions>
                     <action type="Rewrite" url="app.js"/>
                </rule>
           </rules>
      </rewrite>
    <iisnode 
      debuggingEnabled="false"
      logDirectory="..\..\LogFiles\nodejs" 
      watchedFiles="*.js;iisnode.yml;node_modules\*;views\*.jade;views\*.ejb;routes\*.js;views\*.vash" />
   </system.webServer>
 </configuration>

For confirmation, I wrote to David Ebbo via twitter getting this answer:

Confirmation

Unfortunately (or not), right now there is no way to run Node outside of IIS on Azure Websites but maybe it's not a problem, it works :smirk:

By Ugo Lattanzi on June 30th , 2014 in .net | comments

The idea of this post is born talking with my colleague Antonio about the frameworks a .NET web developer should know (from my point of view of course). That's funny because the list is very long and I asked myself if there is something wrong with my idea of .NET web developer or something else.

From my point of view (and also for my company) there are two kind of web developers, Back End (server side code) and Front End (javascript, css and HTML). Unfortunately, in Italy, these figures matches in only one person and the quality of the product obviously falls.


Back End Developer


ASP.NET MVC

Website: http://www.asp.net/mvc
Source code: http://aspnetwebstack.codeplex.com/

Most of our application runs on top of ASP.NET MVC, so for me it's really important to know it. I think it's a good framework, with a lot of extensibility points, good implementation of the MVC pattern, and we use it in heavy traffic projects without particular problems. Of course there are few things I don't like of it (first of all System.Web), but fortunately the vNext will solve these "problems".


ASP.NET WEBAPI

Website: http://www.asp.net/web-api
Source code: http://aspnetwebstack.codeplex.com/

ASP.NET Web API is probably the best solution if you know ASP.NET MVC and you don't have time to learn something else like NancyFx, NodeJs and so on; The approach is very similar to MVC (Controller + Action). Fortunately it doesn't have the dependency to System.Web but it's a Framework totally separate from MVC and sometime you have to duplicate the same code on MVC and Web API because the same interface/class has a different namespace.


SignalR

Website: http://www.asp.net/signalr
Source code: https://github.com/SignalR/SignalR

I'm not sure if an another Framework for real time applications exists in .NET world, but surely SignalR it is the most famous and used. Build by the same team of ASP.NET MVC / Web API it offers several clients (iOS and Android with Xamarin, Windows 8 and Windows Phone) and finally it supports old browsers (with fallback of course forever-frame, polling and son on).


NancyFX

Website: http://nancyfx.org/
Source code: https://github.com/NancyFx/Nancy

Nancy is a lightweight framework for building HTTP based services on .Net and Mono (yes it runs on linux and OSX). The main difference between Nancy and Web API is the routing approach. It uses lambdas to identify relative paths and arguments. Really helpful if you can't deploy on a Windows Server.


Common Logging

Website: http://netcommon.sourceforge.net/
Source code: https://github.com/net-commons/common-logging

I really like this library. When I need to deploy my code side by side with another application or I have to use a specific logging framework, Common Logging is the perfect solution. It is an abstraction of different logging implementations like Log4net, NLog, Enterprise library or whatever you want (you can write your custom bridge). Like many frameworks in the .NET world, this is a porting of a Java Framework (here more info). Really useful!


Windsor Container

Website: http://www.castleproject.org/
Source code: https://github.com/castleproject/Windsor

Probably the first package I add in a new project. I'm really a Dependency Injection addicted and Castle Windsor fits very well with my needs. It's fast, easy to use, all needed lifecycle and offers lot of extension point (Interceptor, custom lifecycle, factories and so on).


Automapper

Website: http://automapper.org/
Source code: https://github.com/AutoMapper/AutoMapper

In my Italian blog I wrote about the importance to use a DTO for the views and the responses instead of the Domain Model. Automapper is absolutely the best framework to "copy" data from an entity to a DTO. Easy to use, fast and extensible it's the second package I install on a new project. A must.


Service Stack

Website: https://servicestack.net/
Source code: https://github.com/ServiceStack

An extremely interesting set of Frameworks. It contains a Json serializer, ORM, Redis client and Service Clients. This set of Frameworks matches perfectly with those who are obsessed with performance. The tagline of the Framework is "Simplicity at Speed". Here a good presentation about ServiceStack and performances in .NET application


Quartz.NET

Website: http://www.quartz-scheduler.net/
Source code: https://github.com/quartznet/quartznet

Quartz.NET is a job scheduling system for small or large applications. Like Common Logging, this is a porting from a Java project (here more info). It offers several ways to run a job, from Cron pattern to special calendar, or whatever you like. The nice thing is you can have a storage for your jobs (configurable SQL, Mongo, MySql .....) very useful for scalable applications.


Cache Cow

Source code: https://github.com/aliostad/CacheCow

Caching is really important, specially if you application must answer to lot of requests. The best way to keep performance acceptable is to reduce the number of operation, specially if request and response are the same for most of the total requests. Cache Cow is a Framework built by my Twitter friend @aliostad and it offers an easy way to cache HTTP requests (both from client and server) using WEB API. With few line of code, you can have a good caching in your favorite storage (Redis, Azure Caching, Sql Server and so on).


Redis

Website: http://redis.io/
Source code: https://github.com/antirez/redis

Redis is an open source caching Framework that offers an advanced Dictionary (key/value) storage. Recently it's available (as preview) also on Windows Azure (here a good article explains how to use redis with MVC and Azure). The Performance of this Framework are great: it is very fast, and it also available on distributed infrastructures. If you go in a multi-server application, probably it is the best solution.


XUnit

Source code: https://github.com/xunit/xunit

This is the most active testing framework for .NET applications. It's used on lot of the Frameworks mentioned in this post (MS Stack included). It has support for Resharper, CodeRush Test Runner and Xamarin Test Runner.


Autofixture

Source code: https://github.com/AutoFixture/AutoFixture

It's a framework that helps developers to do Test-Driven Development by automating non-relevant Test Fixture Setup. Really I'm not a fan of TDD but Autofixture contains several features like Automock (helpful if you change frequently the constructor dependencies) and AutoMoqData that can help all developers.


Sharp Tests Ex

Source code: http://sharptestex.codeplex.com/

It's a library born to wrap all testing framework using a fluent syntax. Usually I don't change often the testing framework but sometime I need to copy part or my code to an existing application that uses NUnit or MS-Test. In this case the only thing to do is change the Testing attribute in the test class.

Front End Developer


Sass

Website: http://sass-lang.com/
Source code: https://github.com/sass/sass

Sass (Syntactically Awesome Style Sheets) is an extension to CSS. It is CSS as it should have been. Its key features are the ability to use variables, nesting, mixins and loops within your code. This means you can code more quickly and keep your code neat, tidy and easy to maintain. The Sass or SCSS code you write is then compiled in to standard CSS as browsers can't (yet) understand Sass / SCSS.

Also take a look at Compass, it is a great framework for Sass that contains loads of reusable mixins... no more memorising vendor prefixes and obscure pre-spec CSS3 styles!


Bootstrap

Website: http://getbootstrap.com/
Source code: https://github.com/twbs/bootstrap

Created by the guys at Twitter, Bootstrap gives you a massive selection of reusable, robust and attractive styles for your everyday styling needs. It includes a responsive, mobile-first grid system, basic typography styles, styles for common elements such as buttons and form inputs and lots more. Bootstrap is perfect for rapid prototyping but don't use it for everything otherwise we'll end up with all sites looking the same!


Bower

Website: http://bower.io/

Bower is a front-end package management tool, you use it to speed up your dev workflow. It allows you to simply install packages and their dependencies in your project using the command-line. No more Googling for the latest version of jQuery, downloading it, unzipping it, copying it into your project etc... just: $ bower install jquery


Grunt

Website: http://gruntjs.com/
Source code: https://github.com/gruntjs/

Like Bower, Grunt is another dev tool that can be run from the command-line. It is a JavaScript task runner which allows you to automate pretty much anything... minification, unit testing, compiling of Sass, code linting, image compression, launching a node server, creation of documentation, whatever you like. There are many, many Grunt Plugins available which do the most common tasks so it is easy to get started.


Yeoman

Website: http://yeoman.io/
Source code: https://github.com/yeoman/yeoman

Yeoman makes using Grunt and Bower even easier. It allows you to scaffold out a project very quickly using a "Generator". The Generator will create a bare-bones (boilerplate) project architecture, with certain libraries, frameworks, Grunt tasks and Bower dependencies pre-installed. Different Generators are available for different projects. For example if you are starting a new AngularJS app, you would use the Angular Generator and run $ yo angular and it will set up the architecture, along with basic units tests, install AngularJS and Bootstrap (if you want it).


AngularJS

Website: https://angularjs.org/
Source code: https://github.com/angular/angular.js

AngularJS is an open-source MVC JavaScript framework created by Google - it allows you to very quickly extend HTML's capabilities and create powerful, highly testable web applications. Angular includes two-way declarative data-binding which greatly simplifies complex application development as much of the DOM manipulation is handled automatically by the framework. It is (fairly) well documented and there is an active support community on Stack Overflow.


Karma

Website: http://karma-runner.github.io/
Source code: https://github.com/karma-runner/karma/

Karma (formerly Testacular) is a framework agnostic test runner. You write your unit tests alongside your application code and you can automatically test your code as you develop. Karma allows you to test your code in real browsers on real devices or in PhantomJS. It is definitely worth watching the introduction video from creator Vojta Jína.


Jasmine

Website: http://jasmine.github.io/
Source code: https://github.com/pivotal/jasmine

Jasmine is a framework for testing JavaScript code. You can use Jasmine to write your unit tests and then run them using Karma. The syntax of Jasmine is very clear and easy to understand, yet powerful.

If you want to take things further take a look at Jest (by Facebook), it is built on top of Jasmine, and adds some additional levels to Jasmine's feature-set.


Conclusion

It's an hard life for our developers.

Thanks to my friend Daniel Crisp for the support in this post

By Ugo Lattanzi on June 23rd , 2014 in nodejs | comments

As I wrote in my previous post here, Node Js is becoming a part of my dev life and today I'm gonna write about logging.

Every good application must have a good logging and NodeJs, as all Frameworks, offers several ways to save information.

Unfortunately the most used is the classic console.log method that's a quick and dirty solution. For all people like me that usually use a robust Framework like Log4Net or NLog, console.log doesn't fit so well with my requirements.

All these Frameworks offer the opportunity to add more than one Appender to the same logger instance.

What's an appender?

Basically It's a simple way to have more than one output during logging. To be clearer let's try to think about an application where you want to see your logging in the console, but also in a file or an external service like Raygun.

In my Node sample repository I created a demo of a simple web page (using Express) configuring the web server with the most needed middleware and some logs.

The result of the log is this:

ConsoleLog

As you can see there are just few lines of log but, when you do something more complex, the number of lines could be a lot and difficult to read. The problem here is that lot of them are only debug log but some of them could be errors. Using the same color is difficult to understand what is the error and what is not.

A good solution is to use a logging framework that helps us to log into the console using different colors (red for errors, yellow for warnings and so on) and, in production, switch the log to a file or database.

Winston is the equivalent to Log4Net/Log4J/NLog in a NodeJs world. It offers the opportunity to use the Appenders and, in our case, colored console.

npm install winston --save

now it's enough to configure it. I've logger.js with its configuration

var winston = require('winston');
winston.emitErrs = true;

var logger = new winston.Logger({
    transports: [
        new winston.transports.File({
            level: 'info',
            filename: './logs/all-logs.log',
            handleExceptions: true,
            json: true,
            maxsize: 5242880, //5MB
            maxFiles: 5,
            colorize: false
        }),
        new winston.transports.Console({
            level: 'debug',
            handleExceptions: true,
            json: false,
            colorize: true
        })
    ],
    exitOnError: false
});

module.exports = logger;
module.exports.stream = {
    write: function(message, encoding){
        logger.info(message);
    }
};

The most important thing is the transports section where you can specify your Appenders. In this example, I want to log into a file with verbosity level set to info, max 5 files and 5 MB for each and I want a complete full log (verbosity level debug) in the terminal but the different level should use different colors.

The result is pretty nice:

ConsoleLog

and

ConsoleLog

for the file.

I think that's absolutely more readable if you have different colors in the console app. Another cool thing is that you can switch on/off some logging just changing the level property in the transport configuration.

In my example, I used Express as MVC framework to render HTML. It offers the opportunity to put you log to have some info about the HTTP Requests.

So, here is the code:

var logger = require("../utils/logger");

var express = require("express");
var app = express();

logger.debug("Overriding 'Express' logger");
app.use(require('morgan')({ "stream": logger.stream }));

The only importat thing here is the middleware, app.use where logger.stream comes from the logger configuration file.

All my code is available on my Node Sample github repository available here

Express 4 moved some middleware outside of Express packages, so you have to install it manully (more info here). If you are using an older version of Express my code needs some changes because of middleware.

By Ugo Lattanzi on June 16th , 2014 in azure | comments

In my company we switched from Team Foundation Server to Github more than one year ago and we are really happy. In the same period we switched all our server to Windows Azure (Virtual Machine, Web Sites and Cloud Services) and we are equally happy.

The Azure team works very well and I love to deploy my web sites using Github (there are sevaral providers so, if you don't use github don't worry, you'll find the solution that fits best your needs).

If you are intersted in how to push your code to Azure website using Git, here there is a cool post and here there's also an engine used for the deploy based on NodeJs

The problem about this approach is that I need to be notified when someone of the team deploy something, especially if the deploy fails. Unfortunately right now Windows Azure doesn't send you any notification about a deploy procedure, so you have to remind yourself to go in the administration portal and check the deploy status (see the image below).

Windows Azure Management Portal

Zapier is the solution to this problem. It allows you to select from many sources, to select an event and to attach an action. In my case it means something like "For each failed Azure Website deploy call my phone and read me the report" (You can get a email, sms or whatever you want).

Here the workflow to configure Zapier:

Select the services (here the complete list of the available services)

Windows Azure Management Portal

Write your condition

Windows Azure Management Portal

Compose the message

Windows Azure Management Portal

How f...ing cool is that?

By Ugo Lattanzi on June 3rd , 2014 in nodejs | comments

In the last period I'm working a lot with Node JS and, for a developer like me who loves C# and .NET, I'm still not sure I really love Node :confused:

The first impression is positive but I don't know if it's just because I'm playing with something new or if I really like the approach; however, for the moment, now I'm happy with it.

This friday I was explaining my experience with Node to my colleague speaking about the differences between Node and .NET.

From my point of view the biggest difference among .NET and Node is the async implementation. The first one is MTA (multi thread application) and the second one is STA (single thread application).

STA means you can't open a thread to execute something so your application can manage only one request per time.

Ok, that's absolutely fault because Node is smart :smile:

It's true that Node is STA but it is async by default, in fact when you do something that goes outside of your application (database query, I/O, web request and so on) Node uses the thread to do other stuff like answer to another request, execute other code in you application and so on.

That means Node is really really optimised, in fact the performance among .NET and Node JS are very similar and, in several case, Node is faster than .NET.

But how to increase the number of process with Node so that you can have a good scaling system?

In the .NET world you can find something similar is ASP.NET (hosted on IIS) and it's called "web garden", in Node instead it's called Cluster. Basically there are more than one active process and a "manager".

In that scenario you can use one process for each core of you computer, so your Node application can scale better with you hardware.

Basically it's like running 'node app.js' for each core you have, and another process to manage them all.

First step, install some packages:

npm install cluster --save

The goal of this example is to create one process for each core, so the first thing to do is to read the number of cores installed on your laptop:

var cluster = require('cluster');

if (cluster.isMaster) {
  var numCPUs = require('os').cpus().length;
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  Object.keys(cluster.workers).forEach(function(id) {
    console.log(cluster.workers[id].process.pid);
  });
}

cluster.isMaster is necessary to be sure that you are forking it just one time.

Now if you run the app you should have one process for each core, plus the master

image

In my case I've 8 core, so 9 process because of master.

The next step is to add a webserver, so

npm install http --save

and put your logic for each fork:

var cluster = require('cluster');
var http = require('http');

if (cluster.isMaster) {
  var numCPUs = require('os').cpus().length;
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  Object.keys(cluster.workers).forEach(function(id) {
    console.log(cluster.workers[id].process.pid);
  });
} else{

  // Create HTTP server.
  http.Server(function(req, res) {
    res.writeHead(200);
    res.end("This answer comes from the process " + process.pid);

  }).listen(8080);
}

Now calling the webserver you can see which process is answering your request:

image

Because the code is too simple, probably you'll get the same 'pid' for each request from your browser. The easier way to test it is to lock the thread (yes, I said that) so the "balancer" can switch the request to another process demonstrating the cluster.

In Node there isn't something like Thread.Sleep, so the best way to lock a thread is create something that keeps it busy, something like an infinite loop :smirk:

var cluster = require('cluster');
var http = require('http');

if (cluster.isMaster) {
  var numCPUs = require('os').cpus().length;
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  Object.keys(cluster.workers).forEach(function(id) {
    console.log(cluster.workers[id].process.pid);
  });
} else{

  // Create HTTP server.
  http.Server(function(req, res) {
    res.writeHead(200);
    res.end("This answer comes from the process " + process.pid);

    //that's just for example
    while(true){

    }

  }).listen(8080);
}

If you want to manage all the processes and to log some events, it could be helpful to track some events for each process and to send a message from the "worker" to the "master" or to check when a process dies.

To do that it's necessary to use message event on the worker, so here's the code:

var cluster = require('cluster');
var http = require('http');

if (cluster.isMaster) {

  console.log("Master pid: " + process.pid);

  var numberOfRequests = 0;

  var numCPUs = require('os').cpus().length;
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  Object.keys(cluster.workers).forEach(function(id) {
    console.log('creating process with id = ' + cluster.workers[id].process.pid);

    //getting message
    cluster.workers[id].on('message', function messageHandler(msg) {
      if (msg.cmd && msg.cmd == 'notifyRequest') {
        numberOfRequests += 1;
      }

      console.log("Getting message from process : ", msg.procId);
    });

    //Getting worker online
    cluster.workers[id].on('online', function online()
    {
      console.log("Worker pid: " + cluster.workers[id].process.pid + " is online");
    });

    //printing the listening port
    cluster.workers[id].on('listening', function online(address)
    {
      console.log("Listening on port + " , address.port);
    });

    //Catching errors
    cluster.workers[id].on('exit', function(code, signal) {
      if( signal ) {
        console.log("worker was killed by signal: "+signal);
      } else if( code !== 0 ) {
        console.log("worker exited with error code: "+code);
      } else {
        console.log("worker success!");
      }
    });
  });

  //Printing number of requests
  setInterval(function(){
    console.log("Handled " + numberOfRequests + " requests");
  }, 3000);

} else {

  // Create HTTP server.
  http.Server(function(req, res) {
    res.writeHead(200);
    res.end("This answer comes from the process " + process.pid);

    console.log("Message sent from http server");

    // Notify master about the request
    process.send({ cmd: 'notifyRequest', procId : process.pid });


  }).listen(8080);
}

I've created a repository on github with some demos about Node.js (including this one). You can find the repository here.

Have fun.