Json.Net Issue: does not support namespace in XML

Yesterday I had a talk to a developer at work. She came from the same city as my hometown where I was grown up. She was working on a project to import thousands of XML files SQL database. I recommended my dotNetCommonLibrary.dll, where I have a wrapper class for Json.Net.  The wrapper class provides several APIs to convert XML string to an instance of a class and vice versa.  I have used this feature to convert many application XML configuration settings to instances.

However, when I saw her XML files, I noticed that those XML files contains namespaces, which are used in XML nodes as prefix. I had never tried those types of XML documents before. Today I tried a simple XML file with a namespace for testing. Unfortunately, it looks like that Json.Net converter does not support XML with namespaces.

Example of XML with namespaces:

<?xml version="1.0" encoding="UTF-8" ?>
<Root>
  <b:Log xmlns='http://www.w3.org/1999/xhtml' xmlns:b='http://www.google.com/2005/gml/b' >
     ...
  </b:Log>
</Root>

I think this type XML files are very common. I have to find out a way to remove namespaces and use only local names in XML notes for instance mapping by Json.Net

It was a good talk. At least I find my wrapper class has to provide an API to remove namespaces from an XML string.

References

• Json.NET
• SO: What does XMLNS in XML mean?
• SO: Creating XML Elements without namesake declarations
• SO: How do I remove namespaces from XML, using Java dom?
• SO: XSLT Transform XML with Namespaces
• MSDN article: Using XSLT in .Net

Read More...

Collapse...

CoffeeScript and .Net Nancy Framework

Last week when I listened to my podcasts, two projects got my attention. The firs one is from HerdingCode podcast 114 about: Trevor Burnham on CoffeeScript. This is light language in clean syntax that can be compiled into JavaScripts, in one to one relationship.

The second one is from HansellMinutes on 6/16/2011: Nancy, Sinatra and Exposion of .Net Micro Web Framewor. The Nancy project was introduced on Nov 28th, 2010. The blog site Elegent Code posted a blog about this podcast as well on Jun 28, 2011.

Read More...

Collapse...

Create Shortcut in C#

Last week I was working on a case to provide support for an application users. The application has two executables: one is app's main exe like app.exe and another one is for app's update like appUpdate.exe. For this update, we need to add a shortcut on clients machine desktop pointed to appUpdate.exe.

Like a typical shortcut, the one I need to put on client PCs has several settings: target, start in, run mode, icon and comments.



I tried to create a shortcut on my box first. Aftert that, I tried to copy this shortcut file (lnk) to a remote PC. It did not work. It seems that the lnk file some information are embedded in the file. When it is not in the machine where it was created. The machine name appears in the target.

I need a way to create a shortcut on fly. That's why I went to .net to create a simple console app do the job. This time SO helped me with an excellent open source library from a QA: ShellLink.cs. Within the source codes, there are actually two files I need: ShellLink.cs and FileIcon.cs. The library contains a wrapper class for Windows APIs. It is very straight forward.

The only change I made is to add a class level flag for any changes in shortcut properties. If any setter changes a value, this flag is true. Then I'll only save chances for the shortcut if any of properties are changed.

public class ShellLink : IDisposable {
  private bool mChanged = false;
  ....
  public string Target {
    get { .... }
    set { if (!value.Equals(Target)) {
      mChanged = true;
      ....
    }
    ....
}

Read More...

Collapse...

Pex and Code Contrat

Last week I found two new .Net libraries from Microsoft Research: Pex and Code Contract. Both are right now available as review versions for Visual Studio 2008 Professional version and will be part of .Net framework 4.0 and Visual Studio 2010.

Pex is a library with tools for unit test. It looks very handy and easy to use. It reduces a lot of tedious codes to create test with data. The Pex Exploration integrated with VS 2005 will be able to generate tests with all the data to cover up to 100% of codes. Pex is smart enough to search all the possible branches with data and tests. I remember that I had created a lots of tests to cover all the possible cases and it is very tedious to find branches and repeated uncountable times. The biggest issue to maintain unit tests is that when source codes are changed, the coverage will changed again. In order to cover new cases or update tests, re-examinate source codes and tests need a lots effort. Pex looks much better to handle all those cases. That's really awesome.

In additional to auto-generate test codes, Pex also cover mocking data cases. Actually, I found Pex from the resent DNRTV show: White Box Tesing with Pex, where mocking demo is shown there simulate reading file case. I'll be very interested in trying this great library.

Another very interesting library is Code Contract library. This is really cool stuff I have been looking for. Basically, it is used to define contract for class members, method parameters and method return expectations. The contract is used as static class within .Net codes. Within VS, you can enable is static checking and run-time checking. Any violations will generate exceptions as expected by Contract. In addition to that, it will generate code documentation as well. From its syntax, it is not too complicated but makes the coding much easier. This is definitely a great enhancement to .Net 4.0 and VS 2008 and coming VS 2010.

Read More...

Collapse...

RESTful Service Start Kit

I have been very interested in RESTful services, since it is a service based on HTTP Get, Put and Post protocol. Basically, you can use this service by using URL to provide your resources to end users with simple web protocol service. One way to explore the service is by using a web browser, which is URL based application to handle resources such as HTML and XML.

I have watched Pluralsight Screencast demos on Microsoft WCF REST Startkit. The Start Kit project was launched last year with Preview 1. Now it is in the stage of Preview 2. As MSD's artical: A Developer's Guid to WCF REST Start Kit, this new framework will be in the future versions of the .Net Framework.

However, what I tried the Preview 2 last week, I realized that some of template projects behavior differently from the version of Preview 1, as I saw in the Screencast. For example, one class of Service.basic.svc.cs is missing. From the class of Service.svc.cs I can see base class such CollectionServiceBase and ICollectionService but they are not available for editing. This makes it impossible to customized help descriptions, implementations and templates. Not sure if all these changes are available from other alternative classes or interfaces. I think I have to spend a little more time to read information about the Preview 2 and to get it work as I expected.

At the same time, I posted my questions about the change of Preview 2 and asking if there is other alternative Open Source based library available on StackOverflow. It seems like that there are very a few EST kits or libraries for .Net platform. One alternative is OpenRasta. It looks like that OpenRasta is very good package. Basically its architecture is based on three elements: resources, handlers and codec. Resources are information to be exposed to the REST service, handlers are HTTP handlers for GET, PUT and POST for various request patterns, and Codec is enCoding/deCoding of HTTP requests.

Though the person who provides OpenRasta is a very smart .Net developer, he has very limit resources, time and effort to move it out. So far it is not so widely used and it looks like that it does not support Atom and feed services. Maybe in the new release coming the next month, some more enhancement will be available.

I'll keep eye on this library and at the same time, I have to look at Preview 2 in a deep level. Enjoy exploring!

Read More...

Collapse...

MSDN Channel 9

In the past month I have been watching talks or conversations on MSDN Channel 9. Microsoft is going to be more open than before. I enjoyed those open talks on the technology and new stuff. Actually, I think that Microsoft attract many talent people who have been on Open development for years and Microsoft has learned that they can gain benefit by join the Open world!

I found the Channel 9 one day when I tried to find out any shows or videos about REST web service with .Net or Visual Studio. To my surprise, I found a series talks on this at Channel 9. The best talks are offered by PluralSight.com, which is a search on its site for REST. PuralSight has better quality videos. Based on the talks, I found that Microsoft provided a REST package for REST web service development, and it is very impressive.

In addition to that, from Channel 9, I found many other great Open source projects posted to CodePlex, which is Microsoft Open Source web site. I have used many Open source libraries there, including Json.Net, MVC...

Read More...

Collapse...

Json.NET and Its Usage (3)

In this blog, I'l continue to the issues related to XML. In many cases, I have XML strings as data source, such as configuration files, and data in the format of XML from ADO.Net. JSon.Net provides some APIs to convert from XML to Json strings or vice versa. Based on those APIs, I have added some methods to my wrap class.

The first method is to convert from a Json string to an XML string:

public static string ConvertToXMLString(string jsonString)
{
  XmlNode xmlNode = JsonConvert.DeserializeXmlNode(jsonString);
  string xmlString = xmlNode.OuterXml;
  
  return xmlString;
}

Note: make sure the JsonString must have a single root item on top. If the JsonString contains more than one property values at the top or root, this conversion will throw exception since the converted XML has to have a single root node.

The opposite method is to convert an XML string to a JsonString:

public static string ConvertToJsonString(string xmlString)
{
  var xmlDoc = new XmlDocument();
  xmlDoc.Load(new StringReader(xmlString));
  string jsonString = JsonConvert.SerializeXmlNode(xmlDoc.DocumentElement);

  return jsonString;
}

public static string ConvertToFormattedJsonString(
    string xmlString,
    bool quoteName)
{
  string jsonStr;
  using (MemoryStream msJson = new MemoryStream(xmlString.StrToByteArray()))
  {
    using (MemoryStream ms = new MemoryStream())

    {
       using (JsonTextWriter jtw = new JsonTextWriter(new StreamWriter(ms)))
       {
          jtw.QuoteName = quoteName;
          jtw.Formatting = Newtonsoft.Json.Formatting.Indented;
          jtw.WriteToken(new JsonTextReader(new StreamReader(msJson)));

          jtw.Flush();

          ms.Flush();
          ms.Position = 0;
          using (StreamReader sr = new StreamReader(ms))
          {
              jsonStr = sr.ReadToEnd();
          }
       }
    }
  }

  return jsonStr.Replace(@"\r\n", Environment.NewLine);
}

Depending on the usage, you may need to get a nice formatted JsonString or just a long JsonString.

One reason I added some XML API methods in my wrapper class is that I find out it is very easy to manipulate XML strings by using XML Parser or XMLDoc class. For example, when I get an XML string from a configuration file, before I convert it to an instance, I have to prepare the XML in a correct format. By the time I got Json.Net library, the library did not support XML string with comments(the author promised to handle this issue), So I have to remove all the comments before converting the XML string to JsonString.

Another example is that I may have only one node in XML file as a property value. However, to convert the property value to an array of property values, I have to add a dummy node to XML file so that the JsonString from XML file will be in the correct layout before I map it to an instance. Therefore, I have the following API methods to cover those cases:

public static int AddNewNodeToRefNode(
  ref string xmlString,
  string refNodeXPath,
  string newNodeName,
  string newInnerText,
  bool beforeOrAfter,

  bool firstOrAll)
{
  //  Add a new node to a reference node
  //  ...
}

public static int AddNewNodeToRefNodeAsChildNode(
  ref string xmlString,
  string refNodeXPath,

  string newNodeName,
  string newInnerText,
  bool firstOrLastChildren,
  bool firstOrAll)
{
  // Add new node as a child node to a reference node]
  // ...
}

public static int GetXMLNodesCount(

  string xmlString,
  string nodeXPath)
{
  // Get count of a node in XML string
  // Use this method get count before adding new dummy nodes
  // ...
}

public static string RemoveComments(
  string xmlString)
{

  // Remove all comments in XML string
  // ...
}

public static bool UpdateXMLNodes(
  ref string xmlString,
  string nodeXPath,
  string newInnerText,
  bool firstNodeOrAll)
{

  // Use this method to update XML node content
  // if you use XML as input to JsonString then to Instance
  // and save the changes in Instance back to XML
  // ...
}

This is the conclusion of my serials of brief introduction of Json.Net library and my wrapper class. I'll continue to provide some examples about how to use the library.

Read More...

Collapse...

Json.NET and Its Usage (2)

In this blog, I'll continue to describe my wrapper class based on Json.Net. The main reason I want to write a wrapper class to hide Json.Net is to provide only APIs I am interested in. Json.Net provides a rich framework of classes, interfaces, types, and enums. I don't know all of them. In practice, there is no need to know all.

The second reason is to break direct dependency on Json.Net. My wrapper class provides all the APIs I need. There may be some cases in the future that a better library available than Json.Net or some potential problems preventing me from using Json.Net. Then what I need to do is to rewrite the wrapper class's internal implementation without change my application or libraries which are dependent on the wrapper class. Actually, I find this the best practice to use other library or components.

I created a common .Net library, DotNetCommonLibrary, with some common and generic classes, including my wrapper class for Json.Net. The first thing I need is to add a reference to Newtonsoft.Json to the library.

Next, I include the following namespace in the wrapper class:

# region Using
using System;
using System.IO;
using System.Xml;
using System.Xml.Linq;
using System.Xml.XPath;
using Newtonsoft.Json;
using JsonFormatting = Newtonsoft.Json.Formatting;
using Newtonsoft.Json.Converters;
# endregion

The wrapper class is called as JsonXMLHelper. The class does not contain any private instance data, therefore, the class is a static class.

public static class JsonXMLHelper
{
...
}

The first method is a simple one: convert a Jsonstring to an instance.

public static T GetInstance<T>(
  string jsonString)
  where T : class
{
  T instance = JsonConvert.DeserializeObject(jsonString, typeof(T)) as T;
  return instance;
}

Vice-versa, there is the method to convert instance to Jsonstring:

public static string GetJsonString<T>(
  T instance) where T : class
{
  return GetJsonString<T>(instance, false);
}

public static string GetJsonString<T>(
  T instance,
  bool indented)
  where T : class
{
  return GetJsonString<T>(
    instance, indented, true, null);
}

public static string GetJsonString<T>(
  T instance, 
  bool indented,
  bool quoteName,
  string dtFormat) where T : class
{
  JsonFormatting f = indented ? 
    JsonFormatting.Indented :
    JsonFormatting.None;
  string jsonStr;
  // Date format available?
  if (!string.IsNullOrEmpty(dtFormat))
  {
    JsonSerializerSettings jss = 
      new JsonSerializerSettings();
    IsoDateTimeConverter dtConvert = 
      new IsoDateTimeConverter();
    dtConvert.DateTimeFormat = dtFormat;
    jss.Converters.Add(dtConvert);
    jsonStr = JsonConvert.SerializeObject(
      instance, f, jss);
  }
  else
  {
    jsonStr = JsonConvert.SerializeObject(
      instance, f);
  }

  if (!quoteName)
  {
    jsonStr = ConvertToFormattedJsonString(
      jsonStr, quoteName);
  }

  return jsonStr;
}

GetJsonstring() has several overloads, which provide options to specify if a Jsonstring is indented, if a quote " char is used for Jsonstring names, and what is the format for DateTime type. By default, a Jsonstring is not indented as one long string and Json names are quoted by ", and DateTime values are displayed in the format of Date(tick_numbers). I need to log some instances as Jsonstrings in a nice and readable format. Those overloads provide options I need.

I used this GetJsonstring() method a lot. It saves me a lots of time to override ToString() method to format instance in a nice string. The most frustrated thing is that when I update class property names, I often forget to update ToString(). With this API method, I don't need to worry about this.

I even don't need to override ToString() any more. I can directly call this method with an instance to get a nice Json string. It is the most handy way to print or log .Net or third party class instances. You can even print or log an instance of List<T> type, but be prepared for very long strings.

Read More...

Collapse...

VIM, Initial Configuration and PlugIns

I have been used VIM for about weeks and started to fall in love with it. I am still in learning stage and found many some settings very neat.

The first thing is to configure VIM initial settings. This is done through the configuration file ~/.vimrc for Mac OS and C:\_rimrc for Windows. You can find many sites with good .vimrc file and use it as start. If you don't like some settings, you can change it.

Based on vimrc in the article of VIM Introduction and Tutorial, I made some changes. For example, I prefer the background color as white and tab as 2:

  "Tabs are 2 spaces:
  set tabstop=2
  "We use a white background, don't we?
  set bg=white

I found some settings and added them in .vimrc:

  "Set characters shown for special cases such as:
  "wrap lines, trail spaces, tab key, and end of line.
  "(must be turned on whith set list)
  set listchars=extends:»,trail:°,tab:>¤,eol:¶
  "Set moving around at the end of line back to previous line by
  " key and coursor keys, and normal movememt h and l keys
  set whichwrap=b,s,<,>,h,l

  "Enable C style indention
  set cindent

I love special char setting, which displays tab, trial spaces, and end of line as special characters. I have to use command ":set list" to enable the setting. The move around setting enables backspace, arrow keys, and move keys h and l to move back previous lines.

There are many VIM plugins available. This is quite new for me. I found only one is working for me so far: Align. To use it, just type in the command ":Aligh =" when you in visual mode with several lines selected. Then the text will be aligned by = sign to separate left part and right part aligned by = sign.



After aligning, the text are aligned by =:



To install it, find the folder where the file "Align.vba.gz" located in Terminal. Type in the command:

  vim Align.vba.gz

The file is opened by VIM. Then typing the command to read scripts and then quit:

  :so %
  :q

Reopen VIm again. The command :Align is available!

Read More...

Collapse...

Initialize Castle.AR and NHiberate

I tried to use Castle Project's ActiveRecord to map a .Net class to an Oracle DB in an ASP project. I followed the instruction to create an xml configuration file, to load the configuration and then to initialize ActiveRecord like this:

  XmlConfigurationSource source = new XmlConfigurationSource(
  "C:\\temp\\ActiveRecords.xml");

  ActiveRecordStarter.Initialize(source, typeof(Employee));

I got a failure exception when Initialize() method is called. I struggled for almost a whole day to find out why, and downloaded Castle's example project as well. I got the same error message (System.TypeInitializationException).

Finally, I found out in the detail information in debug mode that the inner exception was caused by "could not find log4net.dll file". When I checked the path for this file in the project references, I realized that it points to a wrong dll file in the MBUnit folder. After I removed and re-added the correct dll file, the problem was solved!

Read More...

Collapse...