Type-safe JSON Parsing. Easy with DukeScript!

DukeScript is primarily optimized for designing client side responsive applications in Java and HTML, however libraries in its core are built with the goal of making Java/JavaScript interoperability easier. One of the essential concepts in JavaScript is JSON. Thus it is no surprise, DukeScript makes processing JSON from Java really smooth.

Where is My JSON?

First of all we need to obtain a JSON file. One of the free JSON services is the GitHub REST API, so let’s use it. The following command lists all repositories for a github account (actually the Jersey JAX-RS developers account):

$ curl https://api.github.com/users/jersey/repos

The ouput of this query has the following format. Save it into a file and let’s design a type-safe way to parse such a file with the help of DukeScript:

[
  {
    "id": 6109440,
    "name": "hol-sse-websocket",
    "full_name": "jersey/hol-sse-websocket",
    "owner": {
      "login": "jersey",
      "id": 399710,
      "avatar_url": "https://avatars.githubusercontent.com/u/399710?v=3",
      "gravatar_id": "",
      "url": "https://api.github.com/users/jersey",
      "html_url": "https://github.com/jersey",
      "followers_url": "https://api.github.com/users/jersey/followers",
      "following_url": "https://api.github.com/users/jersey/following{/other_user}",
      "gists_url": "https://api.github.com/users/jersey/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/jersey/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/jersey/subscriptions",
      "organizations_url": "https://api.github.com/users/jersey/orgs",
      "repos_url": "https://api.github.com/users/jersey/repos",
      "events_url": "https://api.github.com/users/jersey/events{/privacy}",
      "received_events_url": "https://api.github.com/users/jersey/received_events",
      "type": "Organization",
      "site_admin": false
    },
    "private": false,
    "html_url": "https://github.com/jersey/hol-sse-websocket",
    "description": "Hands-on-lab on using server-sent events and web socket with Jersey and Tyrus.",
    "fork": false,
    "url": "https://api.github.com/repos/jersey/hol-sse-websocket",
    "forks_url": "https://api.github.com/repos/jersey/hol-sse-websocket/forks",
    "keys_url": "https://api.github.com/repos/jersey/hol-sse-websocket/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/jersey/hol-sse-websocket/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/jersey/hol-sse-websocket/teams",
    "hooks_url": "https://api.github.com/repos/jersey/hol-sse-websocket/hooks",
    "issue_events_url": "https://api.github.com/repos/jersey/hol-sse-websocket/issues/events{/number}",
    "events_url": "https://api.github.com/repos/jersey/hol-sse-websocket/events",
    "assignees_url": "https://api.github.com/repos/jersey/hol-sse-websocket/assignees{/user}",
    "branches_url": "https://api.github.com/repos/jersey/hol-sse-websocket/branches{/branch}",
    "tags_url": "https://api.github.com/repos/jersey/hol-sse-websocket/tags",
    "blobs_url": "https://api.github.com/repos/jersey/hol-sse-websocket/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/jersey/hol-sse-websocket/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/jersey/hol-sse-websocket/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/jersey/hol-sse-websocket/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/jersey/hol-sse-websocket/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/jersey/hol-sse-websocket/languages",
    "stargazers_url": "https://api.github.com/repos/jersey/hol-sse-websocket/stargazers",
    "contributors_url": "https://api.github.com/repos/jersey/hol-sse-websocket/contributors",
    "subscribers_url": "https://api.github.com/repos/jersey/hol-sse-websocket/subscribers",
    "subscription_url": "https://api.github.com/repos/jersey/hol-sse-websocket/subscription",
    "commits_url": "https://api.github.com/repos/jersey/hol-sse-websocket/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/jersey/hol-sse-websocket/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/jersey/hol-sse-websocket/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/jersey/hol-sse-websocket/issues/comments/{number}",
    "contents_url": "https://api.github.com/repos/jersey/hol-sse-websocket/contents/{+path}",
    "compare_url": "https://api.github.com/repos/jersey/hol-sse-websocket/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/jersey/hol-sse-websocket/merges",
    "archive_url": "https://api.github.com/repos/jersey/hol-sse-websocket/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/jersey/hol-sse-websocket/downloads",
    "issues_url": "https://api.github.com/repos/jersey/hol-sse-websocket/issues{/number}",
    "pulls_url": "https://api.github.com/repos/jersey/hol-sse-websocket/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/jersey/hol-sse-websocket/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/jersey/hol-sse-websocket/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/jersey/hol-sse-websocket/labels{/name}",
    "releases_url": "https://api.github.com/repos/jersey/hol-sse-websocket/releases{/id}",
    "created_at": "2012-10-07T04:44:32Z",
    "updated_at": "2014-06-29T22:29:42Z",
    "pushed_at": "2013-05-29T16:56:03Z",
    "git_url": "git://github.com/jersey/hol-sse-websocket.git",
    "ssh_url": "git@github.com:jersey/hol-sse-websocket.git",
    "clone_url": "https://github.com/jersey/hol-sse-websocket.git",
    "svn_url": "https://github.com/jersey/hol-sse-websocket",
    "homepage": null,
    "size": 7750,
    "stargazers_count": 11,
    "watchers_count": 11,
    "language": "Java",
    "has_issues": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 5,
    "mirror_url": null,
    "open_issues_count": 1,
    "forks": 5,
    "open_issues": 1,
    "watchers": 11,
    "default_branch": "master"
  },
  {
     "etc." : "etc."
  }
]

Setting the Project Up

Let’s get started following the same way we start with regular HTML UI based DukeScript application. Follow the getting started tutorial if you are OK with using NetBeans. If you want to stay on command line, you can use:

$ mvn archetype:generate -DarchetypeGroupId=org.apidesign.html \
  -DarchetypeArtifactId=knockout4j-archetype \
  -DarchetypeVersion=1.1.2

Once our application is generated, we can start modifying the sample application to avoid displaying the UI, but rather parse our JSON file. Let’s remove:

src/main/webapp/pages/index.html - no UI definitions
src/main/java/your/pkg/DataModel.java - no UI model
src/test - we don’t need tests for this simple sample

The next step is to change dependencies in our pom.xml - the sample project can parse JSON inside of a running browser, but we want to do it in Java. Luckily there is a JAR that can handle that. Just include it on runtime classpath and it will handle all the parsing for us. Add the following dependency:

<dependency>
  <groupId>org.netbeans.html</groupId>
  <artifactId>ko-ws-tyrus</artifactId>
  <version>${net.java.html.version}</version>
  <scope>runtime</scope>
</dependency>

In addition to that let’s clean the Main.java so it is ready for our parsing code and looks like:

package your.pkg;

public final class Main {
    private Main() {
    }
    
    public static void main(String... args) throws Exception {
    }
}

Now we have an empty skeletal application what we can use to do the parsing, which can be verified by executing ‘mvn clean install’ - the project should built without any issues.

Parsing JSON Files

We are ready to start the parsing. There is the net.java.html.json.Models class in the core DukeScript API and it contains two overloaded variants of the method parse. One method can parse a single JSON object, the second can parse a JSON array of multiple JSON objects. Given the fact that the file we obtained from GitHub lists an array of repositories, we should use the more complicated variant. Here is the code:

public static void main(String... args) throws Exception {
    BrwsrCtx ctx = parsingContext();
    FileInputStream is = new FileInputStream("/tmp/jersey.json");
    List<RepositoryInfo> arr = new ArrayList<>();
    Models.parse(ctx, RepositoryInfo.class, is, arr);

    System.err.println("all parsed data: " + arr);

    RepositoryInfo first = arr.get(0);
    System.err.println("id: " + first.getId());

    is.close();
}

As part of the setup we get the parsing context. Then we open the JSON file we want to parse. We allocate an array to hold the results and then ask the parse method to do the parsing and fill the list. Then we print the whole array to output and also access the first element in the array in a type-safe way and obtain its id property. Here is the output from the execution of the above program:

all parsed data: [{"id":6109440}, {"id":4368712}, {"id":14029306}, {"id":4522462}, {"id":911627}]
id: 6109440

Now one important question: How does the code know there is an id property in the JSON?

Defining the Model

DukeScript core APIs support a type-safe way of accessing JSON structures. Moreover, in order to avoid writing tons of boiler-plate code, we describe the JSON structure using annotations and generate the rest with the help of annotation processors. As such the RepositoryInfo class is defined by:

@Model(className="RepositoryInfo", properties = {
    @Property(name = "id", type = int.class)
})
public final class Main {
   // now the main method
}

The above definition generates the RepositoryInfo class with a single integer property id and code to unmarshall JSON into instances of that class. The RepositoryInfo class also defines a reasonable toString() method - hence the whole list of such objects can be dumped into output in JSON format.

What to do when you want to parse other JSON properties? Take a look at the format of the downloaded JSON response file and add a new property definition! For example there is the name property which seems to be of type String. Just change the definition to:

@Model(className="RepositoryInfo", properties = {
    @Property(name = "id", type = int.class),
    @Property(name = "name", type = String.class),
})

Just by adding a single line, a getter getName() is added to the RepositoryInfo info class and you can use it in Java code:

RepositoryInfo first = arr.get(0);
System.err.println("id: " + first.getId());
System.err.println("name: " + first.getName());

The output when running the modified program is now:

all parsed data: [{"id":6109440,"name":"hol-sse-websocket"}, {"id":4368712,"name":"jersey"}, {"id":14029306,"name":"jersey-1.x"}, {"id":4522462,"name":"jersey-1.x-old"}, {"id":911627,"name":"jersey-old"}]
id: 6109440
name: hol-sse-websocket

And we can continue adding as many properties into our @Model definition as we need. For example one named private of type boolean. And so on, and so on, until we can access enough information via type-safe Java getters. The remaining, not listed, properties are silently ignored.

Parsing Nested Objects

We can see that the JSON format contains a nested object named owner. How can we parse such a structure in a type-safe way? Just define yet another model definition for it:

@Model(className="RepositoryInfo", properties = {
    @Property(name = "id", type = int.class),
    @Property(name = "name", type = String.class),
    @Property(name = "owner", type = Owner.class),
    @Property(name = "private", type = boolean.class),
})
public final class Main {
    @Model(className = "Owner", properties = {
        @Property(name = "login", type = String.class)
    })
    static final class OwnerCntrl {
    }
    // the main method...
}

With the above definition one can call getOwner().getLogin() to obtain the login information from a nested object. In addition to that, one can also access an array of values by specifying @Property(array = true, ...) when defining the JSON structure we want to access from Java. Here is the final code of our class:

import java.io.FileInputStream;
import java.util.ArrayList;
import java.util.List;
import net.java.html.BrwsrCtx;
import net.java.html.json.Model;
import net.java.html.json.Models;
import net.java.html.json.Property;
import org.netbeans.html.context.spi.Contexts;

@Model(className="RepositoryInfo", properties = {
    @Property(name = "id", type = int.class),
    @Property(name = "name", type = String.class),
    @Property(name = "owner", type = Owner.class),
    @Property(name = "private", type = boolean.class),
})
public final class Main {
    @Model(className = "Owner", properties = {
        @Property(name = "login", type = String.class)
    })
    static final class OwnerCntrl {
    }

    private Main() {
    }
    
    public static void main(String... args) throws Exception {
        BrwsrCtx ctx = parsingContext();
        FileInputStream is = new FileInputStream("/tmp/jersey.json");
        List<RepositoryInfo> arr = new ArrayList<>();
        Models.parse(ctx, RepositoryInfo.class, is, arr);

        System.err.println("all parsed data: " + arr);
        
        RepositoryInfo first = arr.get(0);
        System.err.println("id: " + first.getId());
        System.err.println("name: " + first.getName());
        System.err.println("private: " + first.isPrivate());
        System.err.println("owner.login: " + first.getOwner().getLogin());
    }
}

Here is the final output:

all parsed data: [{"id":6109440,"name":"hol-sse-websocket","owner":{"login":"jersey"},"private":false}, {"id":4368712,"name":"jersey","owner":{"login":"jersey"},"private":false}, {"id":14029306,"name":"jersey-1.x","owner":{"login":"jersey"},"private":false}, {"id":4522462,"name":"jersey-1.x-old","owner":{"login":"jersey"},"private":false}, {"id":911627,"name":"jersey-old","owner":{"login":"jersey"},"private":false}]
id: 6109440
name: hol-sse-websocket
private: false
owner.login: jersey

Parsing JSON in a type-safe way in Java has never been easier!

Appendix: Setting the Context

The last thing to mention is the way to setup our parsing context. It is a bit magical, but as it is done just once, when setting up the classpath of the whole application, it can be hidden, for example, in the following method:

private static BrwsrCtx parsingContext() {
    Contexts.Builder builder = Contexts.newBuilder("tyrus");
    Contexts.fillInByProviders(Main.class, builder);
    return builder.build();
}

What does it do? Well it uses the ‘magic string’ “tyrus” to request the Java implementation of JSON parser which is provided by the ‘ko-ws-tyrus’ module (that is the reason why we added a dependency on this module in our pom.xml).

This is exactly the kind of configuration done by Jersey Faces to register proper entity convertors, so one can use the classes generated by the @Model annotation inside of Jersey’s @GET, etc. methods just by including the appropriate JAR on the Jersey application classpath.

Once again, enjoy DukeScript everywhere!