Skip to main content

REST Connector - How to configure a recursive root entity when data come in one big flat list

Consider that the REST service will return one big JSON or XML document containing all categories with its id, its name and a reference to its parent.

Here is an example with two root categories (a and b) and two sub-categories (b1 and b2) as children of b.

Endpoint:

https://example.com/rest/v1/categories

1 Example Data​

JSON

{"categories": [
 {"id": "a", "name": "A", "parent": null}, 
 {"id": "b", "name": "B", "parent": null}, 
 {"id": "b1", "name": "B 1", "parent": "b"}, 
 {"id": "b2", "name": "B 2", "parent": "b"}
]}

XML

<data>
    <categories id="a" name="A" />
    <categories id="b" name="B" />
    <categories id="a1" name="A1" parent="a" />
    <categories id="a2" name="A2" parent="a" />
</data>

2 Configuration in EntityModel​

One entity called "category" and set as "root" having a sub-.buckets reference pointing on "category" itself.

Connector Entity Identifier

"category"

3 Configuration in RestConnector​

We use JXPath to filter JSON data.

<json-data-type>org.apache.commons.jxpath.JXPathContext</json-data-type>

And this is our example connector entity definition.

<entity name="category">
    <cache>
      <max-duration>3600</max-duration>
    </cache>
    <url>https://example.com/rest/v1/categories</url>
    <query for="children"  
        <items-path>categories[($id and @parent=$id) or (not($id) and not(@parent))]</items-path>
    </query>  
    <query for="identifier">
        <items-path>categories[@id=$id]</items-path>
    </query>
    <mappings>
      <mapping for="identifier"><field-path>@id</field-path></mapping>
      <mapping for="label"><field-path>@name</field-path></mapping>
    </mappings>
</entity>

4 Comments on the Configuration​

Some comments are required.

The endpoint part is very simple. We are always using the result of a fixed url. That is why we can write the directly to the and do not use specific variant for each query.

The mapping is also simple. We just map to the attributes.

Note: We will use "@" as attribute prefix here. This is not really required for JSON. For JXPath on JSON "@id" is just the same as "id". But our variant with "@" will make our XML example running too (see below).

4.1 Item-path for query for identifier​

xml<items-path/> for the "identifier" query is also simple. Just filter the flat list of categories for the one with the matching id.

4.2 Item-path for query for children​

This is the trickier part.

Query "for=children" is using some logic to separate queries for root buckets and queries for child buckets. We will always select the "categories" field within the json. That is easy. But then we have two conditions.

Root bucket condition

not($id) and not(@parent)

A root bucket query is defined by having no $id from the call - and a root bucket is defined by having no @parent attribute in the data.

Note: _In older connector versions you should use "$id='null'" instead of "not($id)", because an empty $id evaluated to the string 'null'.

Child bucket condition

$id and @parent=$id)

A child bucket query is defined by having a non empty $id string from the call - and a child bucket is found by matching this $id to the @parent attribute in the data.

Of course: Do enclose the parts in nice simple brackets.

4.3 Caching​

Caching should be enabled for such an entity. Typically the tree structure changes with low frequency meaning that the duration can be set to a high value. We use 1h (3600 sec) in this example.

If your REST service supports E-Tags for this endpoint you may profit from adding to the caching config. Like

<cache>
  <max-duration>3600</max-duration>
  <revalidate-after>60</revalidate-after>
</cache>

This will trigger a "conditional" HTTP request for the "categories" if an item retrieved from cache is older than 60 sec. The request will contain ETag and/or "Last-Modified" headers. The REST service will compare the values to its own cache and will hopefully send a very fast "304 Not modified" response in most of the cases. In case the connector will prolong the cache duration for the cached item and use the cached item to build the entitydata response.