UI Development

Consuming Json Facet API

Use case 

Query documents(or records) where ‘manufacturer’ & ‘price’ fields exists but exclude records where price is NOT $O.O

And for each manufacturer

1. Calculate total price of all products from all manufacturers

2. Aggregate prices of all their products (i:e rollup prices of manufacturer’s each product into a single bucket)

3. Sort price buckets in ascending order

Json facet query

Query can be formulated as follows

http://localhost:8983/solr/techproducts2/select?indent=on&q=*:*
AND manu_exact:[* TO *] AND price:[* TO *]
AND -price:0.0&rows=0&wt=json&
json.facet={
 "total_price":"sum(price)",
 category_facet:{
                   type:terms,
                   field:manu_exact,
                   limit:-1,
                   sort:"manu_level_price asc",
                   facet:{
                        "manu_level_price":"sum(price)"
                    }
                }
             }
 }

Above Solr query is a combination of common params q and json.facet elements.

Solr response

{
  "responseHeader":{
    "zkConnected":true,
    "status":0,
    "QTime":7,
    "params":{
      "q":"*:* AND manu_exact:[* TO *] AND price:[* TO *] AND -price:0.0",
      "json.facet":"{ \"total_price\":\"sum(price)\",manu_facet:{ type:terms,field:manu_exact,limit:-1,sort:\"cat_level_price asc\",facet:{ cat_level_price:\"sum(price)\"}}}}",
      "_stateVer_":"techproducts2:136",
      "indent":"true",
      "rows":"0",
      "wt":"json"}},
  "response":{"numFound":13,"start":0,"maxScore":3.0,"docs":[]
  },
  "facets":{
    "count":13,
    "total_price":5251.270030975342,
    "manu_facet":{
      "buckets":[{
          "val":"Belkin",
          "count":2,
          "cat_level_price":31.450000762939453},
        {
          "val":"Samsung Electronics Co. Ltd.",
          "count":1,
          "cat_level_price":92.0},
        {
          "val":"Corsair Microsystems Inc.",
          "count":2,
          "cat_level_price":259.98999786376953},
        {
          "val":"ViewSonic Corp.",
          "count":1,
          "cat_level_price":279.95001220703125},
        {
          "val":"Maxtor Corp.",
          "count":1,
          "cat_level_price":350.0},
        {
          "val":"Apple Computer Inc.",
          "count":1,
          "cat_level_price":399.0},
        {
          "val":"ASUS Computer Inc.",
          "count":1,
          "cat_level_price":479.95001220703125},
        {
          "val":"Canon Inc.",
          "count":2,
          "cat_level_price":509.9400177001953},
        {
          "val":"ATI Technologies",
          "count":1,
          "cat_level_price":649.989990234375},
        {
          "val":"Dell, Inc.",
          "count":1,
          "cat_level_price":2199.0}]}}}

How to consume Solr response in Java application?

1.   Create Solr cluster with techproducts collection and start nodes as suggested in posts SolrCloud setup on Mac

2.  Index sample documents that come with techproducts

cd ~/solr-6.4.1/bin 
./post -c techproducts ../example/exampledocs/*.*

3.  Run the following Spring boot Junit test

package org.demo;
import java.util.List;
import javax.annotation.Resource;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrRequest.METHOD;
import org.apache.solr.client.solrj.impl.CloudSolrClient;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.params.CommonParams;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.common.util.SimpleOrderedMap;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.core.env.Environment;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit4.SpringRunner;
@RunWith(SpringRunner.class)
@TestPropertySource(locations="classpath:test.properties")
public class SolrTest {
 
 private CloudSolrClient solrClient;
 private static final String BUCKET_VAL = "val";
 private static final String JSON_FACET = "json.facet";
 private static final String BUCKETS = "buckets";
 public static final String FACETS = "facets";
 public static final String ZKHOST = "zkhost";
 public static final String COLLECTION_NAME = "techproducts.collection";
 public static final String TECHPRODUCTS_SOLR_QUERY = "techproducts.solr.query";
 public static final String TECHPRODUCTS_JSON_FACET = "techproducts.json.facet";
 public static final String MANU_FACET = "manu_facet";
 
 @Resource
 private Environment environment;
 
 @Before
 public void createSolrCloudClient() {
     if (solrClient == null) {
           solrClient = new CloudSolrClient.Builder().withZkHost(environment.getProperty(ZKHOST)).build();
           solrClient.setDefaultCollection(environment.getProperty(COLLECTION_NAME));
     }
 }
 
 @SuppressWarnings("unchecked")
 @Test
 public void test() {
 try {
 SolrQuery query = new SolrQuery();
 query.set(CommonParams.Q, environment.getProperty(TECHPRODUCTS_SOLR_QUERY));
 query.set(JSON_FACET, environment.getProperty(TECHPRODUCTS_JSON_FACET));
 query.setRows(0);
 QueryResponse response = solrClient.query(query, METHOD.POST);
 NamedList<Object> bucketList = response.getResponse();
 // notice "findRecursive" usage to get the buckets list
 List<SimpleOrderedMap<Object>> buckets = (List<SimpleOrderedMap<Object>>) bucketList.findRecursive(FACETS, MANU_FACET, BUCKETS);
 if (buckets != null) {
 for (SimpleOrderedMap<Object> bucket : buckets) {
 String manufacturerLevel = (String) bucket.get(BUCKET_VAL);
 // manufacturer's name and total of all prices for this manufacturer 
 // Notice "manu_level_price" that was specified in json.facet
 System.out.println("manufacturerLevel:"+manufacturerLevel+", manu_level_price:"+bucket.get("manu_level_price"));
 }
 // Total prices of all manufacturers
 // notice "total_price" that was specified in json.facet and "findRecursive" usage
 System.out.println("total_price_all_manufacturers:"+bucketList.findRecursive(FACETS, "total_price"));
 }
 } catch (Exception solrExc) {
 solrExc.printStackTrace();
 }
 }
}

4.  Externalize Solr query, json.facet and any properties to test.properties file.

5. Following is the output of Junit test showing sum of all prices per each manufacturer.

manufacturerLevel:Belkin, manu_level_price:31.450000762939453
manufacturerLevel:Samsung Electronics Co. Ltd., manu_level_price:92.0
manufacturerLevel:Corsair Microsystems Inc., manu_level_price:259.98999786376953
manufacturerLevel:ViewSonic Corp., manu_level_price:279.95001220703125
manufacturerLevel:Maxtor Corp., manu_level_price:350.0
manufacturerLevel:Apple Computer Inc., manu_level_price:399.0
manufacturerLevel:ASUS Computer Inc., manu_level_price:479.95001220703125
manufacturerLevel:Canon Inc., manu_level_price:509.9400177001953
manufacturerLevel:ATI Technologies, manu_level_price:649.989990234375
manufacturerLevel:Dell, Inc., manu_level_price:2199.0
total_price_all_manufacturers:5251.270030975342

6.  The take away from this Junit test:
NamedList<Object>.findRecursive() allows to consume Solr response (even arbitrarily nested buckets) and analytics such as sum, max, min, avg
7. Ensure SolrJ client lib version matches with Solr version. For example, If using Solr 6.4.1 version, use the same versions of SolrJ​ client that comes with Solr download.

8. Spring boot project with above test is checked in to Git repository at https://github.com/chakrayel/myrepo.git.

About The Author

Leave a Reply

*