Quality Assurance

Optimization for SOLR Search validation through automation

Do you know that SOLR implementation testing can be automated and it can be extended to any CMS integration project -developed with similar architecture? In this blog we will discuss the advantages of automating SOLR search results -functionality. Automation of the manual search activities can help quick validation and reduce the manual efforts of QA activities involved.

Manual Validation Process:

To verify the SOLR search functionality QA must compare the search results from SOLR admin and the CMS implementation. This can be done in the following way:

Step 1: The QA associate need to make a search query in CMS application for a keyword at a specific country/locale say search for “automobile” in EMEA/italy/it/it.

Step 2: Now perform the same operation at SOLR admin ie. search for keyword “automobile” at EMEA/italy/it/it

Step 3: Now compare the count of search results that were recorded in the CMS application and SOLR admin.

The number of search results in the CMS application and SOLR admin should be approximately equal as the CMS application pulls the results from SOLR admin

Note: The search results that come up in SOLR admin doesn’t include Featured Results as Feature Result is a customized feature which may vary from project to project and not a part of SOLR admin.

Process to execute the search query in CMS application & SOLR Admin:

Step 1: Publish all the pages under the locale where search has to be done. For Ex: search has to be done in EMEA/italy/it, so all the pages under locale “it” should be published.

Step 2: Add the tags to the pages that are under locale “it”.

Step 3: Create a search page from a search page template under locale “it” so that when search button is clicked the page is redirected to the search page.

Step 4: Publish the search page.

Step 5: Create a collection in the CMS application (EMEA/italy/it).

Step 6: Index the pages inside EMEA/italy/it by clicking on Index and give the required path.

Step 7: Go to the SOLR admin and select the locale “EMEA/italy/it” and search for “automobile”.

Step 8: Observe the number of search results in Admin.


The above process is error prone and will consume considerable amount of time if it has to be repeated for a combination of different country/locales and keywords.


The above challenge can be overcome by automating the search results functionality. The automation of the above process includes performing the search for a keyword “page” at SOLR admin by selecting a specific country/locale (itlay/it) and capturing the search result count in SOLR admin in an excel file. Now the same keyword “page” is searched in the CMS application for italy/it and the no. of search results are captured and written to the excel file.QA now gets to know the difference in search results from the excel and that decides the quality of SOLR connector implementation.

Automated script flow:

1.Opens the page under EMEA/italy/it.

2.Enters the keyword inside the search field and clicks on search button.

3.Records the number of search results in the CMS application and pastes the search result data in the   excel file.

4.Redirect the url to SOLR admin and selects the EMEA/italy/it from the drop down.

5.Enters the keyword “automobile” in the search query.

6.Captures the search result data and pastes it to the excel file.


  1. This solution can also be used to perform quick sanity check.
  2. The above solution can be included in regression testing suite
  3. Efforts saving by avoiding the manual validation
  4. The script can be customized for multiple iterations with different test data sets
  5. This solution can be extended to any other search platforms like Coveo with minimum configuration settings
  6. This solution can be customized to work with multiple CMS integration with Search platforms

About The Author