Public Lab Research note

GSoC 2016: Final Work Product of Advanced Searching and Sorting Tool for

by Ujitha | August 22, 2016 18:51 22 Aug 18:51 | #13383 | #13383

Project Details

Project Title: Advanced Searching and Sorting Tool for

Project Proposal: advanced-searching-and-sorting-tool-for-publiclab-org

Link to GSoC Project:

Open source Platform Used: Github

Organization mailing list:!forum/plots-dev

Github Organization Page: Publiclab

Repository Contributed to: plots2

Languages/Framework Used: Ruby on Rails, solr search server with sunspot, grape api framework, JavaScript(jQuery), HTML/CSS

Code Links

For this project, we used two repositories.

  1. Main Repo (upstream) -
  2. Primary Repo (forked from main repo) -

Since we are implementing completely new technology to the system, one of my mentors (Bryan Bonvallet) advice was to use separate repository for this advanced search tool and avoid technological conflicts from other contributors. Specially from other GSoC projects.

All the commits -

On going pull requests

Project Description

Analysed existed code base and refactored for performance.

when i starting to contribute to publiclab, there was a normal sql query based search in the for the system. But that search was not able to cover up the current searching requirement of the system. Mainly advanced search page was needed to be improved.

Previous advanced search page


As the first step I went through all the search queries in the code based and refactored them to follow up MVC architecture and modified them to make search controller small and thinner, and moved those load to model. Actually here we used separated set of classes to execute these search queries and by putting this queries in this classes (service classes) anyone can touch the search logic without braking the normal request - response cycle.

After this sub task, we managed to generate search results similar to last state of the system. Then we planned this search for resource based structure. with this format we can enable few more features for this advanced searching tool. Such as search logging, search history, search suggestions, etc.

Introducing solr search server to the system

Since Publiclab is a text based (research notes and wiki pages) system and be supportive to Q&A tool, we used advanced text searching facilities provided by sunspot. And by using one text field (like the search box in we can query easily from sunspot. Sunspot is a middle ware that use to communicate with solr search server through a rails application. Hence by using sunspot libraries we were able to install solr server top of the rails rake. with the previous query setup there were less work to be done for develop solr end points. Mainly text heavy research notes and its attributes were indexed to the solr database and facilitated advanced search page from solr search results.

With the observations of the modifications, step by step I added several additional features to the advanced search page. Few implementations got fail. But most of the tools were added to the page successfully.


one of the key tool that I was Implemented is Grouping. When you do a search for a word by using advanced search page, you can category these search results according to given fields and you can deep look for this results to touch ideal result in from the list.


Results are categorised for updated month

Dynamic search page and typeahead re-design

Real sense of the word, advanced search page is providing lot more details regarding your searching word. It's displaying results in some kind of analytical approach. If you are searching for a simple keywords all around the project, Dynamic search is the ideal one for you.

  1. simple preview
  2. full coverage
  3. auto-complete suggestions
  4. Quick access


Previous typeahead was a heavy library for the system and it consumes more resources to execute the service. It was hard and time consuming to add modifications or use typeahead with few customisations for a separate feature. Hence as a part of this project we re-design usage of typeahead and avoid heavy libraries to complete the particular requirement.


Previously reload was happened after three characters, Now it's reloading for individual keyup event and more faster.

RESTful API for search and it's configurations

This API developed using Grape api framework and by using this api all any api user can retrieve search results through the API. Dedicated end points are designed to cover up all the API level searches and by using grape we were able to deliver the required informations to the api user. Then an user can follow the details and connect to the search tool through the API.

Solr Search engine and my experience.

GSoC experience with my mentor David and publiclab


@Ujitha Just small mistake here. The GSoC project link you gave is the one that links to your dashboard which needs log in. This is the link you should give:

Reply to this comment...

Hi, Ujitha -- thanks for your update -- your project's been really exciting to see, with vast behind-the-scenes changes for a function (search) which has long needed a thorough revamp. I appreciate your effort in bringing the power of Solr/Sunspot to plots2, while putting extra effort into making Solr installation easy for new coders. Very excited to see so much of your work now live on the site, and eager to get the remaining stuff merged, which I'm very optimistic about!

Thanks; it's been great working with you and I hope to do more together!

Reply to this comment...

This is one of the largest merges I've been involved in, and in retrospect I think it may be good to be more careful and not merge master branch code into a feature branch before a final rebase -- the intermixing of commits led to some difficulties we could maybe have avoided if we'd just done one clear run-through as a rebase, and avoided any merge commits at all. Or tried to break it into smaller pieces and done a series of smaller rebase/merges instead of just two giant ones. But we're doing pretty well, and I'm sure we'll figure it out!

Reply to this comment...

Login to comment.