#1958: Replace ferret with Solr

Type: FeatureItem Feature: FullTextSearch Tags: BSC
ScheduledFor: 0.38.0 Assigned to: BraulioBhavamitraBO, RafaelMartins Sites:  
Priority: 5 Status: Done  

Why choosing Solr:

  • As said in AI1611, it’s time to get over ferret... It may be usefull as a small-scale full text search engine, but it is definitively not suited for Noosfero.
  • In 1611, sphinx was being thought as an alternative. It’s really faster, and much better, than ferret, but it doesn’t connect to sqlite and its connector to RoR?, thinking_as_sphinx, do not easily support facets.
  • On the other hand, Solr, which is synchronized to Lucent, is a really mature search machine, with some important advantages to sphinx, such as connection to any database, indexing of PDF and office suites, stemming for brazilian portuguese language, etc (for a complete list: http://beerpla.net/2009/09/03/comparison-between-solr-and-sphinx-search-servers-solr-vs-sphinx-fight/ )
  • So Solr would be the easiest way of getting rid of ferret and finally improving the search-indexing in Noosfero, a long awaited feature.

Connecting Solr to Rails:

There are basically two Rails interfaces: sunspot (http://outoftime.github.com/sunspot/)
  • Actively maintained and constantly updated to latest Solr features.
  • Well documented.
  • Support only Rails 2.3 and 3.0.
acts_as_solr (https://github.com/mattmatt/acts_as_solr) Sunspot is a much better alternative, but only viable after the Noosfero upgrade. Migrating to it would be as simple as changing lib/acts_as_searchable.rb and some parameters

Implementation planning:

Use solr server 1.4.
  • Offer solr configurations files to be used by the system administrator with Solr
  • It can’t be embedded with Noosfero (it adds 11mb of mainly binary files). acts_as_solr and and sunspot does embbed it with jetty web server
  • Packages are available Ubuntu (solr-jetty) but not yet for Debian stable, so instructions should be made available to guide the user to installed it manually.
Replace all ferret usages basically by modifying lib/acts_as_searchable.rb
  • Change method acts_as_shttps://github.com/mattmatt/acts_as_solr/blob/master/lib/class_methods.rbearchable to use acts_as_solr (and change options used at article, product and profile)
  • Change method find_by_contents to use find_by_solr (https://github.com/mattmatt/acts_as_solr/blob/master/lib/class_methods.rb) (and change specific options to ferret used in it, like in app/models/environment_finder.rb)
  • Offer compatibility with the old query method (ferret + SQL) using a combined Solr plus SQL method
Use acts_as_solr while Noosfero don’t run on Rails 2.3. Later a switch to sunspot is possible using the same strategy at b.

-- BraulioBhavamitraBO -- 23 Mar 2011

Patches sent to noosfero-dev are available at https://github.com/brauliobo/noosfero-bsc

-- BraulioBhavamitraBO - 23 Apr 2011

Merged with mainline and Leandro's rails 2.3.5 port patch. Checkout from https://github.com/brauliobo/noosfero-bsc

-- BraulioBhavamitraBO - 16 May 2011

https://gitorious.org/noosfero/noosfero/merge_requests/3

-- BraulioBhavamitraBO - 18 May 2011

Merge request open at https://gitorious.org/noosfero/noosfero/merge_requests/3

-- BraulioBhavamitraBO - 18 May 2011

Add comment
You need to login to be able to comment.
 

ActionItemForm edit

Title Replace ferret with Solr
ActionItemType? FeatureItem
Priority 5
Tags BSC
Feature FullTextSearch
Plugin
ResponsibleDevelopers BraulioBhavamitraBO, RafaelMartins
ScheduledFor? 0.38.0
AffectsVersion?
Status Done
Ticket SAC:
Topic revision: r20 - 23 Jul 2012 - 09:58:07 - DanielaFeitosa
 
Translations: English
Search on Docs:
   
ActionItem Search:

Copyright © 2007-2014 by the Noosfero contributors
Colivre - Cooperativa de Tecnologias Livres