Tuesday, June 5, 2012

Search is the Dominant Metaphor for Working with Big Data

Found another good article on dzone.com from Eric Pugh of the Apache Software Foundation. He wrote down some quick thoughts around his attendance of the recent LuceneRevolution 2012. He has a few more articles on dzone.com here.

If you'r enot paying attending to search, you really need to be. Search absolutely is a core component of Big Data processing. It's great that you can process data with Hadoop, but what do you do with it? Search/Lucene/SOLR/ElasticSearch can be used on both the input and output side. You can pre-process your data with Lucene analyzers or use search to define the data sets you need to run through Hadoop. You can also have search as your endpoint, especially in a streaming process to make your processed data available to your users.

Search is king, and no where is that more evident than with Big Data!

  - Craig