Lucid Imagination Expands Beyond Search with Big Data Application Development Platform

Lucid Imagination expanded its horizons today with the beta release of a new Big Data platform that merges two distinct but complimentary capabilities – Big Data processing/analytics and Big Data search.

LucidWorks Big Data combines Lucene/Solr, the open source, scalable search technology, with Hadoop in one platform for building end-user applications that leverage both structured and unstructured data sets. It includes support for Mahout and R for machine learning and advanced analytics, respectively.

The platform is optimized for cloud deployments but can also be deployed on-premise, and includes a secure REST API for application integration and administration, according to the company.

The company envisions typical deployments to play out this way:

With LucidWorks Big Data, Lucid Imagination equips technologists and business users with the ability to initially pilot Big Data projects utilizing technologies such as Apache Lucene/Solr, Mahout and Hadoop, in a cloud sandbox. Once satisfied, the project can remain in the cloud, be moved on premise or execute within a hybrid configuration.

If you’re not familiar with Lucid Imagination, the company commercializes the open source Lucene/Solr distributed search platform. When I spoke with CEO Paul Doscher back in January, he told me his company’s mission is to make Lucene/Solr enterprise ready for application development purposes by adding ranking and relevancy capabilities, better user interface tools, and tighter security to the open source core.

Our secret sauce, Doscher told me, is that we have optimized Lucene/Solr to be more efficient. Specifically, the LucidWorks platform allows users to search large volumes of Big Data on less hardware than vanilla Apache Lucene/Solr and/or proprietary enterprise search technologies which translates into lower cost, he said.

With the new platform, Lucid Imagination is clearly attempting to beef up its capabilities to be viewed as a more comprehensive Big Data application development platform rather than just a Big Data enterprise search-based application development platform. That’s a good move, as increasingly the value of Big Data is coming from applications built on top of the infrastructure – be it Hadoop, Lucene/Solr or other approach — rather than the plumbing itself.

Below check out Doscher when he visited theCUBE at Strata Conference back in March.

About Jeffrey Kelly

Jeffrey F. Kelly is a Principal Research Contributor at The Wikibon Project, an open source research and advisory firm based in Boston. His research focus is the business impact of Big Data and the emerging Data Economy. Mr. Kelly's research has been quoted and referenced by the Wall Street Journal, the Financial Times, Forbes, CIO.com, IDG News, TechTarget and more. Reach him by email at jeff.kelly@wikibon.org or Twitter at @jeffreyfkelly.