Apache Solr Installation

   Solr Installation and New core Configuration:


Apache Solr achieve fast search responses because, instead of searching the text directly, it searches an index. This is like retrieving pages in a book based on a word by scanning the index at the back of a book, as opposed to searching every word of every page of the book. This type of index is called an inverted index, because it inverts a page-centric data structure (page->words) to a keyword-centric data structure (word->pages). Solr stores this index in a directory called index in the data directory.

Apache Solr is powered by Lucene, a powerful open-source full-text search library, under the hood. The relationship between Solr and Lucene, is like that of the relationship between a car and engine.

In Solr, a Document is the unit of search and index. An index consists of one or more Documents, and a Document consists of one or more Fields.


Solr Installation and New core Configuration:

1. Installation-
Download solr version & then
extract solr-version
go inside solr-version/example/ and run command: java -jar start.jar
see this will show solr screen in browser by following url
http://localhost:8983/solr/


2. New core Configuration-

solr-version/example/solr/mkdir coll1
inside coll1
mkdir conf
inside conf make schema.xml with following content:

<?xml version="1.0" encoding="UTF-8" ?>
<schema version="1.5">
  <fields>
    <field name="id" type="string" indexed="true" stored="true" required="true"/>
    <field name="addr_from" type="string" indexed="true" stored="true" required="true"/>
    <field name="addr_to" type="string" indexed="true" stored="true" required="true"/>
    <field name="subject" type="string" indexed="true" stored="true" required="true"/>
  </fields>
  <uniqueKey>id</uniqueKey>
  <types>
    <fieldType name="string" class="solr.StrField" />
  </types>
</schema>

inside conf make solrconfig.xml with following content:

<?xml version="1.0" encoding="UTF-8" ?>
<config>
  <luceneMatchVersion>LUCENE_43</luceneMatchVersion>
  <requestDispatcher handleSelect="false">
    <httpCaching never304="true" />
  </requestDispatcher>
  <requestHandler name="/select" class="solr.SearchHandler" />
  <requestHandler name="/update" class="solr.UpdateRequestHandler" />
  <requestHandler name="/admin" class="solr.admin.AdminHandlers" />
  <requestHandler name="/analysis/field" class="solr.FieldAnalysisRequestHandler" startup="lazy" />
</config>

come back in coll1 dir
make core.properties here with following content:

#Written by CorePropertiesLocator
#Wed May 29 16:26:21 IST 2014
name=coll1
config=solrconfig.xml
schema=schema.xml
dataDir=data

make core.properties.unloaded here with following content:
coll1

then start solr: go inside solr-version/example/ and run command: java -jar start.jar
see this will show solr screen in browser by following url
http://localhost:8983/solr/

and if in your coll1 dir data dir will be made automatically beside conf. means your core is created.

import csv file in newly created core in solr:
create input1.csv file inside /solr-version/example/exampledocs/
where post.jar file is also available then run following cmd:

[user@localhost exampledocs]$ java -Dauto -Durl=http://localhost:8983/solr/collection1/update -jar post.jar input1.csv

you will like this:
SimplePostTool version 1.5
Posting files to base url http://localhost:8983/solr/collection1/update..
Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file input1.csv (text/csv)
1 files indexed.
COMMITting Solr index changes to http://localhost:8983/solr/collection1/update..
Time spent: 0:00:00.450

Comments

Post a Comment

Popular posts from this blog

Setup Nginx as a Reverse Proxy for Thingsboard running on different port/server

In this blog, we will Setup Nginx as a Reverse Proxy for Thingsboard IOT dashboard running on different port Requirement: Ubuntu Server Thingsboard installed on Ubuntu server Nginx installed on Ubuntu server For installing please refer thingsboard documentation page windows:  https://thingsboard.io/docs/user-guide/install/windows/ Linux:  https://thingsboard.io/docs/user-guide/install/linux/ Let us know in comment if you face any issue in installation we will help you. NGINX installation: sudo apt-get install nginx sudo service nginx status sudo service nginx start Open your browser and you will see the default NGinx home page, nginx also runs on default port 80 In some case if you don't see default page up then check: If using cloud machine then need to open "http port 80" on 0.0.0.0/0 or a specific ip If still don't see UI then restart server, sometime it comes after restart. Configure...
Read more

How to auto re-launch a YARN Application Master on a failure.

1)Use Case: The fundamental idea of Hadoop2 (Map-Reduce + Yarn) is to split up the two major functionalities of the JobTracker, resource management and job scheduling/monitoring, into separate daemons. The idea is to have a global ResourceManager ( RM ) and per-application ApplicationMaster ( AM ). An application is either a single job in the classical sense of Map-Reduce jobs or a DAG of jobs. The ResourceManager and per-node slave, the NodeManager ( NM ), form the data-computation framework. The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system. The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks. The ResourceManager has two main components: 1. Scheduler: is responsible for allocating resources to the various running applications. 2. Appl...
Read more

Hive partitioned tables Issue with schema & PrestoDB

It's very strange for Hive and PrestoDB user that the schema of partitioned tables in Hive is defined on partition level as well. Partition level schema and table level schema in Apache Hadoop is letting complex. Let's see the details in below example: Table schema In Hive you can change the schema of an existing table. Let’s say you have a table: CREATE TABLE TEST1 (ID INT,  NAME STRING,  RATING INT ) ROW FORMAT DELIMITED FIELDS TERMINATED BY "\t" LINES TERMINATED BY '\n' STORED AS TEXTFILE; We will focus on the third column named rating. We load few records for hdfs to this table. The file looks like this: 1         john   3.5 2   Dav   4.6 3   andy   5 hive -e "load data inpath 'input.txt' into table test1" The third column has some decimal values, but we have defined this column as integer, so the we won’t see the decimal part in the data: hive&gt; select * from te...
Read more