Installing SolrCloud for Sitecore 9 Part 1

by Corey McClelland

SolrCloud is the name of the functionality in Solr that supports high availability and fault tolerance through the use of a cluster. Using SolrCloud takes away a single point of failure that exists in many Sitecore solutions. Our goal is to create a 3 node SolrCloud cluster.

ZooKeeper

ZooKeeper is the underlying system that maintains the configuration for the SolrCloud environment. It can be thought of as a distributed filesystem that all Solr nodes have aceess to.

Installation

Let's install ZooKeeper onto our 3 node cluster. It can theoretically be installed on its own cluster but it's more common to put it on the same servers that Solr will use.

For this installation I'm using ZooKeeper v3.4.9.

Install the Java Runtime Environment
Since both ZooKeeper and Solr are Java apps, we will need to install the JRE. I used v1.8 for this installation as the latest version had some compatibility issues with Solr.

After the installer completes you will need to set the JAVA_HOME system environment variable. Its value should be: C:\Program Files\Java\jre1.8.0_172.

Next we need to update the Path environment variable to include the JAVA_HOME variable: %JAVA_HOME%\bin;.

Install ZooKeeper
Extract the ZooKeeper package to the data drive of your server. In our case it will be located in D:\zookeeper-3.4.9.

Create a system environment variable for ZooKeeper. It should be called ZOOKEEPER_HOME and have a value of D:\zookeeper-3.4.9.

Update the system path to include the new variable:

Note that like the JAVA_HOME we've appended the bin directory to the path.

Next we need to copy the %ZOOKEEPER_HOME%\conf\zoo_sample.cfg to zoo.cfg. Open the file and change the dataDir variable to point to this path: dataDir=d:\\zookeeper-3.4.9\\data. Make sure this directory exists.

Create a file called "myid" in the data folder. It must contain the integer id of the server.

Add the DNS entries for the servers in the cluster to the bottom of the zoo.cfg file:

server.1=SERVER1.COMPANY.COM:2888:3888
server.2=SERVER2.COMPANY.COM:2888:3888
server.3=SERVER3.COMPANY.COM:2888:3888

Confirm that this zookeeper node starts correctly by running "zkserver" from the command line.

Run ZooKeeper as a Service
Create the directory %ZOOKEEPER_HOME%\service. Download a copy of prunsrv.exe to this folder. Create a new file called ZooKeeperInstallService.bat with the following contents:

@echo off

SET HOME=d:\zookeeper-3.4.9
SET SERVICE_DIR=d:\zookeeper-3.4.9\service
SET SERVICE_LOGPATH=d:\zookeeper-3.4.9\logs

REM Service configuration
SET SERVICE_NAME=ZooKeeperService
SET PR_INSTALL=%SERVICE_DIR%\prunsrv.exe
SET PR_PIDFILE=servicepid

REM Service log configuration
SET PR_LOGPREFIX=%SERVICE_NAME%
SET PR_LOGPATH=%SERVICE_LOGPATH%
SET PR_STDOUTPUT=%SERVICE_LOGPATH%\stdout.txt
SET PR_STDERROR=%SERVICE_LOGPATH%\stderr.txt
SET PR_LOGLEVEL=All

REM Zookeeper - Classpath Configuration
SET ZOOCFGDIR=%HOME%\conf
SET ZOO_LOG_DIR=%HOME%
SET ZOO_LOG4J_PROP=INFO,CONSOLE
SET ZOOCFG=%ZOOCFGDIR%\zoo.cfg
SET ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain

SET CLASSPATH=%ZOOCFGDIR%
SET CLASSPATH=%HOME%\zookeeper-3.4.9.jar;%CLASSPATH%
SET CLASSPATH=%HOME%\lib\jline-0.9.94.jar;%CLASSPATH%
SET CLASSPATH=%HOME%\lib\log4j-1.2.16.jar;%CLASSPATH%
SET CLASSPATH=%HOME%\lib\netty-3.10.5.Final.jar;%CLASSPATH%
SET CLASSPATH=%HOME%\lib\slf4j-api-1.6.1.jar;%CLASSPATH%
SET CLASSPATH=%HOME%\lib\slf4j-log4j12-1.6.1.jar;%CLASSPATH%
SET PR_CLASSPATH=%CLASSPATH%

SET PR_JVMOPTIONS="-Dzookeeper.log.dir=%ZOO_LOG_DIR%";"-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%"

REM Startup configuration
SET PR_STARTUP=manual
SET PR_STARTMODE=java
SET PR_STARTCLASS=%ZOOMAIN%
SET PR_STARTPARAMS=%ZOOCFG%
 
REM JVM configuration
SET PR_JVM=auto
SET PR_JVMMS=256
SET PR_JVMMX=1024

REM Service Shutdown configuration
SET PR_STOPMODE=exe
SET PR_STOPIMAGE=%SERVICE_DIR%\ServiceKiller.exe
SET PR_STOPPARAMS=%SERVICE_LOGPATH%\servicepid
 
REM Install service
prunsrv.exe //IS//%SERVICE_NAME%

Create the %ZOOKEEPER_HOME%\logs folder if it doesn't already exist.

Run the bat file and confirm that it created a service entry in services.msc. You will also need to change the startup mode to be automatic.

Open the Firewall
Next up is adding the required ZooKeeper ports to the firewall. These TCP ports need to allow incoming connections: 2181, 2888, 3888.

Rinse and Repeat
Repeat these steps on the other two servers in the cluster. Remember to change the server id in the myid file.

Install Solr
Installing Solr will be covered in part 2.