How to Setup Mongo and Solr on Docker for Sitecore xDB: Part 2
Vasiliy Fomichev
In How To, Infrastructure Posted July 15, 2015
In How to Setup Mongo and Solr on Docker for Sitecore xDB: Part 1 we’ve looked at the importance of having access to immutable infrastructure and how Docker can help Sitecore developers with easily creating and maintaining Mongo and Solr instances for Sitecore xDB. In the following post, we will walk through creating our own Docker container for Mongo and Solr, as well as introducte Solr for Sitecore Docker repository.
How to Install Docker
Docker was created to run in a Linux kernel, however, Microsoft and other companies have worked on what has become boot2docker, an app that allows to install and communicate with a Docker container on a Virtual Box running a stripped down Linux OS based on Tiny Core Linux.
Docker website provides a great tutorial on how to install boot2docker, so simply follow these steps and move on to Mongo setup once done.
How to Setup a Mongo Docker Container for Sitecore
This setup was very simple. It was so quick, I wasn’t sure that I did everything properly, but I did and here are the steps –
- Launch the boot2docker bash window (double-click on boot2docker icon on the Desktop)
- Note the IP address of the hositing machine that came up in the bash window
Docker host IP address
- Run the following command
docker run -d -p 27017:27017 mongo
That’s it! Docker will now download the image from the Hub reposiroty and install it (this all happened under one minute with my home connection powered by a starving hamster in a wheel) Once the installation is complete, you can access the new Mongo instance by connecting to {IP address of the host}:27017 with your Mongo DB tool of choice (in my case I used MongoVUE and connected to 192.168.59.103:27017).
Note the –p 27017:27017 switch, it creates a port forward required for boot2docker VM to forward requests to the container and the image did not have a username prepended as it’s one of the official Docker images.
Having Mongo setup, you can now configure and customize it in a way you want. For more information on this image navigate to the Docker Hub and click on the mongoDB tile under official repositories.
How to Setup a Solr Docker Container for Sitecore
Solr container was a bit more “fun” to setup, as I needed to transfer the schema.xml file generated by Sitecore into the container, plus I’ve uncovered a few bugs in boot2docker, workarounds for which I added at the end of this article.
In the following steps we will:
- Download the currenlty latest stable image of Solr (v 5.2.1) and install it in a Docker container
- Setup the shared volume for transferring files between the Windows OS and the Docker container
- Setup and configure cores required for Sitecore 8 (assuming Sitecore 8 has already been installed)
- Commit our changes and create an image for distribution
- Export the image and upload it to Docker Hub
On we go!
docker run -v //c/Users/{path to shared folder in Windows}:/winshare --name solr -d -p 8983:8983 -t makuk66/docker-solr
This command downloads and installs the latest most popular Solr image from Docker Hub (notice that once you’ve downloaded the image, you can spin off new instances in just a second!). It also sets up port forwarding, passing on requests targeted at the Docker host to the container. Lastly, the part that took a while for me to figure out was mounting a shared volume in the container, which would allow us to move the schema.xml files to the Solr container and back. By default, boot2docker shares the C:\Users folder with the host and mounts it as /c/Users. With the command above we are taking advantage of this default share and mounting a new folder /winshare to a folder under /c/users. In my case the command looked as follows: docker -v //c/Users/Vasiliy/Desktop/Docker_Share:/winshare –name solr -d -p 8983:8983 -t makuk66/docker-solr.
Notice the double forward slash for the “Windows” path, this is the trick that keeps boot2docker from interpreting the path into Windows format.
This installation will take only a couple of minutes (depending on the speed of your Internet connection), so this is a good time to grab a quick cup of coffee. After the install completes Solr dashboard can be accessed via {IP address of the host}:8983/solr (http://192.168.59.103:8983/solr in my case). Once everything looks good, we can then extract schema.xml and create one for use by Sitecore.
Let’s SSH into the container to grab our base schema.xml
docker exec -it solr bash
Now let’s download the schema.xml into our Windows shared folder –
cp /opt/solr-5.2.1/server/solr/configsets/basic_configs/conf/schema.xml /winshare
We should find the copied schema.xml file in the folder we mounted to /winshare volume to in our Solr installation command. Next – generate a new schema.xml file to use in Solr using following steps outlined in Sitecore Search Scaling Guide. Also don’t forget about to implement changes to schema.xml for Solr 4.8 and above.
Once the file has been generated copy it back into the folder shared with the container. This is also a good time to configure Sitecore 8 for Solr. The Sitecore Search Scaling Guide provides the steps, Dan Solovay also wrote a good walkthrough of Solr setup for Sitecore. Do not blindly copy all binary files from the Solr package, make sure you understand what really belongs in your particular instance! Complete all the steps required for Sitecore only, we will setup Solr cores next.
At this point we should still be SSH’d into our container running Solr, if not, re-launch the boot2docker bash environment and run the “exec” command above to connect. Now we are ready to setup Sitecore cores.
Solr 5.2.1 core setup varies a little from the one for versions 4 or older, outlined in the Sitecore Search Scaling Guide and Dan’s tutorial. Solr 5.2.1 no longer has the default collection1 core, but rather has a set of sample “base” core configurations under /opt/solr-5.2.1/server/solr/configsets/. If you try to create a core using the Solr Core Admin tool, you will get the following error –
Error CREATEing SolrCore ‘new_core’: Unable to create core [new_core] Caused by: Can’t find resource ‘solrconfig.xml’ in classpath or ‘/opt/solr/server/solr/new_core/conf’
Solr starting with version 5 requires core configuration folders along with solrconfig.xml to already exist in the folder specified when creating a new core. There is a way to overcome this with solr create_core command, then configure it manually, however, we’ll take a shortcut and use one of the “base” core configs. So, let’s get to it –
cp -af server/solr/configsets/basic_configs server/solr/{corename}
Then let’s copy our new schema.xml file generated by Sitecore into the new base core folder –
cp /winshare/schema.xml /opt/solr-5.2.1/server/solr/{corename}/conf/
Using the Solr Core Admin, add a core with the same name as the one we gave to the target folder in the command above. These steps should be repeated for all cores required by your Sitecore 8 installation (the screenshot below displays a full default set of cores).
Solr Core Admin with Sitecore cores.
Once the cores have been setup we can now update our Sitecore instance to point to {IP address of the host}:8983/solr in Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config. Lastly, login to Sitecore and rebuild the indexes. Voila! Solr is running in a Docker container.
How to Create a Solr Docker Sitecore 8 Image
Well, that was a good amount of work, especially for developers unfamiliar with Linux, so it would be great, if we didn’t have to do all that ever again to setup Solr for Sitecore 8. Docker images to the rescue! Once we create an image from our container, we can upload it to Docker Hub or export it and share with others, or just keep it handy for future use. With Docker creating and exporting an image from a container is a breeze (exit from the container bash, if you are still there by using “exit” command) –
docker commit -a "{commit comment}" -m "{your name}" {container name or id} {docker username}/{image name}:{image_tag}
We should now have a new image created with the name {docker username}/{image name}. Note that the username/image format is only required if you are planning to upload it to the Hub. We can verify the new image existence by using docker images command. The -a parameter allows us to add a comment, and -m – the author’s name. This is very similar to Git – none of your changes get actually saved to the image, until you do a commit. The {image_tag} is optional.
If you want to export the image run the following command –
docker save {docker username}/{image name}> {shared path with Docker under C:\Users}/{file name}.tar
The command looked like this in my case – docker save sc8solr > /c/Users/Vasiliy/Desktop/Docker_Share/sc8_solr.tar. To load the image on another machine use docker load -i {path to image}, and then docker run … to start create a new container out of it.
However, if you would like to upload the image to the hub, you would first need to register at https://hub.docker.com/account/signup/ and then login from the boot2docker bash window –
docker login -u XXX -p XXX -e XXX
Notice that you have to use the full login command providing credentials inline, there is currently a bug in boot2docker making the command prompt hang at the Username: prompt if simple docker login is used.
Once logged in, we can push the image up to the Hub –
docker push {docker username}/{image name}
Voila! Now you can spin out as many Docker images as you want. Just try using docker run –d –p {port}:{port} {docker username}/{image name} several times. Every time you are creating a new Solr instance with preexisting configuration for Sitecore 8 in just a second! Amazing, isn’t it (take a note of fun ato-generated container names 🙂 )?
Solr for Sitecore Docker Repository
Solr for Sitecore is the initiative to provide base Solr installations for use with Sitecore. Each installation comes preconfigured with cores and schema.xml required by the version of Sitecore specified by the tag when creating a container. Once installed Solr version should be ready to be connected to Sitecore using default search configuration settings.
Solr for Sitecore URL: https://registry.hub.docker.com/u/vasiliyfomichev/solr-for-sitecore/
Solr for Sitecore is build using a Dockerfile, the contents of which you can view either on the page above or on Github by going into tag folders. This way the scripts running to create an image are exposed, unsuring all users that there is nothing tricky happening behind the scenes ))
Make sure to read the Information page on how to properly run Solr for Sitecore containers.
I hope readers will see the enormous benefit of using Docker for Mongo and Solr instances with Sitecore. I encourage all Sitecore developers to further research Docker, as there are many more features that can be of use. I also encourage developers to create and share their own image in Docker Hub, or contribute to Solr for Sitecore by submitting pull requests through Github!
Generating Images with Bootdocker File
If your image exports become too big and you don’t feel like packing several GBs everywhere, they can alternatively be distributed using a Dockerfile, which contains a number of instructions (commands) that run to create the image. For more information check out the article on Dockerfile reference on the Docker website.
How to Avoid Certificate Errors after Shutting Down Docker
Lastly, there is a bug in version 1.7 of boot2doocker which throws a certificate error after running boot2docker down / reset, or turning the VM off via Virtual Box. What you want to do is run boot2docker save instead, which saves Docker’s state and boot2docker start to get it back up.
If you do end up getting the following error –
An error occured trying to connect: Get https://{host IP}/v1.19/{command}/json: x509: certificate is valid for 127.0.0.1, 10.0.2.15, not {host IP}
I found it easiest to save all images you would like to retain, delete the VM, boot2docker delete, close the shell and relaunch boot2docker, then load all images back in, boot2docker load -i {path to image}.
Workaround for Docker Connection Timeout Issues
For some reason port forwarding for the VM can get get knocked out by boot2docker, in which case you would get the following error:
An error occured trying to connect: Post https://192.168.59.999:2376/v1.19/auth: dial: tcp 192.168.59.999: ConnectEx tcp: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
To fix this, simply add a port-forward manually to the boot2docker-vm in Virtual Box Network Settings as shown below (make sure to only update the NAT interface)
Updating boot2docker port-forward.
Cheers!