Saturday, 29 March 2014

RStudio/Shiny server on digital ocean fuelled by hot chocolate and a kit-kat chunky

Introduction

A number of people had told me about digitalocean recently so when a new project came along I decided to try it out. Digital Ocean I had gathered was a cloud hosting platform in the vein of amazon and co. I had used AWS previously so I was interested to see how this platform would compare.

Setting up

Creating an account involved giving an email address and creating a password. After this an option was given to load credit onto the account. Paypal was accepted so I used this option to credit the account. After this I was taken to a basic screeen with a big green button labelled create. Seemed simple enough so I clicked it.

Stunning Stats

The next screen presented the various options for the instance. I choose the 1GB / 1 CPU 30GB SSD Disk 2TB Transfer weighing in at $10 a month. I chose Singapore for my region. This was a plus point if you were in South east Asia like me. I picked ubuntu 12.04 as the OS. Apart from when installing tex live 2013 ubuntu 12.04 and I usually got on. There was an applications tab with things like LAMP on Ubuntu 12.04 and GitLab 6.6.5 CE. This was neat nothing stat orientated however but the option to have LAMP running could have been useful. After scrolling down and choosing these options I was present with a large green create droplet button which I pressed.

Droplets

It took about a minute to create my "droplet". Upon creation I was presented with a tab based control panel

Power
Access
Resize
Snapshots
Settings
Graphs
Destroy

It all seemed fairly straightforward. I picked access which led to a large blue connect button. This led to failure initially as I had a vpn running. Turning of the vpn allowed the vnc to connect and we were ready to go. Initially I had to login as root with a password that digital ocean emailed once the droplet was created.

Setting up ubuntu

I was now logged in. The first thing to do was change my root password

$ passwd

With that done I set about setting up a non root account john

$ adduser john

I didnt want user john to be a second class citizen so I gave the user root access.

$ visudo

brought up a nano editor and I copied the root settings for user john:

# User privilege specification
root    ALL=(ALL:ALL) ALL

became

# User privilege specification
root    ALL=(ALL:ALL) ALL
john    ALL=(ALL:ALL) ALL

Pressing ctrl+x and saving I figured I was done. Next I would look to install rstudio server.

RStudio Server

I followed the guide at https://www.rstudio.com/ide/download/server.html .

$ nano /etc/apt/sources.list

At the bottom of this file I added

deb http://cran.stat.nus.edu.sg/bin/linux/ubuntu precise/

as the server was based in singapore.

From the cran project README

The Ubuntu archives on CRAN are signed with the key of Michael Rutter with key ID E084DAB9. To add the key to your system with one command use (thanks to Brett Presnell for the tip):

$ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9

Now I was ready to begin installing rstudio. I updated the sources and installed R.

$ apt-get update
$ apt-get install r-base-dev

We then checked if R was installed.

Now we tried to install RStudio Server

$ apt-get install gdebi-core
$ apt-get install libapparmor1
$ wget http://download2.rstudio.org/rstudio-server-0.98.501-amd64.deb
$ gdebi rstudio-server-0.98.501-amd64.deb

If everything had gone smoothly I should now be able to log into rstudio remotely.

Excellent everything had gone smoothly.

Shiny Server

Next up was shiny server. The relevant rstudio guide was at https://www.rstudio.com/shiny/server/install-opensource . Firstly I installed the shiny package as root.

$ R -e "install.packages('shiny', repos='http://cran.rstudio.com/')"

I then ran the commands to install shiny server

$ wget http://download3.rstudio.org/ubuntu-12.04/x86_64/shiny-server-1.0.0.42-amd64.deb
$ gdebi shiny-server-1.0.0.42-amd64.deb

That seemed to work. Now I needed to setup the shiny server so that user john could serve shiny apps from a Shinyapps folder in his home directory. When I last did this (version 0.4 or so) it required adding some detail to /etc/shiny-server/shiny-server.conf. So lets try and see if that still works.

$ mkdir /etc/shiny-server
$ touch /etc/shiny-server/shiny-server.conf
$ nano /etc/shiny-server/shiny-server.conf

I added the following to this file:

# Define the user we should use when spawning R Shiny processes
run_as shiny;

# Define a top-level server which will listen on a port
server {
  # Instruct this server to listen on port 3838
  #listen 3838 127.0.0.1;
  listen 3838;

  location /userApps {
    user_apps;
    directory_index on;
  }
}

It was a very simple configuration script that would allow users to serve shiny apps from a ShinyApps folder in their home directory. In theory all I needed to do now was to login to rstudio server and create such a setup. I did that and added the example 01_hello from the shiny package. I now navigate to http://128.199.255.233:3838/userApps/john/helloapp/ and amazingly its all there.

Bring your life belt

Often one of the biggest problem people have with VPS and R involves RAM. When people use AWS EC2 micro instances or digital river $5 512mb droplets they experience the following:

www.inside-r.org

For some reason, when I run install.packages() to install any package, it always just dumps the source package to the /tmp folder and does not actually install packages.

www.stackoverflow.com

When trying to install certain (or all?) R packages, I always bump into the following type of error, which seems to me to be not specific to a particular package, but rather specific to for my R environment.

www.stackoverflow.com

I installed the latest version of R for Ubuntu. I enter R, no issues at all, gives me the latest version, and I can load native packages. But When I try to install new packages, they download, but I get nothing.

What is happening? Well they are running out of RAM and R is silently failing. Normally on your desktop this wouldnt likely be a problem as you would have swap. We can check for swap on our current server.

$ swapon -s

If the result returns empty then we need to assign some swap to this droplet. We can check the current storage state using df:

$ df

We are going to make a 512mb swap space

$ dd if=/dev/zero of=/swapfile bs=1024 count=512k

An explanation of the size is here. We then create a swap area and activate it.

$ mkswap /swapfile
$ swapon /swapfile

Now rerunning swapon -s we can see:

This file will last on the virtual private server until the machine reboots. To ensure it returns on reboot we need to add an entry to /etc/fstab

$ nano /etc/fstab

and add

/swapfile       none    swap    sw      0       0

The swap area has an attribute "swappiness" which we would like to set to 10

$ echo 10 | sudo tee /proc/sys/vm/swappiness
$ echo vm.swappiness = 10 | sudo tee -a /etc/sysctl.conf

An explantion of swappiness is found here. Finally we set the permissions of the swap area so only root can access:

$ chown root:root /swapfile 
$ chmod 0600 /swapfile

On the VNC that all looks like this:

Conclusion

Digital Ocean has a very nice simple interface. Setting up a "droplet" is a breeze. The only tricky part with the installation process is using the command line and nano if you are not used to it. Installing RStudio Server and Shiny Server is now very straightforward. What did it cost? $10 for a month on the package I picked, 2 cups of hot chocolate and a kit-kat chunky.

UPDATE: I added a section on including a swap file sourced from here. This is slightly more involved but I have given some pointers to extra information.