Installing Docker Tools on Ubuntu VMs on Azure to Run Superset
As a first step, we are creating a VM on Azure through the portal. We make sure of 2 things in this example
- Select Ubuntu 18.04
- Make sure the ports 80, 443, 22, 3389 are open
- Make sure the VM has a public ip address so that you can access it using Putty or other secured shell client from your local machine.
Since we are dealing with running Apache superset so we will first login and clone the repository and change the admin password
Cloning Superset from repository and change Default Password
Clone the official repository using
$ git clone https://github.com/apache/incubator-superset.git
The default password is "admin" so we can change it so that no one else knows it
$ cd incubator-superset
$ cd docker
$ vi docker-init.sh
#change the line to your desired password ADMIN_PASSWORD="admin"
Installing Docker
I have followed instructions listed on the url : https://docs.docker.com/engine/install/ubuntu/ using the method "install using the repository". Majority of the steps listed here will be from the url listed above and purpose of documenting this is to note down how to resolve the errors while doing a docker compose.
#SET UP THE REPOSITORY
#Update the apt package index and install packages to allow apt to use a repository over HTTPS:
$ sudo apt-get update
$ sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
Add Docker’s official GPG key:
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
#Verify that you now have the key with the fingerprint 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88, by searching for the last 8 characters of the fingerprint.
$ sudo apt-key fingerprint 0EBFCD88
pub rsa4096 2017-02-22 [SCEA]
9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88
uid [ unknown] Docker Release (CE deb) <docker@docker.com>
sub rsa4096 2017-02-22 [S]
Use the following command to set up the stable repository. To add the nightly or test repository, add the word nightly or test (or both) after the word stable in the commands below. Learn about nightly and test channels.
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
INSTALL DOCKER ENGINE
#Update the apt package index, and install the latest version of Docker Engine and containerd, or go to #the next step to install a specific version:
$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io
To install a specific version of Docker Engine, list the available versions in the repo, then select and install:
a. List the versions available in your repo:
$ apt-cache madison docker-ce
b. Install a specific version using the version string from the second column, for example, 5:18.09.1~3-0~ubuntu-xenial.
$ sudo apt-get install docker-ce=<VERSION_STRING> docker-ce-cli=<VERSION_STRING> containerd.io
In my case it was
$ sudo apt-get install docker-ce=5:20.10.1~3-0~ubuntu-bionic docker-ce-cli=5:20.10.1~3-0~ubuntu-bionic containerd.io
Verify that Docker Engine is installed correctly by running the hello-world image.
$ sudo docker run hello-world
Install Docker Compose
You can install docker-compose using the command below
$ sudo apt install docker-compose
Compose Superset
we will try to compose the downloaded superset by the command below
$ docker-compose up
On running the above command you will face issue where it will complain about the version of the docker-compose.yaml. In order to resolve it, please change the version to 2.2
After making the above change when you run the docker-compose again it will still complain and the error should be "Couldn't connect to docker daemon". In order to resolve this we need to do the following
$export DOCKER_HOST=internalIp of the VM
$ sudo usermod -aG docker <<username>>
$ service docker restart
$ sudo docker-compose up -d
The -d option is to run it in a detached mode so even if your terminal session closes the service would still keep running. When installation is complete and the services are running you can check what are the services running using the command below
$docker-compose ps
:~/incubator-superset$ docker-compose ps
WARNING: The CYPRESS_CONFIG variable is not set. Defaulting to a blank string.
Name Command State Ports
--------------------------------------------------------------------------------
superset_app /usr/bin/docker- Up 8080/tcp,
entrypoint ... 0.0.0.0:8088->8088/tcp
superset_cache docker-entrypoint.sh Up 127.0.0.1:6379->6379/t
redis ... cp
superset_db docker-entrypoint.sh Up 127.0.0.1:5432->5432/t
postgres cp
superset_init /usr/bin/docker- Exit 0
entrypoint ...
superset_node docker-entrypoint.sh Up
/app/ ...
superset_tests_worker /usr/bin/docker- Exit 1
entrypoint ...
superset_worker /usr/bin/docker- Up 8080/tcp
entrypoint ...
Now you should be able to access the superset instance by http://{Private of Public IP}:8088 and login using the username = admin and password = <<password set in docker-init.sh>>
In order to stop the services from running use the command below
$ docker-compose stop
In order to remove the container
$ docker-compose down
