🕵️ Setup scripts for spy, a monitoring and database server for dahak workflows.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
Charles Reid c92aacb64c use dahak-metagenomics official for cloud init script 6 years ago
cloud_init use dahak-metagenomics official for cloud init script 6 years ago
docker Reorg: moving container directories to docker; updating readmes. 6 years ago
img new dotfiles structure 6 years ago
tasks_sudo reconcile sudo tasks between yeti and spy 6 years ago
tasks_user reconcile dahak spy with dahak yeti 6 years ago
.aliases new dotfiles structure 6 years ago
.bash_profile fix bash history t prefix issue 6 years ago
.bash_prompt new dotfiles structure 6 years ago
.bashrc new dotfiles structure 6 years ago
.gitconfig new dotfiles structure 6 years ago
.gitignore update bootstrap and gitignore to reflect repo content 6 years ago
.hushlogin new dotfiles structure 6 years ago
.inputrc new dotfiles structure 6 years ago
.vimrc new dotfiles structure 6 years ago
LICENSE 2 legit 2 init commit 6 years ago
README.md add cloud init scripts and mirroring yeti repo org 6 years ago
bootstrap.sh update bootstrap and gitignore to reflect repo content 6 years ago

README.md

dahak-spy

The dahak-spy repo contains files for setting up a node for monitoring and logging data from nodes working on dahak workflows.

Services

dahak spy runs most of its services through Docker (some stock, some using custom Dockerfiles). It runs:

  • databases (prometheus, mongodb, mongoexpress)
  • monitoring (netdata)
  • dashboards (grafana)
  • messaging (zmq)

See docker/README.md for more info on the services that are running in containers.

Cloud Deployment

To deploy these scripts to a cloud node, use the cloud init functionality of Ubuntu and upload the cloud init script

By Hand Deployment

Installing by hand is a three-step process:

  • Install git
  • Run sudo init script
  • Run user init script

Step 1: Install Git

To install git:

apt-get update && apt-get install -y git

Now you can check out a copy of the repo:

git clone https://github.com/charlesreid1/dahak-spy.git
cd dahak-spy/

Step 2: Run Sudo Init Script

To run the sudo init script, which calls several other scripts, run:

# as the sudo user:
$PWD/sudo_init/sudo_init.sh

Step 3: Run User Init Script

To run the user init script, run:

# as the regular dahak user, 
$PWD/sudo_init/sudo_init.sh

# or if you are still sudo,
sudo -H -i -u dahak $PWD/sudo_init/sudo_init.sh

Using the Dotfiles

The dotfiles are installed for the regular user on the spy node. These dotfiles make it easy to define an environment, either for all users (by changing the dotfiles in the repo) or for an individual user (using site-specific dotfiles).

The $PATH is set in .bash_profile

The prompt is set in .bash_prompt

The aliases are set in .aliases

To set your own aliases, source your own dotfiles, or otherwise insert steps into the dotfiles initialization process, use the ~/.extras file.

To set your git credentials, add the following to the ~/.extras file:

# Git credentials
# Not in the repository
# This prevents people from using incorrect github credentials
GIT_AUTHOR_NAME="<<< your name here >>>"
GIT_AUTHOR_EMAIL="<<< your email here >>>"

GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME"
GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL"

git config --global user.name "$GIT_AUTHOR_NAME"
git config --global user.email "$GIT_AUTHOR_EMAIL"

Screenshots

Netdata running on a beefy node @ 10.11.0.194:

dahak-yeti netdata instance

Netdata running on dahak-spy @ 10.11.0.192 (spy monitors itself and other nodes):

dahak-spy netdata instance