From 5fb9253466fb3bdc88ae386408c33904358810bc Mon Sep 17 00:00:00 2001 From: Richard Hitier Date: Mon, 6 Nov 2023 10:03:30 +0100 Subject: [PATCH] More documentation --- .gitignore | 2 ++ README.md | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------- config.yml-dist | 3 +++ 3 files changed, 117 insertions(+), 39 deletions(-) diff --git a/.gitignore b/.gitignore index fc3a8fe..e2b7b26 100644 --- a/.gitignore +++ b/.gitignore @@ -22,3 +22,5 @@ venv/* VISITS .ipynb_checkpoints/ config.yml +compose.override.yaml +compose.env diff --git a/README.md b/README.md index 2872888..06432b8 100644 --- a/README.md +++ b/README.md @@ -1,82 +1,159 @@ # Heliopropa Web -A web site writen in flask for the visualisation of solarwind propagation simulations. +A website writen in flask for the visualisation of solarwind propagation simulations. Builds and serves heliopropa's visualizations. It's a `flask` webserver, serving `d3.js` plots. + It also gathers NetCDF data from AMDA, and serves it as CSV to the plotter. -## Files Overview +## Quick Start -- `config.yml` : the main configuration file. -- `web/run.py` : the front controller, holding most of the code. -- `web/run.log` : you'll find more information about failures in the log. -- `web/view/home.html.jinja2` : the HTML template. -- `web/static/js/swapp.ls` : most of the javascript client-side. +You can very quickly have a running app out of the box. -## Quick Start +Just: docker compose build - docker compose up + docker compose up -d + +and point your browser to `http://127.0.0.1:8080/` + +See later about configuring port number -## Docker compose configuration +## Docker + +Preferred way of running heliopropa app. + +### Configuration docker stack mainly consists in two files: -- `Dockerfile` for the container building -- `compose.yaml` for the volumes configuration and links with nginx service and gunicorn +- `Dockerfile` : for the container building +- `compose.yaml` : for the volumes configuration and links with nginx service and gunicorn -### Volumes configuration and `compose.override.yaml` + +And your can tweak port configuration thanks to `$PORT` environment variable. +(@see ports: section in `compose.yaml` ) + +Either by setting it in your shell + + PORT=8081 docker compose up -d + +Or in a config file: + + cp compose.env-dist compose.env + ${EDITOR} compose.env # set to your needs + docker compose --env-file=compose.env up -d + +Or + + cp compose.env-dist .env + ${EDITOR} .env # set to your needs + docker compose up + +### Volumes and `compose.override.yaml` You may want to understand where data is stored: -The web application saves `*.csv` files in the `./cache/` directory, which happens to be a docker volume named `helio_cache` +The web application saves `*.csv` files in the `./cache/` directory, which happens to be a docker volume +named `helio_cache`. -The speasy library holds a cache dir located at `container:/root/.cache/speasy/Cache/` which is also mounted as a volume: `speasy_cache` +The speasy library holds a cache dir located at `container:/root/.cache/speasy/Cache/` which is also mounted as a +volume: `speasy_cache`. Two configuration files are also used: -- `./config.yml` for the web app -- `${HOME}/.config/speasy/config.ini` for the speasy library +- `./config.yml` : for the web app. +- `${HOME}/.config/speasy/config.ini` : for the speasy library. +You can use the `compose.override.yaml` file to make dynamic links between your local files or directories and the +container. -You can use the `compose.override.yaml` file to make dynamic links between your local files and the container. That way you can live edit the config files, and also use already loaded cache directories. +To do so: + cp compose.override.yaml-dist compose.override.yaml ${EDITOR} compose.override.yaml # make sure its ok, comment what you dont need docker compose up -d -## Develop +## Developer + +### Files Overview + +- `config.yml` : the main configuration file. +- `web/run.py` : the front controller, holding most of the code. +- `web/run.log` : you'll find more information about failures in the log. +- `web/view/home.html.jinja2` : the HTML template. +- `web/static/js/main.js` : most of the javascript client-side. ### Configuration -``` bash -cp config.yml-dist config.yml -$(EDITOR) config.yml # but defaults should be ok -``` + cp config.yml-dist config.yml + $(EDITOR) config.yml # but defaults should be ok + +Also take a look at the `./ressources/` directory for more available configuration files. ### Python venv -``` bash -python3 -m venv venv -source venv/bin/activate -pip install -r requirements.txt -``` + python3 -m venv venv + source venv/bin/activate + pip install -r requirements.txt ### Run locally -``` bash -source venv/bin/activate -DEBUG=true python web/run.py -``` + + source venv/bin/activate + DEBUG=true python web/run.py Then, browse [localhost:5000](http://localhost:5000). -## Gunicorn and Nginx deployment on OVH for development +### Admin task -see [ Digital Ocean documentation ] (https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-20-04-fr) +A tool called `hp_cmd.py` is available to make so administrative task. + +It allows you to nightly populate the csv cache so later visitors won't have to wait for data download. + +Basic usage: + + # look at what it would do + python web/hp_cmd.py --dry-run -clog csv_gen -a + + # do it + python web/hp_cmd.py -clog csv_gen -a + + # see all usages + python web/hp_cmd.py --help + + +Docker usage: + + docker compose exec web python web/hp_cmd.py --all + + +Http tasks: + http://${HOSTNAME}/cache/clear + http://${HOSTNAME}/cache/cleanup + http://${HOSTNAME}/cache/warmup + http://${HOSTNAME}/log + http://${HOSTNAME}/log/clear + +### Notebooks + +If your want to dig into code usage take a look at our notebooks: + + pip install -r requirements-nb.txt + jupyter notebook ./notebooks/ + +then browse to given url. + +Each file name is self-explanatory, and each notebook contains explanations. + + +## Gunicorn and Nginx deployment + +see [ Digital Ocean documentation ] (https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-20-04-fr) ### wep app service and Gunicorn @@ -114,19 +191,17 @@ see ./resources/heliopropa.nginx } } - ### Start and Allow at system reboot sudo systemctl start nginx sudo systemctl start heliopropa sudo systemctl enable heliopropa - ## Git hook auto deploy see ./resources/post-receive.git-hook -Set bare repo with hook on vps +Set bare repo with hook on vps ssh user@vps git clone --bare path-to-heliopropa-repo heliopropa.git @@ -134,8 +209,6 @@ Set bare repo with hook on vps vim post-receive chmod +x post-receive - - Push from dev computer git remote add helio-dev user@vps:heliopropa.git diff --git a/config.yml-dist b/config.yml-dist index 00968dd..c1853d4 100644 --- a/config.yml-dist +++ b/config.yml-dist @@ -912,6 +912,7 @@ targets: atse: 'psp_sta_da' locked: false default: false + # - type: 'probe' # slug: 'rosetta' # name: 'Rosetta' @@ -935,6 +936,7 @@ targets: # - slug: 'tao_ros_sw' # locked: true # default: false + # - type: 'probe' # slug: 'juno' # name: 'Juno' @@ -954,6 +956,7 @@ targets: # - slug: 'tao_juno_sw' # locked: true # default: false + # - type: 'comet' # slug: 'p67' # name: 'Churyumov-Gerasimenko' -- libgit2 0.21.2