README.md
http://heliopropa.irap.omp.eu
What
Web Server
Build and serve heliopropa's visualizations.
It's a flask webserver, serving d3.js plots.
It also gathers NetCDF data from AMDA, and serves it as CSV to the plotter.
Rationale
- Reading NetCDF from javascript is doable, but still very hacky.
- The bridge can handle pagination and collecting multiple NetCDF into one CSV.
How
Source
https://gitlab.irap.omp.eu/CDPP/SPACEWEATHERONLINE
Overview
config.yml: the main configuration file.web/run.py: the front controller, holding most of the code.web/run.log: you'll find more information about failures in the log.web/view/home.html.jinja2: the HTML template.web/static/js/swapp.ls: most of the javascript client-side.
Install
Prerequisites
gcc libncurses-dev python3 python3-virtualenv libapache2-mod-wsgi-py3
Python venv
python3 -m venv venv
source venv/bin/activate
pip install matplotlib
pip install numpy==1.8.*
pip install -r requirements.txt
CentOs 6 Shenanigans
yum install https://centos6.iuscommunity.org/ius-release.rpm
yum install python27*
But then virtualenv yields
File "/var/www/html/HELIOPROPA/web/run.py", line 24, in <module>
[Fri Nov 30 14:30:38 2018] [error] [client 10.10.135.119] from netCDF4 import Dataset, date2num
[Fri Nov 30 14:30:38 2018] [error] [client 10.10.135.119] File "/var/www/html/HELIOPROPA/venv/lib/python2.7/site-packages/netCDF4/__init__.py", line 3, in <module>
[Fri Nov 30 14:30:38 2018] [error] [client 10.10.135.119] from ._netCDF4 import *
[Fri Nov 30 14:30:38 2018] [error] [client 10.10.135.119] ImportError: /var/www/html/HELIOPROPA/venv/lib/python2.7/site-packages/netCDF4/_netCDF4.so: undefined symbol: PyUnicodeUCS2_FromStringAndSize
Solution:
Configure Apache to use web/run_ssa.wsgi and use pip2.7 install system wide.
Install libcdf.
wget https://spdf.gsfc.nasa.gov/pub/software/cdf/dist/cdf38_1/cdf38_1-dist-all.tar.gz
tar -xzvf cdf38_1-dist-all.tar.gz
cd cdf38_1-dist
make all OS=linux ENV=gnu
make INSTALLDIR=/usr/local/lib/libcdf install
Develop
source venv/bin/activate
DEBUG=true python web/run.py
Then, browse localhost:5000.
Troubleshoot
Installing scipy
Getting MemoryErrors ?
Try with the --no-cache-dir option.
pip install scipy --no-cache-dir
Fix permissions problems with the cache
sudo setfacl -R -m u:apache:rwx -m u:`whoami`:rwx cache
sudo setfacl -dR -m u:apache:rwx -m u:`whoami`:rwx cache
Gunicorn and Nginx deployment on OVH for development
see Digital Ocean documentation
wep app service and Gunicorn
see ./resources/heliopropa.service
/etc/systemd/system/heliopropa.service
[Unit]
Description=Gunicorn instance to serve Heliopropa
After=network.target
[Service]
User=debian
Group=www-data
WorkingDirectory= /home/debian/heliopropa/
Environment="PATH=/home/debian/heliopropa/venv/bin/"
ExecStart=/home/debian/heliopropa/venv/bin/gunicorn --workers 3 --bind unix:heliopropa.sock -m 007 web.run:app
[Install]
WantedBy=multi-user.target
Nginx
see ./resources/heliopropa.nginx
/etc/nginx/sites-available/heliopropa
server {
listen 80;
server_name heliopropa heliopropa.co-libri.org;
location / {
include proxy_params;
proxy_pass http://unix:/home/debian/heliopropa/heliopropa.sock;
}
}
Start and Allow at system reboot
sudo systemctl start nginx
sudo systemctl start heliopropa
sudo systemctl enable heliopropa
Git hook auto deploy
see ./resources/post-receive.git-hook
Set bare repo with hook on vps
ssh user@vps
git clone --bare path-to-heliopropa-repo heliopropa.git
cd heliopropa.git/hooks
vim post-receive
chmod +x post-receive
Push from dev computer
git remote add helio-dev user@vps:heliopropa.git
git push heliopropa HEAD:DEV