Merge branch 'ApacheDeploy' into dev

hitier
2 parents d69e07cb 6a6d19ff
Showing 7 changed files with 402 additions and 253 deletions Show diff stats
CHANGELOG.md
README.md
TODO.md
heliopropa.wsgi
resources/apache.conf -> resources/apache-python2.conf
resources/apache-python3.conf
web/run.py
-## Nicolas
-
-- [ ] rajouter MAVEN, New Horizons, Cassini, Galileo, ExoMars comme spacecraft cible
-- [ ] ajouter un bouton ALERT qui permet d'ouvrir une nouvelle page de formulaire :
-    * email
-    * target (= planète, spacecraft, comet)
-    * submit (envoyé ensuite vers un webservice d'AMDA à construire)
-- [ ] prévoir un bouton EPN-TAP ou l'on mettrait l'outil en relation avec le client VESPA développé par nathanaël
-- [ ] séries temporelles en échelle log ou linéaire
-      /!\ D3 ne supporte pes les échelles log pour un domaine qui contient le 0.
-- [ ] juno/rosetta doivent aussi bouger le long de leur orbite lorsque l'on parcourt à la souris les données de la série temporelle; pour les intervalles de temps où rosetta  n'existe plus (crash à la surface de la comète) il faut qu'elle disparaisse; pour les intervalles de temps où juno est en orbite autour de jupiter il faut que juno suive jupiter
-      -> tout ceci est configurable dans le fichier de configuration, je n'ai pas les ids !
-
-- [ ] pour l'option jupiter thermospheric models dans l'onglet layer, une fois activée il faut que sur un tracé de la pression dynamique à jupiter tu fasses apparaître trois lignes horizontales en trois couleurs différentes pour les valeurs:
-      Model A compressed Pdyn=0.121  nPa
-      Model B baseline   Pdyn=0.060  nPa
-      Model C expanded   Pdyn=0.020  nPa
-
-
-## CME Catalogs
-
-1) https://www.helcats-fp7.eu/catalogues/wp4_icmecat.html
-et télécharger le catalogue en ASCII, JSON ou VOTAble XML
-les traits verticaux à repérer sur les données Terre sont délimités par ICME_START_TIME et ICME_STOP_TIME, et il faut prendre le catalogue pour WIND uniquement (qui se trouver au point L1)
-et prendre aussi
-2) https://wind.nasa.gov/fullcatalogue.php
-
-
-## Future ?
-
-- [ ] Rework the images of Rosetta and Juno
-- [ ] Optimize data aggregation (numpy vectorization?)
-- [ ] Bump D3JS to v5 (and its promises)
-- [ ] Enable p67
-- [ ] IE compat, if you can (mouahahahahaha good luck)
-- [ ] Fix the warmup on the prod server (I was kicked out, do it yourselves)
-- [ ] Set the log level to _error_ in production (see `web/run.py` -- DIY)
-
-
-## 1.9
-
-- [x] Auroral Emissions (Jupiter)
-- [x] Auroral Emissions Fixes
-- [x] Enable the Input selector
-
-
-## 1.8.3
-
-- [x] Fix whitespaces
-
-
-## 1.8.2
-
-- [x] Fix Earth temperature
-
-
-## 1.8
-
-- [x] Auroral Emissions (Saturn)
-
-
-## 1.7
-
-- [x] Only get SWRT after OMNI
-- [x] Fix Rosetta & Juno squatting the CDF Download
-- [x] Layer : Models (disabled)
-- [x] Layer : Campaigns (disabled)
-- [x] Layer : Catalogs (disabled)
-- [x] Layer : Inputs (disabled)
-- [x] Solve the ticks' month issue
-
-
-## 1.6
-
-- [x] Re-enable Earth
-- [x] Layer : CME Catalogs
-- [x] Add an About page
-- [x] Rewrite everything using ES6 instead of Livescript
-- [x] Force domain re-sizing of time series to be sequential
-- [x] Fix broken, confusing ticks on time series
-- [x] Make date format in time series saner (DOY)
-
-
-## 1.5
-
-- [x] Use the new AMDA API
-- [x] Add Mercury Real Time
-- [x] Add Venus Real Time
-
-
-## 1.4.1
-
-- [x] Add logs to hunt a Heisenbug
-
-
-## 1.4
-
-- [x] Enable Uranus
-- [x] Enable Neptune
-
-
-## 1.3.1
-
-- [x] Disable Earth again
-- [x] Fix the layers being shown outside the clipping area in the time series
-
-
-## 1.3
-
-- [x] Earth pdyn = pv² (TBD)
-- [x] Earth omnihour B is a Vector3, use the first scalar
-- [x] Layers
-    - SW Catalogs
-
-
-## 1.2
-
-- [x] Restore Earth
-- [x] Clean NetCDF, stage 1
-- [x] Accordion on the parameters list
-- [x] Check IE compat
-- [x] Bind the Alert button to a website
-
-
-## 1.1
-
-- [x] Credit the author of the pixel art planets (`Jr Astronaut`)
-- [x] … and buy his stuff from https://www.redbubble.com/people/spacefruitdox
-- [x] Fix a small glitch with the time series cursor on zoom
-
-
-## 1.0.1
-
-- [x] Fix an issue with the Time Series axes
-- [x] CRON statements to call the cache cleanup and warmup
-
-
-## 1.0.0
-
-- [x] Fix a bug with data that is offset by one hour.
-- [x] Use `git describe` as version number
-
-
-## 1.0.0-rc8
-
-- [x] Make the parameters' toogle buttons smaller
-- [x] Set time format to 24h in the time series
-- [x] Add the disabled versions of Uranus and Neptune
-- [x] Add the `CME Catalogs` layer
-
-
-## 1.0.0-rc7
-
-- [x] Add disabled layer buttons
-- [x] Give the future data another color
-- [x] Change the default interval (from 2 months in the past to one in the future)
-- [x] Prepare horizontal lines in the time series
-- [x] Add buttons to the orbits plot zoom to inner or outer heliosphere
-- [x] Make the Y-Axis of time series dynamic (at the expense of animations)
-
-
-## 1.0.0-rc6
-
-- [x] Re-lock Earth until we have better support
-- [x] Add support for inputs STEREO-A, STEREO-B, L1
-- [x] More ticks along X-axis (I tried, but it does not behave as expected)
-- [x] Smaller footer credits
-- [x] Move the version to the footer
-- [x] Fix various issues with spacepy and CDF generation
-
-
-## 1.0.0-rc5
-
-- [x] Cache warmup (generate for today's default interval) `/cache/warmup`
-- [x] Improve the generated CDF for SAMP
-- [x] Enable Earth
-- [x] Enable more precise configuration of each models' parameters
-
-
-## 1.0.0-rc4
-
-- [x] Sort times series by closeness to the sun
-- [x] Generate a CDF file (not NetCDF)
-- [x] Normalize time interval for time series
-- [x] Make the footer images clickable
-- [x] Highlight the visits counter
-- [x] Use a default interval up to 3 weeks in the future
-- [x] Add interval constraints for orbit models (Rosetta uses P67 after a time)
-- [x] Make the download with a netcdf file instead of a tarball of CSVs
-- [x] Support having no position to display (for Rosetta in some intervals)
-- [x] Make the local cache more resilient to corrupted downloads
-- [x] Make the local cache more resilient to naming collisions
-
-
-## 1.0.0-rc3
-
-- [x] Make the targets dynamic in the orbit plot, allowing zoom
-- [x] Refactor some more to move as much as we can to the config
-- [x] Cache clear (remove all files) `/cache/clear`
-
-
-## 1.0.0-rc2
-
-- [x] Refactor and clean up
-- [x] Add loaders to targets' toggle button
-
-
-## 1.0.0-rc1
-
-- [x] Support multiple data and orbit models for each target
-- [x] Cache cleanup (remove old files) `/cache/cleanup`
-
-
-## 0.1.0
-
-- [x] Cache generated CSVs on the server side
-- [x] Cache generated CSVs on the client side
-- [x] Count visits
-- [x] Zoom in on time series
-- [x] Loader and targets loading animation
-- [x] Start/Stop datetime fields
-- [x] Retry CSV generation when it fails due to a bug in AMDA's API
-- [x] Remove duplicate NetCDFs from AMDA's API response
-- [x] Add a version number somewhere
-- [x] Move the visitors counter to the footer
-- [x] Download raw data (tarball of CSV) for current time interval and targets
-- [x] Download raw data as NetCDF for current everything, via SAMP
-- [x] Add a warning for users with javascript disabled
-
-
-## 0.0.0
-
-- [x] Initial website skeleton
-- [x] Initial data bridge between netCDF and CSV
-- [x] New parameters
-    - [x] Temperature `T` (K)
-    - [x] Density `N` (cm^-3)
-    - [x] Angle Planet-Sun-Earth `Delta_angle` (degrees)
-    - [x] Magnetism => B Tangential
-- [x] Planets
-- [x] Probes
-    - [x] Rosetta
-    - [x] Juno
-- [x] Comets (the one visited by rosetta, p67)
-- [x] Orbits axes titles
-- [x] Logo Europlanet
-- [x] Cache remote NetCDFs locally
-- [x] Add source name on time series
-- [x] Support multiple sources
-- [x] Support enabling/disabling sources
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
+
+Version numbering will follow those principles:
+- MAJOR: for big changes, like inner architecture
+- MINOR: for new functionnalities, like adding hardware and corresponding servers and clients.
+- PATCH: when fixing bugs or adding very small details for previous MINOR functionnality.
+
+`Unreleased` version holds ongoing changes.
+
+Changes are kept under subsections:
+- `New` for new features.
+- `Changed` for changes in existing functionality.
+- `Deprecated` for soon-to-be removed features.
+- `Removed` for now removed features.
+- `Fixed` for any bug fixes.
+- `Security` in case of vulnerabilities.
+
+Should be noted only functionnal changes,
+or major refactoring improvments.
+
+## [2.0-alpha] - On Going
+### Fixed
+### Changed
+### New
@@ -37,8 +37,9 @@ https://gitlab.irap.omp.eu/CDPP/SPACEWEATHERONLINE
 ### Prerequisites
 gcc
 libncurses-dev
-python2
-virtualenv
+python3
+python3-virtualenv
+libapache2-mod-wsgi-py3
 ### Python venv
@@ -0,0 +1,250 @@
+## Nicolas
+
+- [ ] rajouter MAVEN, New Horizons, Cassini, Galileo, ExoMars comme spacecraft cible
+- [ ] ajouter un bouton ALERT qui permet d'ouvrir une nouvelle page de formulaire :
+    * email
+    * target (= planète, spacecraft, comet)
+    * submit (envoyé ensuite vers un webservice d'AMDA à construire)
+- [ ] prévoir un bouton EPN-TAP ou l'on mettrait l'outil en relation avec le client VESPA développé par nathanaël
+- [ ] séries temporelles en échelle log ou linéaire
+      /!\ D3 ne supporte pes les échelles log pour un domaine qui contient le 0.
+- [ ] juno/rosetta doivent aussi bouger le long de leur orbite lorsque l'on parcourt à la souris les données de la série temporelle; pour les intervalles de temps où rosetta  n'existe plus (crash à la surface de la comète) il faut qu'elle disparaisse; pour les intervalles de temps où juno est en orbite autour de jupiter il faut que juno suive jupiter
+      -> tout ceci est configurable dans le fichier de configuration, je n'ai pas les ids !
+
+- [ ] pour l'option jupiter thermospheric models dans l'onglet layer, une fois activée il faut que sur un tracé de la pression dynamique à jupiter tu fasses apparaître trois lignes horizontales en trois couleurs différentes pour les valeurs:
+      Model A compressed Pdyn=0.121  nPa
+      Model B baseline   Pdyn=0.060  nPa
+      Model C expanded   Pdyn=0.020  nPa
+
+
+## CME Catalogs
+
+1) https://www.helcats-fp7.eu/catalogues/wp4_icmecat.html
+et télécharger le catalogue en ASCII, JSON ou VOTAble XML
+les traits verticaux à repérer sur les données Terre sont délimités par ICME_START_TIME et ICME_STOP_TIME, et il faut prendre le catalogue pour WIND uniquement (qui se trouver au point L1)
+et prendre aussi
+2) https://wind.nasa.gov/fullcatalogue.php
+
+
+## Future ?
+
+- [ ] Rework the images of Rosetta and Juno
+- [ ] Optimize data aggregation (numpy vectorization?)
+- [ ] Bump D3JS to v5 (and its promises)
+- [ ] Enable p67
+- [ ] IE compat, if you can (mouahahahahaha good luck)
+- [ ] Fix the warmup on the prod server (I was kicked out, do it yourselves)
+- [ ] Set the log level to _error_ in production (see `web/run.py` -- DIY)
+
+
+## 1.9
+
+- [x] Auroral Emissions (Jupiter)
+- [x] Auroral Emissions Fixes
+- [x] Enable the Input selector
+
+
+## 1.8.3
+
+- [x] Fix whitespaces
+
+
+## 1.8.2
+
+- [x] Fix Earth temperature
+
+
+## 1.8
+
+- [x] Auroral Emissions (Saturn)
+
+
+## 1.7
+
+- [x] Only get SWRT after OMNI
+- [x] Fix Rosetta & Juno squatting the CDF Download
+- [x] Layer : Models (disabled)
+- [x] Layer : Campaigns (disabled)
+- [x] Layer : Catalogs (disabled)
+- [x] Layer : Inputs (disabled)
+- [x] Solve the ticks' month issue
+
+
+## 1.6
+
+- [x] Re-enable Earth
+- [x] Layer : CME Catalogs
+- [x] Add an About page
+- [x] Rewrite everything using ES6 instead of Livescript
+- [x] Force domain re-sizing of time series to be sequential
+- [x] Fix broken, confusing ticks on time series
+- [x] Make date format in time series saner (DOY)
+
+
+## 1.5
+
+- [x] Use the new AMDA API
+- [x] Add Mercury Real Time
+- [x] Add Venus Real Time
+
+
+## 1.4.1
+
+- [x] Add logs to hunt a Heisenbug
+
+
+## 1.4
+
+- [x] Enable Uranus
+- [x] Enable Neptune
+
+
+## 1.3.1
+
+- [x] Disable Earth again
+- [x] Fix the layers being shown outside the clipping area in the time series
+
+
+## 1.3
+
+- [x] Earth pdyn = pv² (TBD)
+- [x] Earth omnihour B is a Vector3, use the first scalar
+- [x] Layers
+    - SW Catalogs
+
+
+## 1.2
+
+- [x] Restore Earth
+- [x] Clean NetCDF, stage 1
+- [x] Accordion on the parameters list
+- [x] Check IE compat
+- [x] Bind the Alert button to a website
+
+
+## 1.1
+
+- [x] Credit the author of the pixel art planets (`Jr Astronaut`)
+- [x] … and buy his stuff from https://www.redbubble.com/people/spacefruitdox
+- [x] Fix a small glitch with the time series cursor on zoom
+
+
+## 1.0.1
+
+- [x] Fix an issue with the Time Series axes
+- [x] CRON statements to call the cache cleanup and warmup
+
+
+## 1.0.0
+
+- [x] Fix a bug with data that is offset by one hour.
+- [x] Use `git describe` as version number
+
+
+## 1.0.0-rc8
+
+- [x] Make the parameters' toogle buttons smaller
+- [x] Set time format to 24h in the time series
+- [x] Add the disabled versions of Uranus and Neptune
+- [x] Add the `CME Catalogs` layer
+
+
+## 1.0.0-rc7
+
+- [x] Add disabled layer buttons
+- [x] Give the future data another color
+- [x] Change the default interval (from 2 months in the past to one in the future)
+- [x] Prepare horizontal lines in the time series
+- [x] Add buttons to the orbits plot zoom to inner or outer heliosphere
+- [x] Make the Y-Axis of time series dynamic (at the expense of animations)
+
+
+## 1.0.0-rc6
+
+- [x] Re-lock Earth until we have better support
+- [x] Add support for inputs STEREO-A, STEREO-B, L1
+- [x] More ticks along X-axis (I tried, but it does not behave as expected)
+- [x] Smaller footer credits
+- [x] Move the version to the footer
+- [x] Fix various issues with spacepy and CDF generation
+
+
+## 1.0.0-rc5
+
+- [x] Cache warmup (generate for today's default interval) `/cache/warmup`
+- [x] Improve the generated CDF for SAMP
+- [x] Enable Earth
+- [x] Enable more precise configuration of each models' parameters
+
+
+## 1.0.0-rc4
+
+- [x] Sort times series by closeness to the sun
+- [x] Generate a CDF file (not NetCDF)
+- [x] Normalize time interval for time series
+- [x] Make the footer images clickable
+- [x] Highlight the visits counter
+- [x] Use a default interval up to 3 weeks in the future
+- [x] Add interval constraints for orbit models (Rosetta uses P67 after a time)
+- [x] Make the download with a netcdf file instead of a tarball of CSVs
+- [x] Support having no position to display (for Rosetta in some intervals)
+- [x] Make the local cache more resilient to corrupted downloads
+- [x] Make the local cache more resilient to naming collisions
+
+
+## 1.0.0-rc3
+
+- [x] Make the targets dynamic in the orbit plot, allowing zoom
+- [x] Refactor some more to move as much as we can to the config
+- [x] Cache clear (remove all files) `/cache/clear`
+
+
+## 1.0.0-rc2
+
+- [x] Refactor and clean up
+- [x] Add loaders to targets' toggle button
+
+
+## 1.0.0-rc1
+
+- [x] Support multiple data and orbit models for each target
+- [x] Cache cleanup (remove old files) `/cache/cleanup`
+
+
+## 0.1.0
+
+- [x] Cache generated CSVs on the server side
+- [x] Cache generated CSVs on the client side
+- [x] Count visits
+- [x] Zoom in on time series
+- [x] Loader and targets loading animation
+- [x] Start/Stop datetime fields
+- [x] Retry CSV generation when it fails due to a bug in AMDA's API
+- [x] Remove duplicate NetCDFs from AMDA's API response
+- [x] Add a version number somewhere
+- [x] Move the visitors counter to the footer
+- [x] Download raw data (tarball of CSV) for current time interval and targets
+- [x] Download raw data as NetCDF for current everything, via SAMP
+- [x] Add a warning for users with javascript disabled
+
+
+## 0.0.0
+
+- [x] Initial website skeleton
+- [x] Initial data bridge between netCDF and CSV
+- [x] New parameters
+    - [x] Temperature `T` (K)
+    - [x] Density `N` (cm^-3)
+    - [x] Angle Planet-Sun-Earth `Delta_angle` (degrees)
+    - [x] Magnetism => B Tangential
+- [x] Planets
+- [x] Probes
+    - [x] Rosetta
+    - [x] Juno
+- [x] Comets (the one visited by rosetta, p67)
+- [x] Orbits axes titles
+- [x] Logo Europlanet
+- [x] Cache remote NetCDFs locally
+- [x] Add source name on time series
+- [x] Support multiple sources
+- [x] Support enabling/disabling sources
@@ -0,0 +1,42 @@
+#
+# Simple wsgi wrapper to run flask app through apache.
+#
+# Relies on an apache2 virtualhost configuration.
+# See ./resources/apache2-virtual-host.conf
+#
+# It merely calls the app factory and exports it as
+# the application variable.
+
+# The apache WSGIDaemonProcess allows us to set 
+#  a virtualenv
+#  a pythonpath
+
+# Instead we could:
+#
+##  1- activate venv
+# python_home = '/var/www/html/app-web/'
+# activate_this = python_home + 'venv/bin/activate_this.py'
+# with open(activate_this) as file_:
+#     exec(file_.read(), dict(__file__=activate_this))
+#
+##  2- set a python path
+# sys.path.insert(0, '/var/www/html/app-web/')
+#
+##  3- set some logging facilities
+# import logging, sys
+# logging.basicConfig(stream=sys.stderr)
+
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 
+
+#
+# Main Part
+#
+
+# simply use application already set in main app file
+#
+from web.run import app as application
+
+
+
+
+# vim: ft=python
@@ -0,0 +1,76 @@
+# Apache virtual host conf file for Heliopropa web-app
+#
+# Will map a server name and an url to the wsgi script.
+# 
+# Allows to use a python virtualenvironment thanks to two WSGIDaemonProcess
+# attributes:
+#     python-home
+#     python-path
+
+
+# Edit Configuration
+#
+Define flaskapp_path  /home/richard/00DEV/SPACEWEATHERONLINE
+Define flaskapp_wsgiscript  heliopropa.wsgi
+Define flaskapp_server_name heliopropa
+Define flaskapp_server_addr 127.0.0.1
+Define flaskapp_subdir /
+Define flaskapp_user www-data
+Define flaskapp_group www-data
+
+# Warning:
+# --------
+#
+# Stdout/Stderr wont be logged in per virtualhost log files.
+#
+# 1- Unsuccessfull tries to fix it:
+# 
+# WSGIRestrictEmbedded On
+# WSGIRestrictStdout Off
+#
+# 2- what works is to comment out ErrorLog, TransferLog and CustomLog
+#    and let apache log everything in global log files
+#
+# 3- However you should prefer to manage your own log system from inside web-app
+#
+#
+<VirtualHost ${flaskapp_server_addr}:443>
+     # Add machine's IP address (use ifconfig command)
+     ServerName ${flaskapp_server_name}
+     # https stuf
+     #SSLEngine on
+     #SSLProtocol all -SSLv2
+     #SSLCertificateFile /etc/pki/tls/certs/localhost.crt
+     #SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
+     # Give an alias to to start your website url with
+     DocumentRoot ${flaskapp_path}
+     LogLevel warn
+     LogFormat "%a %l %u %t \"%r\" %>s %b"
+     #
+     # Virtualhost log config doesnt show stdout/err see before
+     # ErrorLog  ${flaskapp_path}/flaskapp-error.log
+     # TransferLog ${flaskapp_path}/flaskapp-access.log
+     # CustomLog  ${flaskapp_path}/flaskapp-custom.log combined
+
+     # python-home is the virtual env path
+     # python-path sets the PYTHON_PATH for modules import
+     WSGIDaemonProcess ${flaskapp_server_name} \
+                        user=${flaskapp_user} group=${flaskapp_user} \
+                        processes=2 threads=5 \
+                        python-home=${flaskapp_path}/venv \
+                        python-path=${flaskapp_path} \
+                        display-name=%{GROUP}
+     WSGIScriptAlias /${flaskapp_subdir} ${flaskapp_path}/${flaskapp_wsgiscript}
+     WSGIProcessGroup ${flaskapp_server_name}
+     WSGIApplicationGroup %{GLOBAL}
+     <Directory  ${flaskapp_path}>
+            # set permissions as per apache2.conf file
+            Options -Indexes -MultiViews +FollowSymLinks
+            AllowOverride All
+            Require all granted
+     </Directory>
+</VirtualHost>
+
+
+
+# vim: tabstop=4 sw=4 et tw=0 
@@ -53,7 +53,9 @@ with open(get_path(&#39;../VERSION&#39;), &#39;r&#39;) as version_file:
     version = version_file.read().strip()
 # CONFIG
-with open(get_path('../config.yml'), 'r') as config_file:
+
+
+with open(get_path('../config.yml'), 'r', encoding='utf8') as config_file:
     config = yaml_load(config_file.read(), Loader=Loader)
 FILE_DATE_FMT = "%Y-%m-%dT%H:%M:%S"