From daa744d7c99e4e019a27dc166db9ae894ada7992 Mon Sep 17 00:00:00 2001 From: John Cardinal Date: Fri, 7 Dec 2018 19:26:20 +0000 Subject: [PATCH] --- devdocs/deploy.txt | 257 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100644 devdocs/deploy.txt diff --git a/devdocs/deploy.txt b/devdocs/deploy.txt new file mode 100644 index 00000000..8aa298cb --- /dev/null +++ b/devdocs/deploy.txt @@ -0,0 +1,257 @@ + +## DEPLOYMENT + +### DEPLOY TO DIGITAL OCEAN TEST SERVER + +PUBLISH TO DEVOPS STEPS + +BUILD CLIENT + - Clean out the last build in the server wwwRoot folder at: C:\data\code\raven\server\AyaNova\wwwroot + - Delete all but the \docs subfolder that contains the AyaNova manual unless planning on rebuilding that as well + - Go to client folder: C:\data\code\raven\app\ayanova + - Run npm run build + - Copy build from C:\data\code\raven\app\ayanova\dist to server wwwRoot folder: C:\data\code\raven\server\AyaNova\wwwroot + +BUILD SERVER + - Make sure docs are built, use makedocs batch file which will build and put in wwwRoot folder + - Make sure updated version number first!! + - Need to be in C:\data\code\raven\server\AyaNova\ + - Then run command: + - dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release + +- COPY TO SERVER + - Use filezilla to copy files that are new up to server + - Copy to "/home/john/xfer/ayanovadocker/files" + - These two files (and any other changes that are relevant) + - C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll + - C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb + +- CONSOLE TO SERVER VIA PUTTY + - Bring down current containers: + - navigate to ~/xfer folder + - execute sudo docker-compose down + + - Build new image forcing it to update as it sometimes doesn't + - sudo docker-compose build --force-rm --pull + + - Run new image + - sudo docker-compose up -d + + - Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error) + - use the restartnginx.sh script in xfer at the server + - or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d + + - Test + - If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it. + - Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck + - Check logs with sudo docker logs [containerID] to find out what happened + - Or in some cases (once) Digital Ocean fucked up something + +- ERASE DB, FETCH LICENSE, GENERATE DATA + - ERASE DB: + - Stop container if not already stopped: execute sudo docker-compose down + - Edit docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db + - sudo docker-compose up -d + - Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable + - Start the container again with the up command + - FETCH TEST KEY: + - Go into the api explorer, authenticate then + - select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"} + - This seems to setup the db to accept a trial key when fetching the regular key next + - select the POST to license route (not the TRIAL one), this will fetch a test key and install it + - SEED DB: + - Go to trial route and pick seed level (HUGE for proper testing) and activate + - NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset + + +### Publish command line: + +Windows 64 bit: +dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained +dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained + + +Linux 64 bit: + +Normal build without all the .net files (not self contained) +This is appropriate for docker based distribution since another image will contain the .net runtime: + +#### DEFAULT BUILD COMMAND +dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release + +(linux) +dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release + + +Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below): +dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained +dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained + +Needed to change permissions on the AyaNova file to make it executable and also it requires these pre-requisites and probably more: +apt-get install libunwind8 +apt-get install libcurl3 + +//.net core 2.x linux native requirements +https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x + + + +Windows 32 bit: +dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained + +Self contained Windows 10 x64: +dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained + +PORTABLE RID's: +win-x64 +win-x86 +linux-x64 + +//D.O. Linux +ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary + + +- https://docs.microsoft.com/en-us/dotnet/core/deploying/index +- https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x +- https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x +- https://docs.microsoft.com/en-us/dotnet/core/rid-catalog + + +### DOCKER + + - Build containers: + - john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build + - Run it: + - :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d + - Build it in prep for running it: + - dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release + - john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release + + + - OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info) + - Save image: + - docker image save -o .\image\ay-alpha2 gztw/ayanova + - Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags + - Not compressed, can be compressed about 60% smaller + - Load image: + - docker image load -i saved_image_file_name_here + + +#### +- Running docker at our D.O. server + - run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/ + - To update: + - run a publish command to publish to my local dist/linux-x64/ayanovadocker/files + - Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files + - Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future) + - If necessary do a docker-compose build to rebuild + - run Nginx server: + - from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d + - If necessary can switch to root with command: sudo su - + - documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx + + + +### DOCKER NGINX LETS ENCRYPT CERTBOT + - https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx + - https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf + + + +INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS) + +#### STAGING +sudo docker run -it --rm \ +-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ +-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ +-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ +-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ +certbot/certbot \ +certonly --webroot \ +--email support@ayanova.com --agree-tos --no-eff-email \ +--webroot-path=/data/letsencrypt \ +--staging \ +-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com + +#### PRODUCTION +sudo docker run -it --rm \ +-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ +-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ +-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ +-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ +certbot/certbot \ +certonly --webroot \ +--email support@ayanova.com --agree-tos --no-eff-email \ +--webroot-path=/data/letsencrypt \ +-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com + + +#### SAMPLE OUTPUT: +john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \ +> -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ +> -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ +> -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ +> -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ +> certbot/certbot \ +> certonly --webroot \ +> --email support@ayanova.com --agree-tos --no-eff-email \ +> --webroot-path=/data/letsencrypt \ +> -d helloayanova.com -d www.helloayanova.com +Saving debug log to /var/log/letsencrypt/letsencrypt.log +Plugins selected: Authenticator webroot, Installer None +Obtaining a new certificate +Performing the following challenges: +http-01 challenge for helloayanova.com +http-01 challenge for www.helloayanova.com +Using the webroot path /data/letsencrypt for all unmatched domains. +Waiting for verification... +Cleaning up challenges + +IMPORTANT NOTES: + - Congratulations! Your certificate and chain have been saved at: + /etc/letsencrypt/live/helloayanova.com/fullchain.pem + Your key file has been saved at: + /etc/letsencrypt/live/helloayanova.com/privkey.pem + Your cert will expire on 2018-06-10. To obtain a new or tweaked + version of this certificate in the future, simply run certbot + again. To non-interactively renew *all* of your certificates, run + "certbot renew" + - Your account credentials have been saved in your Certbot + configuration directory at /etc/letsencrypt. You should make a + secure backup of this folder now. This configuration directory will + also contain certificates and private keys obtained by Certbot so + making regular backups of this folder is ideal. + - If you like Certbot, please consider supporting our work by: + + Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate + Donating to EFF: https://eff.org/donate-le + + +=-=-=-=-=-=-=-=- + + +GRAFANA / INFLUXDB / DOCKER + +Container to run the whole shebang: + +- https://github.com/philhawthorne/docker-influxdb-grafana +docker run -d \ + --name docker-influxdb-grafana \ + -p 3003:3003 \ + -p 3004:8083 \ + -p 8086:8086 \ + -p 22022:22 \ + -v /path/for/influxdb:/var/lib/influxdb \ + -v /path/for/grafana:/var/lib/grafana \ + philhawthorne/docker-influxdb-grafana:latest + + NOTE: you can leave out the paths and it works and the name is a little verbose + + Dashboard for Grafana and app.metrics: + - https://grafana.com/dashboards/2125 + + + MSBUILD reference for csproj file +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= +https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild#BKMK_ProjectFile +