admin/raven
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2c509dcbe3d4df7fbae1f6301e76787a9678acbd
raven/devdocs/deploy.md

476 lines
21 KiB
Markdown
Raw Blame History

# BUIILD AND DEPLOYMENT
### Security scan
If any packages have been changed in the release do a thorough security scan and check of each package
### Bump version numbers:
Search and replace 8.0.5
webapp,server,launcher, v8migrate (don't change v8migrate unless it has it's own code changes, it's version should be it's own thing other than major release changes etc)
Client end ayanova-version.js,
Server end ayanova-version.cs AND AyaNova.csproj
The TWO windows installers are versioned (LAN.ISS and STANDALONE.ISS)
RavenLauncher "single" Launcher is also versioned and needs to be changed individually
Docs are not versioned but dated in mkdocs.yml copyright property so if that needs to change be sure it's udpated
#MyAppVersion in C:\data\code\raven\dist\install\windows\x64\lan.iss
#MyAppVersion in C:\data\code\raven\dist\install\windows\x64\standalone.iss
### run build-release.bat
check for errors
## UPLOAD INSTALLERS TO NEXT FOLDER
UPLOAD releases to ayanova.com/download/next folder where it will be staged before officially being released and moved to parent download folder
### Test install to devops server
#### NEXT folder version
cd /var/ayanova && \
sudo systemctl stop ayanova.service && \
sudo curl -O https://www.ayanova.com/download/next/ayanova-linux-x64-server.zip && \
sudo rm *.dll && \
sudo rm -r ./resource && \
sudo rm -r ./wwwroot && \
sudo unzip -o ayanova-linux-x64-server.zip -x "config.json" && \
sudo systemctl start ayanova.service
(a261 server)
cd /var/ayanovaa261 && \
sudo systemctl stop ayanovaa261.service && \
sudo curl -O https://www.ayanova.com/download/next/ayanova-linux-x64-server.zip && \
sudo rm *.dll && \
sudo rm -r ./resource && \
sudo rm -r ./wwwroot && \
sudo unzip -o ayanova-linux-x64-server.zip -x "config.json" && \
sudo systemctl start ayanovaa261.service
#### MAIN RELEASE folder version
cd /var/ayanova && \
sudo systemctl stop ayanova.service && \
sudo curl -O https://www.ayanova.com/download/ayanova-linux-x64-server.zip && \
sudo rm *.dll && \
sudo rm -r ./resource && \
sudo rm -r ./wwwroot && \
sudo unzip -o ayanova-linux-x64-server.zip -x "config.json" && \
sudo systemctl start ayanova.service
(a261 server)
cd /var/ayanovaa261 && \
sudo systemctl stop ayanovaa261.service && \
sudo curl -O https://www.ayanova.com/download/ayanova-linux-x64-server.zip && \
sudo rm *.dll && \
sudo rm -r ./resource && \
sudo rm -r ./wwwroot && \
sudo unzip -o ayanova-linux-x64-server.zip -x "config.json" && \
sudo systemctl start ayanovaa261.service
### Test install to windows server
### Run e2e tests against Linux and Windows servers
First adjust to the server desired by editing cypress.json and commenting in or out apiBaseUrl and BaseUrl props for local or devops or wherever
C:\data\code\raven-test\e2e>npm run all-chrome
(chrome most reliable but _will_ fail intermittently so needs multiple runs often)
Once tests pass move on to next step
### Run load tests against Linux and Windows servers
(determine if any changes triggered performance issues, needs baseline comparison value)
LOAD TESTING STEPS:
Login to server to test and seed fresh data first, then can run this test
run C:\data\code\raven-test\load\runtest.bat
Test will take approximately 14 minutes to run
Check results in results folder by opening index.html from windows explorer and view in browser
BASELINE WHAT TO CHECK:
DASHBOARD PAGE is the main page with the main stats:
"REquests summary"
The most telling overall stat is the "Requests summary" pie chart pass/fail ratio on the main page
As of today 2021-12-21 release 8-beta.06 the results are PASS: 92.87%, FAIL: 7.13%
Errors _are_ expected, it's a stress / load test but there shouldn't be an entirely failing route, just a scattered set of failures,
typically unauthorized or bad gateway
"Appdex" - skipping this for now, not really relevant at the moment
"STATISTICS" table - this is the important one as it shows failures
- Scroll down and look in "executions" -> "FAIL" column and make sure they are not all in one line item
if see a lot of failures in a single route need to manually look into that exact issue and route and examine the jmeter test for that route / run it manually perhaps
it's normal to see unauthorized or bad gateway errors when the server is super stressed out but other failures might need examination or all of it if less than 90% success
NOTES:
- this also catches reporting as it generates some reports
report-render test has explicit report id number for workorder, currently it's 15 for the dispatch report, if all fail it may have changed the id number
would be better deterministically but for now this works.
- To test local or devops edit the project file "C:\data\code\raven-test\load\stressdevops.jmx"
There are two configuration sections near top for local or devops and it's switched by setting either enabled or disabled:
<ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="Devops request defaults" enabled="true">
or
<ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="Localhost request defaults" enabled="false">
- Load test needs fresh seeding so it can create test users.
Once tests pass within expected parameters move on to next step
## MOVE PRIOR VERSION INSTALLER TO "VAULT"
### PREVIOUS FOLDER
Clean out the previous previous download from the previous folder
Copy the existing downloads to the previous folder preserving (-p) the timestamps
Change directory to the previous folder
From the previous folder: cp ../\* . -p
(note: will not copy folders so ok to use)
### OLD FOLDER
NOTE: Every posted release MUST go into this folder release then it should go in the "old" folder in a versioned sub-folder
This is really important because users are eligible to download the final old release of their support and updates level so we will probably need to give out the urls to download old versions to people that didn't keep up to date
### DELETE CURRENT DOWNLOADS (or ignore and overwrite maybe easier and safer anyway)
(OR JUST OVERWRITE THEM IS EASIER)
root@ubuntu-s-1vcpu-1gb-amd-sfo3-01:/var/www/html/ayanova.com/download# rm \*
(note: will not remove folders just files by default so ok to use)
### COPY INSTALLERS FROM NEXT FOLDER the 4 installer packages found in `ayanova.com/download/next`
- Follow path and folder and url rules in case 4053 https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/4053
- And also v8-migrate.exe if applicable
- Clean out next folder so there's no confusion
### Upload static copy of docs to our website
clean out current one from web server: (DONT USE FILEZILLA TO DELETE IT TAKES FUCKING FOREVER)
root@ubuntu-s-1vcpu-1gb-amd-sfo3-01:/var/www/html/ayanova.com/docs# rm \* -r
(WATCH THE PATH!!)
Upload docs:
from here: C:\data\code\raven\server\AyaNova\wwwroot\docs\
to here: /var/www/html/ayanova.com/docs
### Update website changelog and release notes etc
### profit$$
# LINUX MANUAL UPGRADE PROCEDURE
From /var/ayanova folder
Download new version (the -O will overwrite so no need to delete existing one):
sudo curl -O https://www.ayanova.com/download/ayanova-linux-x64-server.zip
Run this one-liner to update and restart:
sudo systemctl stop ayanova.service && rm \*.dll && rm -r ./resource && rm -r ./wwwroot && unzip -o ayanova-linux-x64-server.zip -x "config.json" && sudo systemctl start ayanova.service
OLD step by step instructions:
sudo systemctl stop ayanova.service
rm \*.dll
rm -r ./resource
rm -r ./wwwroot
unzip -o ayanova-linux-x64-server.zip -x "config.json"
sudo systemctl start ayanova.service
# ==========================================================================================================================
==========================================================================================================================
========================================= OLD OUTDATED INFO BEYOND HERE ==================================================
==========================================================================================================================
==========================================================================================================================
==========================================================================================================================
==========================================================================================================================
PUBLIC / INSTALLER
- bump versions, make sure client main.js devmode is false (probably should keep it that way for ongoing use and only enable when dev issue)
set version in Server, client and installer iss files
- run buildrelease.bat in client project folder
- run buildrelease.bat in server project folder
- Test deploy here first
- Post to downloads on ayanova.com
### DEPLOY TO DIGITAL OCEAN TEST SERVER
PUBLISH TO DEVOPS STEPS
FIRST
=-=-=-
**\*** BEFORE RELEASE DOUBLE CHECK CAN RE-SEED FROM CLIENT UI AS THERE IS OFTEN ISSUES WITH DBUTIL ERASE TABLES AND THE ORDERING ETC **\*\***
CLIENT
=-=-=-=
- TURN OFF DEV MODE IN MAIN.JS
- BUMP PACKAGE.JSON version number
- BUMP CLIENT AyaNova-version.js VERSION NUMBER
- BUILD RELEASE Run buildrelease.bat in client project folder
SERVER
=-=-=-
- BUMP AyaNovaVersion.cs version NUMBER
- BUMP AyaNova.csproj version number
- BUILD RELEASE Run buildrelease.bat in server project folder
https://www.ayanova.com/download/ayanova8.alpha.136-win-x64.7z
https://www.ayanova.com/download/migrate136.7z
\*\*ARCHIVE IT, otherwise it could be false positived
- COPY TO DEVOPS SERVER
NOTE: if need to replace "files" subfolder on server the rights need to be set to 775
- Use filezilla to copy files that are new up to server
- Copy to "/home/john/xfer/ayanovadocker/files"
- These two files (and any other changes that are relevant)
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb
- CONSOLE TO SERVER VIA PUTTY
- Bring down current containers:
- navigate to ~/xfer folder
- execute sudo docker-compose down
- Build new image forcing it to update as it sometimes doesn't
- docker-compose build --force-rm --pull
(NOTE: this will _not_ pull newer postgres as it's an image already present so to update that need to do a sudo docker pull postgres:alpine first then build the image
if the major version is incompatible with the old database then there is a whole process to save it by dumping and re-building or just delete all files in the folder /var/lib/ayanova/db rm \* -r
to completely remove all the db files and the postgres image will make a new empty db on boot)
- Run new image
- sudo docker-compose up -d
- Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error)
- use the restartnginx.sh script in xfer at the server
- or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
- Test
- If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it.
- Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck
execute .\restartnginx.sh
- Check logs with sudo docker logs (containerID) to find out what happened
- Or in some cases (once) Digital Ocean fucked up something
- ERASE DB, FETCH LICENSE, GENERATE DATA
- ERASE DB:
- Stop container if not already stopped: execute sudo docker-compose down
- Edit nano docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db
- sudo docker-compose up -d
- Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable
- Start the container again with the up command
- FETCH TEST KEY:
- Go into the api explorer, authenticate then
- select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"}
- This seems to setup the db to accept a trial key when fetching the regular key next
- select the POST to license route (not the TRIAL one), this will fetch a test key and install it
- SEED DB:
- Go to trial route and pick seed level (HUGE for proper testing) and activate
- NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset
- As of today 2019-10-23 it takes 36 minutes which is actually right on track, not sure why it took 8 a year before but likely due to less stuff to do
### Publish command line:
Windows 64 bit:
dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained
dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained
Linux 64 bit:
Normal build without all the .net files (not self contained)
This is appropriate for docker based distribution since another image will contain the .net runtime:
#### DEFAULT BUILD COMMAND
dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
(linux)
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release
Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below):
dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained
Needed to change permissions on the AyaNova file to make it executable and also it requires these requirements and probably more:
apt-get install libunwind8
apt-get install libcurl3
//.net core 2.x linux native requirements
https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
Windows 32 bit:
dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained
Self contained Windows 10 x64:
dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained
PORTABLE RID's:
win-x64
win-x86
linux-x64
//D.O. Linux
ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary
- https://docs.microsoft.com/en-us/dotnet/core/deploying/index
- https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x
- https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x
- https://docs.microsoft.com/en-us/dotnet/core/rid-catalog
### DOCKER
Docker will eat up all the disk space if I don't prune it back once in a while, this seems safe to use and freed up 21GB of space when I ran out recently on devops
PRUNE: https://docs.docker.com/config/pruning/
- Build containers:
- john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build
- Run it:
- :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d
- Build it in prep for running it:
- dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
- john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release
- OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info)
- Save image:
- docker image save -o .\image\ay-alpha2 gztw/ayanova
- Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags
- Not compressed, can be compressed about 60% smaller
- Load image:
- docker image load -i saved_image_file_name_here
####
- Running docker at our D.O. server
- run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/
- To update:
- run a publish command to publish to my local dist/linux-x64/ayanovadocker/files
- Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files
- Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future)
- If necessary do a docker-compose build to rebuild
- run Nginx server:
- from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
- If necessary can switch to root with command: sudo su -
- documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
### DOCKER NGINX Let's Encrypt CERTBOT
- https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
- https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf
INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS)
#### STAGING
sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
certbot/certbot \
certonly --webroot \
--email support@ayanova.com --agree-tos --no-eff-email \
--webroot-path=/data/letsencrypt \
--staging \
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
#### PRODUCTION
sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
certbot/certbot \
certonly --webroot \
--email support@ayanova.com --agree-tos --no-eff-email \
--webroot-path=/data/letsencrypt \
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
#### SAMPLE OUTPUT:
john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \
> -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
> -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
> -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
> -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
> certbot/certbot \
> certonly --webroot \
> --email support@ayanova.com --agree-tos --no-eff-email \
> --webroot-path=/data/letsencrypt \
> -d helloayanova.com -d www.helloayanova.com
> Saving debug log to /var/log/letsencrypt/letsencrypt.log
> Plugins selected: Authenticator webroot, Installer None
> Obtaining a new certificate
> Performing the following challenges:
> http-01 challenge for helloayanova.com
> http-01 challenge for www.helloayanova.com
> Using the webroot path /data/letsencrypt for all unmatched domains.
> Waiting for verification...
> Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/helloayanova.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/helloayanova.com/privkey.pem
Your cert will expire on 2018-06-10. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew _all_ of your certificates, run
"certbot renew"
- Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
=-=-=-=-=-=-=-=-
DOCKER / LINUX CHEAT SHEET
Alpine linux enter container and run shell from docker:
sudo docker exec -it xfer_ayanova_1 ash
AYANOVA files are stored in droplet outside docker in /var/lib/ayanova
########################################################################################
Windows installer worksheet
INNOSETUP still the way to go apparently
SIGNING
Short story, probably not worth the hassle, needs to come from an authority, expires every year, basically bullshit
But if it becomes necessary then:
Can sign the setup to avoid issues:
Authenticode is what we want
https://docs.microsoft.com/en-us/visualstudio/deployment/clickonce-and-authenticode?view=vs-2019
This guy had to do it and has advice:
https://mkaz.blog/code/code-signing-a-windows-application/
This is how to find the sign tool to sign things:
where /R c:\ makecert.\*
https://stackoverflow.com/questions/51418366/makecert-exe-missing-in-windows-10-how-to-get-it-and-use-it
Useful looking:
https://blog.jayway.com/2014/09/03/creating-self-signed-certificates-with-makecert-exe-for-development/
Reference in New Issue View Git Blame Copy Permalink