459 lines
19 KiB
Plaintext
459 lines
19 KiB
Plaintext
## BUIILD AND DEPLOYMENT
|
|
|
|
|
|
|
|
### Security scan
|
|
If any packages have been changed in the release do a thorough security scan and check of each package
|
|
|
|
|
|
|
|
### Bump version numbers:
|
|
|
|
Search and replace 8.0.0-beta.2
|
|
webapp,server,launcher, v8migrate
|
|
|
|
Client end ayanova-version.js,
|
|
Server end ayanova-version.cs
|
|
The TWO windows installers are versioned
|
|
RavenLauncher "single" Launcher is also versioned and needs to be changed individually
|
|
Docs index.md file is also versioned
|
|
|
|
#MyAppVersion in C:\data\code\raven\dist\install\windows\x64\lan.iss
|
|
#MyAppVersion in C:\data\code\raven\dist\install\windows\x64\standalone.iss
|
|
|
|
### CLIENT DEV MODE TURNED OFF
|
|
Make sure main.js in client devmode is set to false (should just keep it that way unless diagnosing a specific issue during development)
|
|
|
|
### run build-release.bat
|
|
check for errors
|
|
|
|
### Test install to devops server
|
|
### Test install to windows server
|
|
|
|
### Run e2e tests against Linux and Windows servers
|
|
First adjust to the server desired by editing cypress.json and commenting in or out apiBaseUrl and BaseUrl props for local or devops or wherever
|
|
C:\data\code\raven-test\e2e>npm run all-chrome
|
|
(chrome most reliable but *will* fail intermittently so needs multiple runs often)
|
|
Once tests pass move on to next step
|
|
|
|
### Run load tests against Linux and Windows servers
|
|
(determine if any changes triggered performance issues, needs baseline comparison value)
|
|
|
|
LOAD TESTING STEPS:
|
|
Login to server to test and seed fresh data first, then can run this test
|
|
run C:\data\code\raven-test\load\runtest.bat
|
|
Test will take approximately 14 minutes to run
|
|
Check results in results folder by opening index.html from windows explorer and view in browser
|
|
|
|
BASELINE WHAT TO CHECK:
|
|
DASHBOARD PAGE is the main page with the main stats:
|
|
"REquests summary"
|
|
The most telling overall stat is the "Requests summary" pie chart pass/fail ratio on the main page
|
|
As of today 2021-12-21 release 8-beta.06 the results are PASS: 92.87%, FAIL: 7.13%
|
|
Errors *are* expected, it's a stress / load test but there shouldn't be an entirely failing route, just a scattered set of failures,
|
|
typically unauthorized or bad gateway
|
|
|
|
"Appdex" - skipping this for now, not really relevant at the moment
|
|
|
|
"STATISTICS" table - this is the important one as it shows failures
|
|
- Scroll down and look in "executions" -> "FAIL" column and make sure they are not all in one line item
|
|
if see a lot of failures in a single route need to manually look into that exact issue and route and examine the jmeter test for that route / run it manually perhaps
|
|
it's normal to see unauthorized or bad gateway errors when the server is super stressed out but other failures might need examination or all of it if less than 90% success
|
|
|
|
|
|
NOTES:
|
|
- this also catches reporting as it generates some reports
|
|
report-render test has explicit report id number for workorder, currently it's 15 for the dispatch report, if all fail it may have changed the id number
|
|
would be better deterministically but for now this works.
|
|
|
|
- To test local or devops edit the project file "C:\data\code\raven-test\load\stressdevops.jmx"
|
|
There are two configuration sections near top for local or devops and it's switched by setting either enabled or disabled:
|
|
<ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="Devops request defaults" enabled="true">
|
|
or
|
|
<ConfigTestElement guiclass="HttpDefaultsGui" testclass="ConfigTestElement" testname="Localhost request defaults" enabled="false">
|
|
- Load test needs fresh seeding so it can create test users.
|
|
|
|
|
|
Once tests pass within expected parameters move on to next step
|
|
|
|
### MOVE TO PREVIOUS
|
|
Clean out the previous previous download from the previous folder
|
|
NOTE: if this is the final major release then it should go in the applicable download folder
|
|
|
|
Copy the existing downloads to the previous folder preserving (-p) the timestamps
|
|
Change directory to the previous folder
|
|
From the previous folder: cp ../* . -p
|
|
(note: will not copy folders so ok to use)
|
|
|
|
### DELETE CURRENT DOWNLOADS
|
|
(OR JUST OVERWRITE THEM IS EASIER)
|
|
root@ubuntu-s-1vcpu-1gb-amd-sfo3-01:/var/www/html/ayanova.com/download# rm *
|
|
(note: will not remove folders just files by default so ok to use)
|
|
|
|
### upload 4 installer packages found in C:\data\code\raven\dist\installers
|
|
- Follow path and folder and url rules in case 4053 https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/4053
|
|
- And also v8-migrate.exe if applicable
|
|
|
|
### Upload static copy of docs to our website
|
|
clean out current one from web server: (DONT USE FILEZILLA TO DELETE IT TAKES FUCKING FOREVER)
|
|
root@ubuntu-s-1vcpu-1gb-amd-sfo3-01:/var/www/html/ayanova.com/docs# rm * -r
|
|
(WATCH THE PATH!!)
|
|
|
|
Upload docs:
|
|
from here: C:\data\code\raven\server\AyaNova\wwwroot\docs\
|
|
to here: /var/www/html/ayanova.com/docs
|
|
|
|
### Update website changelog and release notes etc
|
|
|
|
### profit$$
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# LINUX MANUAL UPGRADE PROCEDURE
|
|
|
|
From /var/ayanova folder
|
|
|
|
Download new version (the -O will overwrite so no need to delete existing one):
|
|
|
|
sudo curl -O https://www.ayanova.com/download/ayanova-linux-x64-server.zip
|
|
|
|
Run this one-liner to update and restart:
|
|
sudo systemctl stop ayanova.service && rm *.dll && rm -r ./resource && rm -r ./wwwroot && unzip -o ayanova-linux-x64-server.zip -x "config.json" && sudo systemctl start ayanova.service
|
|
|
|
|
|
|
|
OLD step by step instructions:
|
|
sudo systemctl stop ayanova.service
|
|
rm *.dll
|
|
rm -r ./resource
|
|
rm -r ./wwwroot
|
|
unzip -o ayanova-linux-x64-server.zip -x "config.json"
|
|
sudo systemctl start ayanova.service
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
==========================================================================================================================
|
|
==========================================================================================================================
|
|
==========================================================================================================================
|
|
========================================= OLD OUTDATED INFO BEYOND HERE ==================================================
|
|
==========================================================================================================================
|
|
==========================================================================================================================
|
|
==========================================================================================================================
|
|
==========================================================================================================================
|
|
PUBLIC / INSTALLER
|
|
- bump versions, make sure client main.js devmode is false (probably should keep it that way for ongoing use and only enable when dev issue)
|
|
set version in Server, client and installer iss files
|
|
- run buildrelease.bat in client project folder
|
|
- run buildrelease.bat in server project folder
|
|
- Test deploy here first
|
|
- Post to downloads on ayanova.com
|
|
|
|
|
|
|
|
### DEPLOY TO DIGITAL OCEAN TEST SERVER
|
|
|
|
PUBLISH TO DEVOPS STEPS
|
|
|
|
FIRST
|
|
=-=-=-
|
|
***** BEFORE RELEASE DOUBLE CHECK CAN RE-SEED FROM CLIENT UI AS THERE IS OFTEN ISSUES WITH DBUTIL ERASE TABLES AND THE ORDERING ETC ******
|
|
|
|
CLIENT
|
|
=-=-=-=
|
|
- TURN OFF DEV MODE IN MAIN.JS
|
|
- BUMP PACKAGE.JSON version number
|
|
- BUMP CLIENT AyaNova-version.js VERSION NUMBER
|
|
- BUILD RELEASE Run buildrelease.bat in client project folder
|
|
|
|
|
|
SERVER
|
|
=-=-=-
|
|
- BUMP AyaNovaVersion.cs version NUMBER
|
|
- BUMP AyaNova.csproj version number
|
|
- BUILD RELEASE Run buildrelease.bat in server project folder
|
|
|
|
https://www.ayanova.com/download/ayanova8.alpha.136-win-x64.7z
|
|
https://www.ayanova.com/download/migrate136.7z
|
|
**ARCHIVE IT, otherwise it could be false positived
|
|
|
|
- COPY TO DEVOPS SERVER
|
|
NOTE: if need to replace "files" subfolder on server the rights need to be set to 775
|
|
- Use filezilla to copy files that are new up to server
|
|
- Copy to "/home/john/xfer/ayanovadocker/files"
|
|
- These two files (and any other changes that are relevant)
|
|
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll
|
|
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb
|
|
|
|
- CONSOLE TO SERVER VIA PUTTY
|
|
- Bring down current containers:
|
|
- navigate to ~/xfer folder
|
|
- execute sudo docker-compose down
|
|
|
|
- Build new image forcing it to update as it sometimes doesn't
|
|
- docker-compose build --force-rm --pull
|
|
|
|
(NOTE: this will *not* pull newer postgres as it's an image already present so to update that need to do a sudo docker pull postgres:alpine first then build the image
|
|
if the major version is incompatible with the old database then there is a whole process to save it by dumping and re-building or just delete all files in the folder /var/lib/ayanova/db rm * -r
|
|
to completely remove all the db files and the postgres image will make a new empty db on boot)
|
|
|
|
- Run new image
|
|
- sudo docker-compose up -d
|
|
|
|
- Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error)
|
|
- use the restartnginx.sh script in xfer at the server
|
|
- or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
|
|
|
|
- Test
|
|
- If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it.
|
|
- Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck
|
|
execute .\restartnginx.sh
|
|
- Check logs with sudo docker logs [containerID] to find out what happened
|
|
- Or in some cases (once) Digital Ocean fucked up something
|
|
|
|
- ERASE DB, FETCH LICENSE, GENERATE DATA
|
|
- ERASE DB:
|
|
- Stop container if not already stopped: execute sudo docker-compose down
|
|
- Edit nano docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db
|
|
- sudo docker-compose up -d
|
|
- Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable
|
|
- Start the container again with the up command
|
|
- FETCH TEST KEY:
|
|
- Go into the api explorer, authenticate then
|
|
- select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"}
|
|
- This seems to setup the db to accept a trial key when fetching the regular key next
|
|
- select the POST to license route (not the TRIAL one), this will fetch a test key and install it
|
|
- SEED DB:
|
|
- Go to trial route and pick seed level (HUGE for proper testing) and activate
|
|
- NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset
|
|
- As of today 2019-10-23 it takes 36 minutes which is actually right on track, not sure why it took 8 a year before but likely due to less stuff to do
|
|
|
|
|
|
### Publish command line:
|
|
|
|
Windows 64 bit:
|
|
dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained
|
|
dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained
|
|
|
|
|
|
Linux 64 bit:
|
|
|
|
Normal build without all the .net files (not self contained)
|
|
This is appropriate for docker based distribution since another image will contain the .net runtime:
|
|
|
|
#### DEFAULT BUILD COMMAND
|
|
dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
|
|
|
|
(linux)
|
|
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release
|
|
|
|
|
|
Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below):
|
|
dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained
|
|
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained
|
|
|
|
Needed to change permissions on the AyaNova file to make it executable and also it requires these requirements and probably more:
|
|
apt-get install libunwind8
|
|
apt-get install libcurl3
|
|
|
|
//.net core 2.x linux native requirements
|
|
https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
|
|
|
|
|
|
|
|
Windows 32 bit:
|
|
dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained
|
|
|
|
Self contained Windows 10 x64:
|
|
dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained
|
|
|
|
PORTABLE RID's:
|
|
win-x64
|
|
win-x86
|
|
linux-x64
|
|
|
|
//D.O. Linux
|
|
ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary
|
|
|
|
|
|
- https://docs.microsoft.com/en-us/dotnet/core/deploying/index
|
|
- https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x
|
|
- https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x
|
|
- https://docs.microsoft.com/en-us/dotnet/core/rid-catalog
|
|
|
|
|
|
### DOCKER
|
|
|
|
Docker will eat up all the disk space if I don't prune it back once in a while, this seems safe to use and freed up 21GB of space when I ran out recently on devops
|
|
PRUNE: https://docs.docker.com/config/pruning/
|
|
|
|
- Build containers:
|
|
- john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build
|
|
- Run it:
|
|
- :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d
|
|
- Build it in prep for running it:
|
|
- dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
|
|
- john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release
|
|
|
|
|
|
- OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info)
|
|
- Save image:
|
|
- docker image save -o .\image\ay-alpha2 gztw/ayanova
|
|
- Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags
|
|
- Not compressed, can be compressed about 60% smaller
|
|
- Load image:
|
|
- docker image load -i saved_image_file_name_here
|
|
|
|
|
|
####
|
|
- Running docker at our D.O. server
|
|
- run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/
|
|
- To update:
|
|
- run a publish command to publish to my local dist/linux-x64/ayanovadocker/files
|
|
- Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files
|
|
- Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future)
|
|
- If necessary do a docker-compose build to rebuild
|
|
- run Nginx server:
|
|
- from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
|
|
- If necessary can switch to root with command: sudo su -
|
|
- documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
|
|
|
|
|
|
|
|
### DOCKER NGINX Let's Encrypt CERTBOT
|
|
- https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
|
|
- https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf
|
|
|
|
|
|
|
|
INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS)
|
|
|
|
#### STAGING
|
|
sudo docker run -it --rm \
|
|
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
|
|
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
|
|
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
|
|
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
|
|
certbot/certbot \
|
|
certonly --webroot \
|
|
--email support@ayanova.com --agree-tos --no-eff-email \
|
|
--webroot-path=/data/letsencrypt \
|
|
--staging \
|
|
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
|
|
|
|
#### PRODUCTION
|
|
sudo docker run -it --rm \
|
|
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
|
|
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
|
|
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
|
|
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
|
|
certbot/certbot \
|
|
certonly --webroot \
|
|
--email support@ayanova.com --agree-tos --no-eff-email \
|
|
--webroot-path=/data/letsencrypt \
|
|
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
|
|
|
|
|
|
#### SAMPLE OUTPUT:
|
|
john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \
|
|
> -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
|
|
> -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
|
|
> -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
|
|
> -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
|
|
> certbot/certbot \
|
|
> certonly --webroot \
|
|
> --email support@ayanova.com --agree-tos --no-eff-email \
|
|
> --webroot-path=/data/letsencrypt \
|
|
> -d helloayanova.com -d www.helloayanova.com
|
|
Saving debug log to /var/log/letsencrypt/letsencrypt.log
|
|
Plugins selected: Authenticator webroot, Installer None
|
|
Obtaining a new certificate
|
|
Performing the following challenges:
|
|
http-01 challenge for helloayanova.com
|
|
http-01 challenge for www.helloayanova.com
|
|
Using the webroot path /data/letsencrypt for all unmatched domains.
|
|
Waiting for verification...
|
|
Cleaning up challenges
|
|
|
|
IMPORTANT NOTES:
|
|
- Congratulations! Your certificate and chain have been saved at:
|
|
/etc/letsencrypt/live/helloayanova.com/fullchain.pem
|
|
Your key file has been saved at:
|
|
/etc/letsencrypt/live/helloayanova.com/privkey.pem
|
|
Your cert will expire on 2018-06-10. To obtain a new or tweaked
|
|
version of this certificate in the future, simply run certbot
|
|
again. To non-interactively renew *all* of your certificates, run
|
|
"certbot renew"
|
|
- Your account credentials have been saved in your Certbot
|
|
configuration directory at /etc/letsencrypt. You should make a
|
|
secure backup of this folder now. This configuration directory will
|
|
also contain certificates and private keys obtained by Certbot so
|
|
making regular backups of this folder is ideal.
|
|
- If you like Certbot, please consider supporting our work by:
|
|
|
|
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
|
|
Donating to EFF: https://eff.org/donate-le
|
|
|
|
|
|
=-=-=-=-=-=-=-=-
|
|
|
|
DOCKER / LINUX CHEAT SHEET
|
|
Alpine linux enter container and run shell from docker:
|
|
sudo docker exec -it xfer_ayanova_1 ash
|
|
|
|
AYANOVA files are stored in droplet outside docker in /var/lib/ayanova
|
|
|
|
|
|
########################################################################################
|
|
Windows installer worksheet
|
|
|
|
INNOSETUP still the way to go apparently
|
|
|
|
|
|
SIGNING
|
|
Short story, probably not worth the hassle, needs to come from an authority, expires every year, basically bullshit
|
|
But if it becomes necessary then:
|
|
|
|
Can sign the setup to avoid issues:
|
|
Authenticode is what we want
|
|
https://docs.microsoft.com/en-us/visualstudio/deployment/clickonce-and-authenticode?view=vs-2019
|
|
|
|
This guy had to do it and has advice:
|
|
https://mkaz.blog/code/code-signing-a-windows-application/
|
|
|
|
This is how to find the sign tool to sign things:
|
|
where /R c:\ makecert.*
|
|
https://stackoverflow.com/questions/51418366/makecert-exe-missing-in-windows-10-how-to-get-it-and-use-it
|
|
|
|
Useful looking:
|
|
https://blog.jayway.com/2014/09/03/creating-self-signed-certificates-with-makecert-exe-for-development/ |