admin/raven
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
34b85b5e9ff01d3a891adaeed525e76079313954
raven/devdocs/tools.txt
John Cardinal 8c66647e20
2018-11-28 00:38:00 +00:00

493 lines
22 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
## FRONT END DEV TOOLS
LOCALIZATION = MICROSOFT RESOURCE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- They have free access to all translations of all their products searchable
- https://www.microsoft.com/en-us/language/Search?&searchTerm=server&langID=French&Source=true&productid=All%20Products
UI FRAMEWORK = VUE.JS
=-=-=-=-=-=-=-=-=-=
VUE CLI being used
- config: https://github.com/vuejs/vue-cli/blob/dev/docs/config/README.md
INSTALLATION
- https://vuejs.org/v2/guide/installation.html
- https://stackoverflow.com/questions/39478855/how-to-setup-asp-net-core-vue-js#39881050
VUE COMPONENTS AND LIBRARIES
- https://github.com/vuejs/awesome-vue#components--libraries
- NOTE LOOKAT THIS: VETURE supports directly THESE: https://vuejs.github.io/vetur/framework.html
VUE DEV TOOLS FOR BROWSER
- https://github.com/vuejs/vue-devtools#vue-devtools
VSCODE VUE TOOLING
- https://vuejs.github.io/vetur/
VUE WEBPACK TEMPLATE
- https://vuejs-templates.github.io/webpack/
VUE APP ORGANIZING COMPONENTS
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
https://vuejs.org/v2/guide/components.html#Organizing-Components
API ACCESS STRUCTURE / ORGANIZATION
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Something along these lines: https://itnext.io/anyway-heres-how-to-do-ajax-api-calls-with-vue-js-e71e57d5cf12
VUE STYLE GUIDE
=-=-=-=-=-=-=-=-
https://vuejs.org/v2/style-guide/
AJAX LIBRARY = FETCH
=-=-=-=-=-=-=-=-=-=-
Fetch is the built in method in modern browsers for doing ajax and looked at libs like axios but they don't bring much that is needed to the table
- https://scotch.io/tutorials/how-to-use-the-javascript-fetch-api-to-get-data
CSS PREPROCESSOR = SASS
=-=-=-=-=-=-=-=-=-=-=-=-
Sass is the most widely used, and seems to have what I need
UNIT TESTING = JEST
=-=-=-=-=-=-=-=-=-=-
Not sure how much unit testing I'll be doing as I'm more interested in the end to end perhaps, but JEST has something called snapshot tests
where you don't write the test, just capture the output and that gets made into a test, if it changes or breaks then the test will fail.
This sounds ideal to me.
E2E TESTING = CYPRESS
=-=-=-=-=-=-=-=-=-=-=
Did a quick look, seems to be the best option, there is also Nightwatch but it's older and relies on a lot of tools where cypress is all built in supposedly
LINTER = ESLINT+PRETTIER CONFIG
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
VSCODE will use prettier to prettify my js, eslint will complain about some of it so using eslint with prettier config means
they won't conflict and eslint will focus on errors, not style so much that is handled by prettier
ROUTING = NOT HASHBANG, INSTEAD PATHLOCATIONSTRATEGY
=-=-=-=
Going to attempt pathlocationstrategy for url routing, meaning no hashbang like with pecklist and rockfish so the server will need
to be able to handle direct request and return the index.html instead if it looks like an AyaNova app url.
This page has a snippet showing routing handling at the server, seems to rely on identifying the url having no extension and simply returning index.htm instead
https://www.c-sharpcorner.com/article/single-page-application-using-asp-net-core-angular/
STATE = VUEX
=-=-=-=-=-=-
I'm convinced that a flux state library will be appropriate for RAVEN, mostly because several ui elements / modules will need to share the same data
and VUEX allows that easily, plus it's fairly straightforward to use.
Here is an example of how to structure a non trivial application with VUEX broken into modules:
https://vuex.vuejs.org/guide/structure.html
VUEX STATE PERSISTENCE = VUEX-PERSISTEDSTATE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- https://github.com/robinvdvleuten/vuex-persistedstate
- Many things need to be persisted and survive a refresh so this is how it will be done
DATE PARSE AND FORMAT = DayJS
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
- https://github.com/iamkun/dayjs/blob/master/docs/en/API-reference.md
SERVICEWORKER = WORKBOX
=-=-=-=-=-=-=-=-=-=-=-=
For offsite PWA use, works well and is simple to implement
https://developers.google.com/web/tools/workbox/guides/advanced-recipes
- Serviceworker push notifications
- INITIAL THINKING: avoid it for now until the dust settles, rely on email notification or other established means for now and in app popups perhaps as well
- This is also part of service workers api but is radically different than above as it's to do with integrating device notifications with web servers.
- requires permission, works with browser closed, messages go from server to intermediary such as google or microsoft or apple who then push it to the device.
- User can block and say no and then app can never after that re-request push notifications (as far as I can tell, maybe there's a workaround but not sure)
- It's main advantage is browser can be closed but still receive timely messages (i.e. new mail in rockfish would be an ideal use-case)
- Alternatives would be user must stay in web page to get notified via polling like how rockfish works now for mail.
- If I enable notifications via email then it kind of roundabout covers this use-case
- as of now 2018-10-22 15:23:49 iOS Safari (mobile apple devices browser) does not support push notifications, only dekstop Macs and you need to jump through many hoops to get approval
- Though, this guy seems to think he's got it working: https://janaks.com.np/sending-push-notification-to-ios-from-asp-net-core/
- This means basically that it would be a pain in the ass for apple devices but easier for others. Hmmm...
- SERVER SIDE: https://www.tpeczek.com/2017/12/push-notifications-and-aspnet-core-part.html
PWA VERIFICATION TOOL = https://developers.google.com/web/tools/lighthouse/
"You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, and more."
This tool will audit a PWA and ensure it meets all the requirements to work well.
(NOT CONFIRMED YET, MAY BE DRIVEN BY VUE CLI TOOLS ANYWAY) ## BUNDLING AND MINIFICATION
- https://docs.microsoft.com/en-us/aspnet/core/client-side/bundling-and-minification?tabs=visual-studio%2Caspnetcore2x
- Gulp seems best for me: https://docs.microsoft.com/en-us/aspnet/core/client-side/using-gulp
VUE UI FRAMEWORK = VUETIFY
=-=-=-=-=-=-=-=-=-=-=-=-=-
- Settled on Vuetify and Material Design look and feel (fuck iOS)
- Vuetify plugin for VUECLI3: https://github.com/vuetifyjs/vue-cli-plugin-vuetify
- Articles on vuetify on medium: https://medium.com/vuetify
VUE NOTES / IMPORTANT INFO
=-=-=-=-=-=-=-=-=-=-=-=-=-
- always use kebab-case for everything including component names, props, events etc.
- this just avoids all manner of potential issues
- If a runtime error occurs during a component’s render, it will be passed to the global Vue.config.errorHandler config function if it has been set. It might be a good idea to leverage this hook together with an error-tracking service like Sentry, which provides an official integration for Vue.
- Style guide: https://vuejs.org/v2/style-guide/
- DATA FETCHING WITH ROUTER: https://router.vuejs.org/guide/advanced/data-fetching.html#data-fetching
ICONS = FONTAWESOME 5 (free)
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
https://fontawesome.com/icons?d=gallery&s=regular,solid&m=free
UTILITY = LODASH
=-=-=-=-=-=-=-=-=-
- Using modularized one by one to save build space : https://www.blazemeter.com/blog/the-correct-way-to-import-lodash-libraries-a-benchmark
- https://lodash.com/custom-builds
POTENTIALLY USEFUL VUE / GENERAL COMPONENTS AND PLUGINS
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- Shortcuts (enable keyboard shortcuts to activate elements like buttons e.g. ctrl-s to save) https://github.com/iFgR/vue-shortkey
- Clickaway (detect user clicking outside an element e.g. to dismiss a popup or something) https://github.com/simplesmiler/vue-clickaway
- Theres actually many on vue-awesome so if use this check them all out for stars and issues
- parse and format currency for display: http://openexchangerates.github.io/accounting.js/
- CONTROLS (note some of these probably have a VUE component wrapper already if not the one already linked to, seek them out)
- Date time picker: https://flatpickr.js.org/
- Vue plugin version: https://github.com/ankurk91/vue-flatpickr-component
- Input validation component: https://github.com/baianat/vee-validate
- Progress bar (very clean and small, ideal for ajax calls): http://ricostacruz.com/nprogress/
- Charts: https://www.chartjs.org/
- Intro.js Introduction tool to train people in page, AWESOME!!: https://introjs.com/
- ICONS: Font Awesome: https://fontawesome.com/, in rockfish am using Material design icons: https://materialdesignicons.com/
- CSV converter: converts json to csv for download: https://github.com/ynishi/vuecsv
- TAG control for inputing tags: https://github.com/matiastucci/vue-input-tag
- More of them: https://github.com/vuejs/awesome-vue#type-select
- File upload controls: https://github.com/vuejs/awesome-vue#file-upload
- FAB button: https://github.com/PygmySlowLoris/vue-fab
- Image manipulation: https://github.com/vuejs/awesome-vue#image-manipulation
- Markdown: https://github.com/vuejs/awesome-vue#markdown
- Rich text editing: https://github.com/vuejs/awesome-vue#rich-text-editing
- Autocomplete: https://github.com/vuejs/awesome-vue#autocomplete
- QR code generator and readers: https://github.com/vuejs/awesome-vue#qr-code
- BARCODE scanner thing with info on actual scanners: https://github.com/noomerzx/vue-barcode-scanner
- Pull to refresh: https://github.com/vuejs/awesome-vue#pull-to-refresh
- Maps: https://github.com/vuejs/awesome-vue#map
- Notifications / Toast: https://github.com/vuejs/awesome-vue#notification
- Affix (keep sidebars in view or elements that hover): https://github.com/eddiemf/vue-affix
- Query builder: https://dabernathy89.github.io/vue-query-builder/
- Offline / Online detection: https://github.com/vinayakkulkarni/v-offline
- Cookie / Accept / Decline component: https://github.com/promosis/vue-cookie-accept-decline
## DEPLOYMENT
### DEPLOY TO DIGITAL OCEAN TEST SERVER
PUBLISH TO DEVOPS STEPS
BUILD CLIENT
- Clean out the last build in the server wwwRoot folder at: C:\data\code\raven\server\AyaNova\wwwroot
- Delete all but the \docs subfolder that contains the AyaNova manual unless planning on rebuilding that as well
- Go to client folder: C:\data\code\raven\app\ayanova
- Run npm run build
- Copy build from C:\data\code\raven\app\ayanova\dist to server wwwRoot folder: C:\data\code\raven\server\AyaNova\wwwroot
BUILD SERVER
- Make sure docs are built, use makedocs batch file which will build and put in wwwRoot folder
- Make sure updated version number first!!
- Need to be in C:\data\code\raven\server\AyaNova\
- Then run command:
- dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
- COPY TO SERVER
- Use filezilla to copy files that are new up to server
- Copy to "/home/john/xfer/ayanovadocker/files"
- These two files (and any other changes that are relevant)
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb
- CONSOLE TO SERVER VIA PUTTY
- Bring down current containers:
- navigate to ~/xfer folder
- execute sudo docker-compose down
- Build new image forcing it to update as it sometimes doesn't
- sudo docker-compose build --force-rm --pull
- Run new image
- sudo docker-compose up -d
- Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error)
- use the restartnginx.sh script in xfer at the server
- or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
- Test
- If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it.
- Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck
- Check logs with sudo docker logs [containerID] to find out what happened
- Or in some cases (once) Digital Ocean fucked up something
- ERASE DB, FETCH LICENSE, GENERATE DATA
- ERASE DB:
- Stop container if not already stopped: execute sudo docker-compose down
- Edit docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db
- sudo docker-compose up -d
- Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable
- Start the container again with the up command
- FETCH TEST KEY:
- Go into the api explorer, authenticate then
- select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"}
- This seems to setup the db to accept a trial key when fetching the regular key next
- select the POST to license route (not the TRIAL one), this will fetch a test key and install it
- SEED DB:
- Go to trial route and pick seed level (HUGE for proper testing) and activate
- NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset
### Publish command line:
Windows 64 bit:
dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained
dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained
Linux 64 bit:
Normal build without all the .net files (not self contained)
This is appropriate for docker based distribution since another image will contain the .net runtime:
#### DEFAULT BUILD COMMAND
dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
(linux)
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release
Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below):
dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained
Needed to change permissions on the AyaNova file to make it executable and also it requires these pre-requisites and probably more:
apt-get install libunwind8
apt-get install libcurl3
//.net core 2.x linux native requirements
https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
Windows 32 bit:
dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained
Self contained Windows 10 x64:
dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained
PORTABLE RID's:
win-x64
win-x86
linux-x64
//D.O. Linux
ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary
- https://docs.microsoft.com/en-us/dotnet/core/deploying/index
- https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x
- https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x
- https://docs.microsoft.com/en-us/dotnet/core/rid-catalog
### DOCKER
- Build containers:
- john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build
- Run it:
- :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d
- Build it in prep for running it:
- dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
- john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release
- OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info)
- Save image:
- docker image save -o .\image\ay-alpha2 gztw/ayanova
- Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags
- Not compressed, can be compressed about 60% smaller
- Load image:
- docker image load -i saved_image_file_name_here
####
- Running docker at our D.O. server
- run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/
- To update:
- run a publish command to publish to my local dist/linux-x64/ayanovadocker/files
- Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files
- Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future)
- If necessary do a docker-compose build to rebuild
- run Nginx server:
- from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
- If necessary can switch to root with command: sudo su -
- documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
## TESTING
- DATA SEEDING: https://github.com/bchavez/Bogus (a port of faker.js)
### DOCKER NGINX LETS ENCRYPT CERTBOT
- https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
- https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf
INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS)
#### STAGING
sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
certbot/certbot \
certonly --webroot \
--email support@ayanova.com --agree-tos --no-eff-email \
--webroot-path=/data/letsencrypt \
--staging \
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
#### PRODUCTION
sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
certbot/certbot \
certonly --webroot \
--email support@ayanova.com --agree-tos --no-eff-email \
--webroot-path=/data/letsencrypt \
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
#### SAMPLE OUTPUT:
john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \
> -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
> -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
> -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
> -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
> certbot/certbot \
> certonly --webroot \
> --email support@ayanova.com --agree-tos --no-eff-email \
> --webroot-path=/data/letsencrypt \
> -d helloayanova.com -d www.helloayanova.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for helloayanova.com
http-01 challenge for www.helloayanova.com
Using the webroot path /data/letsencrypt for all unmatched domains.
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/helloayanova.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/helloayanova.com/privkey.pem
Your cert will expire on 2018-06-10. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
"certbot renew"
- Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
=-=-=-=-=-=-=-=-
GRAFANA / INFLUXDB / DOCKER
Container to run the whole shebang:
- https://github.com/philhawthorne/docker-influxdb-grafana
docker run -d \
--name docker-influxdb-grafana \
-p 3003:3003 \
-p 3004:8083 \
-p 8086:8086 \
-p 22022:22 \
-v /path/for/influxdb:/var/lib/influxdb \
-v /path/for/grafana:/var/lib/grafana \
philhawthorne/docker-influxdb-grafana:latest
NOTE: you can leave out the paths and it works and the name is a little verbose
Dashboard for Grafana and app.metrics:
- https://grafana.com/dashboards/2125
MSBUILD reference for csproj file
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild#BKMK_ProjectFile
LARGE FILE GENERATION
=-=-=-=-=-=-=-=-=-=-=
Quickly generate large files in windows: http://tweaks.com/windows/62755/quickly-generate-large-test-files-in-windows/
Never download another 100mb test file or waste time searching for a large file. Sometimes you need a large file fast to test data transfers or disk performance. Windows includes a utility that allows you to quickly generate a file of any size instantly.
Open an administrative level command prompt.
Run the following command:
fsutil file createnew <file> <size in bytes>
For example, this command will create a 1GB file called 1gb.test on my desktop:
fsutil file createnew c:\users\steve\desktop\1gb.test 1073741824
The key is to input the size of the file in bytes so here are some common file sizes to save you from math:
1 MB = 1048576 bytes
100 MB = 104857600 bytes
1 GB = 1073741824 bytes
10 GB = 10737418240 bytes
100 GB =107374182400 bytes
1 TB = 1099511627776 bytes
10 TB =10995116277760 bytes
=-=-=-=-=-=-=-=-=-=-
DOCKER DEV
=-=-=-=-=-=-
After a reboot of dev machine the containers are stopped and need to be restarted on reboot with this command:
docker start dock-pg10 dock-pgadmin
**USE PGADMIN**
Browse to localhost 5050
Can view the status of all containers with
docker ps -a
ICON GENERATOR
=-=-=-=-=-=-=-=-
- Used to generate pwa icons from source svg: http://cthedot.de/icongen/
Reference in New Issue View Git Blame Copy Permalink