## FRONT END DEV TOOLS UI FRAMEWORK = VUE.JS =-=-=-=-=-=-=-=-=-= VUE CLI being used - config: https://github.com/vuejs/vue-cli/blob/dev/docs/config/README.md INSTALLATION - https://vuejs.org/v2/guide/installation.html - https://stackoverflow.com/questions/39478855/how-to-setup-asp-net-core-vue-js#39881050 VUE COMPONENTS AND LIBRARIES - https://github.com/vuejs/awesome-vue#components--libraries - NOTE LOOKAT THIS: VETURE supports directly THESE: https://vuejs.github.io/vetur/framework.html VUE DEV TOOLS FOR BROWSER - https://github.com/vuejs/vue-devtools#vue-devtools VSCODE VUE TOOLING - https://vuejs.github.io/vetur/ VUE WEBPACK TEMPLATE - https://vuejs-templates.github.io/webpack/ VUE APP ORGANIZING COMPONENTS =-=-=-=-=-=-=-=-=-=-=-=-=-=-= https://vuejs.org/v2/guide/components.html#Organizing-Components VUE STYLE GUIDE =-=-=-=-=-=-=-=- https://vuejs.org/v2/style-guide/ AJAX LIBRARY = FETCH =-=-=-=-=-=-=-=-=-=- Fetch is the built in method in modern browsers for doing ajax and looked at libs like axios but they don't bring much that is needed to the table CSS PREPROCESSOR = SASS =-=-=-=-=-=-=-=-=-=-=-=- Sass is the most widely used, and seems to have what I need UNIT TESTING = JEST =-=-=-=-=-=-=-=-=-=- Not sure how much unit testing I'll be doing as I'm more interested in the end to end perhaps, but JEST has something called snapshot tests where you don't write the test, just capture the output and that gets made into a test, if it changes or breaks then the test will fail. This sounds ideal to me. E2E TESTING = CYPRESS =-=-=-=-=-=-=-=-=-=-= Did a quick look, seems to be the best option, there is also Nightwatch but it's older and relies on a lot of tools where cypress is all built in supposedly LINTER = ESLINT+PRETTIER CONFIG =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= VSCODE will use prettier to prettify my js, eslint will complain about some of it so using eslint with prettier config means they won't conflict and eslint will focus on errors, not style so much that is handled by prettier ROUTING = NOT HASHBANG, INSTEAD PATHLOCATIONSTRATEGY =-=-=-= Going to attempt pathlocationstrategy for url routing, meaning no hashbang like with pecklist and rockfish so the server will need to be able to handle direct request and return the index.html instead if it looks like an AyaNova app url. This page has a snippet showing routing handling at the server, seems to rely on identifying the url having no extension and simply returning index.htm instead https://www.c-sharpcorner.com/article/single-page-application-using-asp-net-core-angular/ STATE = VUEX =-=-=-=-=-=- I'm convinced that a flux state library will be appropriate for RAVEN, mostly because several ui elements / modules will need to share the same data and VUEX allows that easily, plus it's fairly straightforward to use. Here is an example of how to structure a non trivial application with VUEX broken into modules: https://vuex.vuejs.org/guide/structure.html SERVICEWORKER = WORKBOX =-=-=-=-=-=-=-=-=-=-=-= For offsite PWA use, works well and is simple to implement https://developers.google.com/web/tools/workbox/guides/advanced-recipes - Serviceworker push notifications - INITIAL THINKING: avoid it for now until the dust settles, rely on email notification or other established means for now and in app popups perhaps as well - This is also part of service workers api but is radically different than above as it's to do with integrating device notifications with web servers. - requires permission, works with browser closed, messages go from server to intermediary such as google or microsoft or apple who then push it to the device. - User can block and say no and then app can never after that re-request push notifications (as far as I can tell, maybe there's a workaround but not sure) - It's main advantage is browser can be closed but still receive timely messages (i.e. new mail in rockfish would be an ideal use-case) - Alternatives would be user must stay in web page to get notified via polling like how rockfish works now for mail. - If I enable notifications via email then it kind of roundabout covers this use-case - as of now 2018-10-22 15:23:49 iOS Safari (mobile apple devices browser) does not support push notifications, only dekstop Macs and you need to jump through many hoops to get approval - Though, this guy seems to think he's got it working: https://janaks.com.np/sending-push-notification-to-ios-from-asp-net-core/ - This means basically that it would be a pain in the ass for apple devices but easier for others. Hmmm... - SERVER SIDE: https://www.tpeczek.com/2017/12/push-notifications-and-aspnet-core-part.html PWA VERIFICATION TOOL = https://developers.google.com/web/tools/lighthouse/ "You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, and more." This tool will audit a PWA and ensure it meets all the requirements to work well. (NOT CONFIRMED YET, MAY BE DRIVEN BY VUE CLI TOOLS ANYWAY) ## BUNDLING AND MINIFICATION - https://docs.microsoft.com/en-us/aspnet/core/client-side/bundling-and-minification?tabs=visual-studio%2Caspnetcore2x - Gulp seems best for me: https://docs.microsoft.com/en-us/aspnet/core/client-side/using-gulp VUE NOTES / IMPORTANT INFO =-=-=-=-=-=-=-=-=-=-=-=-=- - always use kebab-case for everything including component names, props, events etc. - this just avoids all manner of potential issues - If a runtime error occurs during a component’s render, it will be passed to the global Vue.config.errorHandler config function if it has been set. It might be a good idea to leverage this hook together with an error-tracking service like Sentry, which provides an official integration for Vue. - Style guide: https://vuejs.org/v2/style-guide/ - DATA FETCHING WITH ROUTER: https://router.vuejs.org/guide/advanced/data-fetching.html#data-fetching POTENTIALLY USEFUL VUE COMPONENTS AND PLUGINS =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Shortcuts (enable keyboard shortcuts to activate elements like buttons e.g. ctrl-s to save) https://github.com/iFgR/vue-shortkey - Clickaway (detect user clicking outside an element e.g. to dismiss a popup or something) https://github.com/simplesmiler/vue-clickaway - Theres actually many on vue-awesome so if use this check them all out for stars and issues - parse and format currency for display: http://openexchangerates.github.io/accounting.js/ - Date time picker: https://flatpickr.js.org/ - Vue plugin version: https://github.com/ankurk91/vue-flatpickr-component - Input validation component: https://github.com/baianat/vee-validate - Progress bar (very clean and small, ideal for ajax calls): http://ricostacruz.com/nprogress/ ## DEPLOYMENT ### DEPLOY TO DIGITAL OCEAN TEST SERVER - PUBLISH: - Make sure updated version number first!! - Need to be in C:\data\code\raven\server\AyaNova\ - Then run command: - dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release - COPY - Use filezilla to copy files that are new up to server - Copy to "/home/john/xfer/ayanovadocker/files" - These two files (and any other changes that are relevant) - C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll - C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb - CONSOLE TO SERVER VIA PUTTY - Bring down current containers: - navigate to ~/xfer folder - execute sudo docker-compose down - Build new image forcing it to update as it sometimes doesn't - sudo docker-compose build --force-rm --pull - Run new image - sudo docker-compose up -d - Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error) - use the restartnginx.sh script in xfer at the server - or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d - Test - If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it. - Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck - Check logs with sudo docker logs [containerID] to find out what happened - Or in some cases (once) Digital Ocean fucked up something - ERASE DB, FETCH LICENSE, GENERATE DATA - ERASE DB: - Stop container if not already stopped: execute sudo docker-compose down - Edit docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db - sudo docker-compose up -d - Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable - Start the container again with the up command - FETCH TEST KEY: - Go into the api explorer, authenticate then - select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"} - This seems to setup the db to accept a trial key when fetching the regular key next - select the POST to license route (not the TRIAL one), this will fetch a test key and install it - SEED DB: - Go to trial route and pick seed level (HUGE for proper testing) and activate - NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset ### Publish command line: Windows 64 bit: dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained Linux 64 bit: Normal build without all the .net files (not self contained) This is appropriate for docker based distribution since another image will contain the .net runtime: #### DEFAULT BUILD COMMAND dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release (linux) dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below): dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained Needed to change permissions on the AyaNova file to make it executable and also it requires these pre-requisites and probably more: apt-get install libunwind8 apt-get install libcurl3 //.net core 2.x linux native requirements https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x Windows 32 bit: dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained Self contained Windows 10 x64: dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained PORTABLE RID's: win-x64 win-x86 linux-x64 //D.O. Linux ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary - https://docs.microsoft.com/en-us/dotnet/core/deploying/index - https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x - https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x - https://docs.microsoft.com/en-us/dotnet/core/rid-catalog ### DOCKER - Build containers: - john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build - Run it: - :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d - Build it in prep for running it: - dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release - john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release - OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info) - Save image: - docker image save -o .\image\ay-alpha2 gztw/ayanova - Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags - Not compressed, can be compressed about 60% smaller - Load image: - docker image load -i saved_image_file_name_here #### - Running docker at our D.O. server - run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/ - To update: - run a publish command to publish to my local dist/linux-x64/ayanovadocker/files - Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files - Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future) - If necessary do a docker-compose build to rebuild - run Nginx server: - from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d - If necessary can switch to root with command: sudo su - - documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx ## TESTING - DATA SEEDING: https://github.com/bchavez/Bogus (a port of faker.js) ### DOCKER NGINX LETS ENCRYPT CERTBOT - https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx - https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS) #### STAGING sudo docker run -it --rm \ -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ certbot/certbot \ certonly --webroot \ --email support@ayanova.com --agree-tos --no-eff-email \ --webroot-path=/data/letsencrypt \ --staging \ -d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com #### PRODUCTION sudo docker run -it --rm \ -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ certbot/certbot \ certonly --webroot \ --email support@ayanova.com --agree-tos --no-eff-email \ --webroot-path=/data/letsencrypt \ -d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com #### SAMPLE OUTPUT: john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \ > -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ > -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ > -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ > -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ > certbot/certbot \ > certonly --webroot \ > --email support@ayanova.com --agree-tos --no-eff-email \ > --webroot-path=/data/letsencrypt \ > -d helloayanova.com -d www.helloayanova.com Saving debug log to /var/log/letsencrypt/letsencrypt.log Plugins selected: Authenticator webroot, Installer None Obtaining a new certificate Performing the following challenges: http-01 challenge for helloayanova.com http-01 challenge for www.helloayanova.com Using the webroot path /data/letsencrypt for all unmatched domains. Waiting for verification... Cleaning up challenges IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at: /etc/letsencrypt/live/helloayanova.com/fullchain.pem Your key file has been saved at: /etc/letsencrypt/live/helloayanova.com/privkey.pem Your cert will expire on 2018-06-10. To obtain a new or tweaked version of this certificate in the future, simply run certbot again. To non-interactively renew *all* of your certificates, run "certbot renew" - Your account credentials have been saved in your Certbot configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Certbot so making regular backups of this folder is ideal. - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le =-=-=-=-=-=-=-=- GRAFANA / INFLUXDB / DOCKER Container to run the whole shebang: - https://github.com/philhawthorne/docker-influxdb-grafana docker run -d \ --name docker-influxdb-grafana \ -p 3003:3003 \ -p 3004:8083 \ -p 8086:8086 \ -p 22022:22 \ -v /path/for/influxdb:/var/lib/influxdb \ -v /path/for/grafana:/var/lib/grafana \ philhawthorne/docker-influxdb-grafana:latest NOTE: you can leave out the paths and it works and the name is a little verbose Dashboard for Grafana and app.metrics: - https://grafana.com/dashboards/2125 MSBUILD reference for csproj file =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild#BKMK_ProjectFile LARGE FILE GENERATION =-=-=-=-=-=-=-=-=-=-= Quickly generate large files in windows: http://tweaks.com/windows/62755/quickly-generate-large-test-files-in-windows/ Never download another 100mb test file or waste time searching for a large file. Sometimes you need a large file fast to test data transfers or disk performance. Windows includes a utility that allows you to quickly generate a file of any size instantly. Open an administrative level command prompt. Run the following command: fsutil file createnew For example, this command will create a 1GB file called 1gb.test on my desktop: fsutil file createnew c:\users\steve\desktop\1gb.test 1073741824 The key is to input the size of the file in bytes so here are some common file sizes to save you from math: 1 MB = 1048576 bytes 100 MB = 104857600 bytes 1 GB = 1073741824 bytes 10 GB = 10737418240 bytes 100 GB =107374182400 bytes 1 TB = 1099511627776 bytes 10 TB =10995116277760 bytes =-=-=-=-=-=-=-=-=-=- DOCKER DEV =-=-=-=-=-=- After a reboot of dev machine the containers are stopped and need to be restarted on reboot with this command: docker start dock-pg10 dock-pgadmin **USE PGADMIN** Browse to localhost 5050 Can view the status of all containers with docker ps -a