## FRONT END DEV TOOLS LOCALIZATION = MICROSOFT RESOURCE =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - They have free access to all translations of all their products searchable - https://www.microsoft.com/en-us/language/Search?&searchTerm=server&langID=French&Source=true&productid=All%20Products UI FRAMEWORK = VUE.JS =-=-=-=-=-=-=-=-=-= VUE CLI being used - config: https://github.com/vuejs/vue-cli/blob/dev/docs/config/README.md INSTALLATION - https://vuejs.org/v2/guide/installation.html - https://stackoverflow.com/questions/39478855/how-to-setup-asp-net-core-vue-js#39881050 VUE COMPONENTS AND LIBRARIES - https://github.com/vuejs/awesome-vue#components--libraries - NOTE LOOKAT THIS: VETURE supports directly THESE: https://vuejs.github.io/vetur/framework.html VUE DEV TOOLS FOR BROWSER - https://github.com/vuejs/vue-devtools#vue-devtools VSCODE VUE TOOLING - https://vuejs.github.io/vetur/ VUE WEBPACK TEMPLATE - https://vuejs-templates.github.io/webpack/ VUE APP ORGANIZING COMPONENTS =-=-=-=-=-=-=-=-=-=-=-=-=-=-= https://vuejs.org/v2/guide/components.html#Organizing-Components API ACCESS STRUCTURE / ORGANIZATION =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Something along these lines: https://itnext.io/anyway-heres-how-to-do-ajax-api-calls-with-vue-js-e71e57d5cf12 VUE STYLE GUIDE =-=-=-=-=-=-=-=- https://vuejs.org/v2/style-guide/ AJAX LIBRARY = FETCH =-=-=-=-=-=-=-=-=-=- Fetch is the built in method in modern browsers for doing ajax and looked at libs like axios but they don't bring much that is needed to the table - https://scotch.io/tutorials/how-to-use-the-javascript-fetch-api-to-get-data PROMISES - how to =-=-=-=-=-=-=-=-=- https://html5hive.org/how-to-chain-javascript-promises/ CSS PREPROCESSOR = SASS =-=-=-=-=-=-=-=-=-=-=-=- Sass is the most widely used, and seems to have what I need UNIT TESTING = JEST =-=-=-=-=-=-=-=-=-=- Not sure how much unit testing I'll be doing as I'm more interested in the end to end perhaps, but JEST has something called snapshot tests where you don't write the test, just capture the output and that gets made into a test, if it changes or breaks then the test will fail. This sounds ideal to me. E2E TESTING = CYPRESS =-=-=-=-=-=-=-=-=-=-= Did a quick look, seems to be the best option, there is also Nightwatch but it's older and relies on a lot of tools where cypress is all built in supposedly LINTER = ESLINT+PRETTIER CONFIG =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= VSCODE will use prettier to prettify my js, eslint will complain about some of it so using eslint with prettier config means they won't conflict and eslint will focus on errors, not style so much that is handled by prettier ROUTING = NOT HASHBANG, INSTEAD PATHLOCATIONSTRATEGY =-=-=-= Going to attempt pathlocationstrategy for url routing, meaning no hashbang like with pecklist and rockfish so the server will need to be able to handle direct request and return the index.html instead if it looks like an AyaNova app url. This page has a snippet showing routing handling at the server, seems to rely on identifying the url having no extension and simply returning index.htm instead https://www.c-sharpcorner.com/article/single-page-application-using-asp-net-core-angular/ STATE = VUEX =-=-=-=-=-=- I'm convinced that a flux state library will be appropriate for RAVEN, mostly because several ui elements / modules will need to share the same data and VUEX allows that easily, plus it's fairly straightforward to use. Here is an example of how to structure a non trivial application with VUEX broken into modules: https://vuex.vuejs.org/guide/structure.html VUEX STATE PERSISTENCE = VUEX-PERSISTEDSTATE =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - https://github.com/robinvdvleuten/vuex-persistedstate - Many things need to be persisted and survive a refresh so this is how it will be done DATE PARSE AND FORMAT = DayJS =-=-=-=-=-=-=-=-=-=-=-=-=-=-= - https://github.com/iamkun/dayjs/blob/master/docs/en/API-reference.md SERVICEWORKER = WORKBOX =-=-=-=-=-=-=-=-=-=-=-= For offsite PWA use, works well and is simple to implement https://developers.google.com/web/tools/workbox/guides/advanced-recipes - Serviceworker push notifications - INITIAL THINKING: avoid it for now until the dust settles, rely on email notification or other established means for now and in app popups perhaps as well - This is also part of service workers api but is radically different than above as it's to do with integrating device notifications with web servers. - requires permission, works with browser closed, messages go from server to intermediary such as google or microsoft or apple who then push it to the device. - User can block and say no and then app can never after that re-request push notifications (as far as I can tell, maybe there's a workaround but not sure) - It's main advantage is browser can be closed but still receive timely messages (i.e. new mail in rockfish would be an ideal use-case) - Alternatives would be user must stay in web page to get notified via polling like how rockfish works now for mail. - If I enable notifications via email then it kind of roundabout covers this use-case - as of now 2018-10-22 15:23:49 iOS Safari (mobile apple devices browser) does not support push notifications, only dekstop Macs and you need to jump through many hoops to get approval - Though, this guy seems to think he's got it working: https://janaks.com.np/sending-push-notification-to-ios-from-asp-net-core/ - This means basically that it would be a pain in the ass for apple devices but easier for others. Hmmm... - SERVER SIDE: https://www.tpeczek.com/2017/12/push-notifications-and-aspnet-core-part.html PWA VERIFICATION TOOL = https://developers.google.com/web/tools/lighthouse/ "You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, and more." This tool will audit a PWA and ensure it meets all the requirements to work well. (NOT CONFIRMED YET, MAY BE DRIVEN BY VUE CLI TOOLS ANYWAY) ## BUNDLING AND MINIFICATION - https://docs.microsoft.com/en-us/aspnet/core/client-side/bundling-and-minification?tabs=visual-studio%2Caspnetcore2x - Gulp seems best for me: https://docs.microsoft.com/en-us/aspnet/core/client-side/using-gulp VUE UI FRAMEWORK = VUETIFY =-=-=-=-=-=-=-=-=-=-=-=-=- - Settled on Vuetify and Material Design look and feel (fuck iOS) - Vuetify plugin for VUECLI3: https://github.com/vuetifyjs/vue-cli-plugin-vuetify - Articles on vuetify on medium: https://medium.com/vuetify VUE DIALOGS / SNACKS / NOTIFICATIONS ETC =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- https://www.npmjs.com/package/vuetify-dialog VUE NOTES / IMPORTANT INFO =-=-=-=-=-=-=-=-=-=-=-=-=- - always use kebab-case for everything including component names, props, events etc. - this just avoids all manner of potential issues - If a runtime error occurs during a component’s render, it will be passed to the global Vue.config.errorHandler config function if it has been set. It might be a good idea to leverage this hook together with an error-tracking service like Sentry, which provides an official integration for Vue. - Style guide: https://vuejs.org/v2/style-guide/ - DATA FETCHING WITH ROUTER: https://router.vuejs.org/guide/advanced/data-fetching.html#data-fetching ICONS = FONTAWESOME 5 (free) =-=-=-=-=-=-=-=-=-=-=-=-=-=- https://fontawesome.com/icons?d=gallery&s=regular,solid&m=free UTILITY = LODASH =-=-=-=-=-=-=-=-=- - Using modularized one by one to save build space : https://www.blazemeter.com/blog/the-correct-way-to-import-lodash-libraries-a-benchmark - https://lodash.com/custom-builds LOCALE NUMERIC FORMATTING UTILITY =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- https://github.com/globalizejs/globalize ? POTENTIALLY USEFUL VUE / GENERAL COMPONENTS AND PLUGINS =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Code editor component (might be good for report editing): https://www.vuescript.com/vue-prism-code-editor/ - Markdown parser: https://github.com/miaolz123/vue-markdown - Shortcuts (enable keyboard shortcuts to activate elements like buttons e.g. ctrl-s to save) https://github.com/iFgR/vue-shortkey - Clickaway (detect user clicking outside an element e.g. to dismiss a popup or something) https://github.com/simplesmiler/vue-clickaway - Theres actually many on vue-awesome so if use this check them all out for stars and issues - parse and format currency for display: http://openexchangerates.github.io/accounting.js/ - CONTROLS (note some of these probably have a VUE component wrapper already if not the one already linked to, seek them out) - Date time picker: - This one works but is new and a bit fucky but it's based on the built in components so it's getting there and looks correct and seems to work - https://github.com/darrenfang/vuetify-datetime-picker - note: can't use the one below because the style is not at all similar - https://madewithvuejs.com/vue-datetime - https://github.com/mariomka/vue-datetime - Input validation component: https://github.com/baianat/vee-validate - Progress bar (very clean and small, ideal for ajax calls): http://ricostacruz.com/nprogress/ - Charts: https://www.chartjs.org/ - Intro.js Introduction tool to train people in page, AWESOME!!: https://introjs.com/ - ICONS: Font Awesome: https://fontawesome.com/, in rockfish am using Material design icons: https://materialdesignicons.com/ - CSV converter: converts json to csv for download: https://github.com/ynishi/vuecsv - TAG control for inputing tags: https://github.com/matiastucci/vue-input-tag - More of them: https://github.com/vuejs/awesome-vue#type-select - File upload controls: https://github.com/vuejs/awesome-vue#file-upload - FAB button: https://github.com/PygmySlowLoris/vue-fab - Image manipulation: https://github.com/vuejs/awesome-vue#image-manipulation - Markdown: https://github.com/vuejs/awesome-vue#markdown - Rich text editing: https://github.com/vuejs/awesome-vue#rich-text-editing - Autocomplete: https://github.com/vuejs/awesome-vue#autocomplete - QR code generator and readers: https://github.com/vuejs/awesome-vue#qr-code - BARCODE scanner thing with info on actual scanners: https://github.com/noomerzx/vue-barcode-scanner - Pull to refresh: https://github.com/vuejs/awesome-vue#pull-to-refresh - Maps: https://github.com/vuejs/awesome-vue#map - Notifications / Toast: https://github.com/vuejs/awesome-vue#notification - Affix (keep sidebars in view or elements that hover): https://github.com/eddiemf/vue-affix - Query builder: https://dabernathy89.github.io/vue-query-builder/ - Offline / Online detection: https://github.com/vinayakkulkarni/v-offline - Cookie / Accept / Decline component: https://github.com/promosis/vue-cookie-accept-decline - CLIPBOARD: https://clipboardjs.com/ ## DEPLOYMENT ### DEPLOY TO DIGITAL OCEAN TEST SERVER PUBLISH TO DEVOPS STEPS BUILD CLIENT - Clean out the last build in the server wwwRoot folder at: C:\data\code\raven\server\AyaNova\wwwroot - Delete all but the \docs subfolder that contains the AyaNova manual unless planning on rebuilding that as well - Go to client folder: C:\data\code\raven\app\ayanova - Run npm run build - Copy build from C:\data\code\raven\app\ayanova\dist to server wwwRoot folder: C:\data\code\raven\server\AyaNova\wwwroot BUILD SERVER - Make sure docs are built, use makedocs batch file which will build and put in wwwRoot folder - Make sure updated version number first!! - Need to be in C:\data\code\raven\server\AyaNova\ - Then run command: - dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release - COPY TO SERVER - Use filezilla to copy files that are new up to server - Copy to "/home/john/xfer/ayanovadocker/files" - These two files (and any other changes that are relevant) - C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll - C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb - CONSOLE TO SERVER VIA PUTTY - Bring down current containers: - navigate to ~/xfer folder - execute sudo docker-compose down - Build new image forcing it to update as it sometimes doesn't - sudo docker-compose build --force-rm --pull - Run new image - sudo docker-compose up -d - Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error) - use the restartnginx.sh script in xfer at the server - or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d - Test - If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it. - Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck - Check logs with sudo docker logs [containerID] to find out what happened - Or in some cases (once) Digital Ocean fucked up something - ERASE DB, FETCH LICENSE, GENERATE DATA - ERASE DB: - Stop container if not already stopped: execute sudo docker-compose down - Edit docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db - sudo docker-compose up -d - Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable - Start the container again with the up command - FETCH TEST KEY: - Go into the api explorer, authenticate then - select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"} - This seems to setup the db to accept a trial key when fetching the regular key next - select the POST to license route (not the TRIAL one), this will fetch a test key and install it - SEED DB: - Go to trial route and pick seed level (HUGE for proper testing) and activate - NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset ### Publish command line: Windows 64 bit: dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained Linux 64 bit: Normal build without all the .net files (not self contained) This is appropriate for docker based distribution since another image will contain the .net runtime: #### DEFAULT BUILD COMMAND dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release (linux) dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below): dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained Needed to change permissions on the AyaNova file to make it executable and also it requires these pre-requisites and probably more: apt-get install libunwind8 apt-get install libcurl3 //.net core 2.x linux native requirements https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x Windows 32 bit: dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained Self contained Windows 10 x64: dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained PORTABLE RID's: win-x64 win-x86 linux-x64 //D.O. Linux ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary - https://docs.microsoft.com/en-us/dotnet/core/deploying/index - https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x - https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x - https://docs.microsoft.com/en-us/dotnet/core/rid-catalog ### DOCKER - Build containers: - john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build - Run it: - :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d - Build it in prep for running it: - dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release - john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release - OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info) - Save image: - docker image save -o .\image\ay-alpha2 gztw/ayanova - Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags - Not compressed, can be compressed about 60% smaller - Load image: - docker image load -i saved_image_file_name_here #### - Running docker at our D.O. server - run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/ - To update: - run a publish command to publish to my local dist/linux-x64/ayanovadocker/files - Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files - Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future) - If necessary do a docker-compose build to rebuild - run Nginx server: - from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d - If necessary can switch to root with command: sudo su - - documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx ## TESTING - DATA SEEDING: https://github.com/bchavez/Bogus (a port of faker.js) ### DOCKER NGINX LETS ENCRYPT CERTBOT - https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx - https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS) #### STAGING sudo docker run -it --rm \ -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ certbot/certbot \ certonly --webroot \ --email support@ayanova.com --agree-tos --no-eff-email \ --webroot-path=/data/letsencrypt \ --staging \ -d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com #### PRODUCTION sudo docker run -it --rm \ -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ certbot/certbot \ certonly --webroot \ --email support@ayanova.com --agree-tos --no-eff-email \ --webroot-path=/data/letsencrypt \ -d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com #### SAMPLE OUTPUT: john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \ > -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \ > -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \ > -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \ > -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \ > certbot/certbot \ > certonly --webroot \ > --email support@ayanova.com --agree-tos --no-eff-email \ > --webroot-path=/data/letsencrypt \ > -d helloayanova.com -d www.helloayanova.com Saving debug log to /var/log/letsencrypt/letsencrypt.log Plugins selected: Authenticator webroot, Installer None Obtaining a new certificate Performing the following challenges: http-01 challenge for helloayanova.com http-01 challenge for www.helloayanova.com Using the webroot path /data/letsencrypt for all unmatched domains. Waiting for verification... Cleaning up challenges IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at: /etc/letsencrypt/live/helloayanova.com/fullchain.pem Your key file has been saved at: /etc/letsencrypt/live/helloayanova.com/privkey.pem Your cert will expire on 2018-06-10. To obtain a new or tweaked version of this certificate in the future, simply run certbot again. To non-interactively renew *all* of your certificates, run "certbot renew" - Your account credentials have been saved in your Certbot configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Certbot so making regular backups of this folder is ideal. - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le =-=-=-=-=-=-=-=- GRAFANA / INFLUXDB / DOCKER Container to run the whole shebang: - https://github.com/philhawthorne/docker-influxdb-grafana docker run -d \ --name docker-influxdb-grafana \ -p 3003:3003 \ -p 3004:8083 \ -p 8086:8086 \ -p 22022:22 \ -v /path/for/influxdb:/var/lib/influxdb \ -v /path/for/grafana:/var/lib/grafana \ philhawthorne/docker-influxdb-grafana:latest NOTE: you can leave out the paths and it works and the name is a little verbose Dashboard for Grafana and app.metrics: - https://grafana.com/dashboards/2125 MSBUILD reference for csproj file =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild#BKMK_ProjectFile LARGE FILE GENERATION =-=-=-=-=-=-=-=-=-=-= Quickly generate large files in windows: http://tweaks.com/windows/62755/quickly-generate-large-test-files-in-windows/ Never download another 100mb test file or waste time searching for a large file. Sometimes you need a large file fast to test data transfers or disk performance. Windows includes a utility that allows you to quickly generate a file of any size instantly. Open an administrative level command prompt. Run the following command: fsutil file createnew For example, this command will create a 1GB file called 1gb.test on my desktop: fsutil file createnew c:\users\steve\desktop\1gb.test 1073741824 The key is to input the size of the file in bytes so here are some common file sizes to save you from math: 1 MB = 1048576 bytes 100 MB = 104857600 bytes 1 GB = 1073741824 bytes 10 GB = 10737418240 bytes 100 GB =107374182400 bytes 1 TB = 1099511627776 bytes 10 TB =10995116277760 bytes =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- //DATA TYPES .net to postgres map //http://www.npgsql.org/doc/types/basic.html ICON GENERATOR =-=-=-=-=-=-=-=- - Used to generate pwa icons from source svg: http://cthedot.de/icongen/ =-=-=-=-=-=-=-=-=-=- POSTGRES SQL PORTABLE: https://github.com/rsubr/postgresql-portable/releases/tag/v12.1 DOCKER POSTGRES DEVELOPMENT STATION COMMANDS AND USEFUL INFO =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- OFFICIAL DOCKER POSTGRES IMAGE https://hub.docker.com/_/postgres/ DEV SETUP Using two containers dock-pg10 and dock-pgadmin Command to list them all: docker container ls Command to see the detailed configuration of them: docker container inspect [CONTAINER_NAME OR ID] They use a volume in docker called "raven" C:\data\code\raven\tools>docker volume inspect raven [ { "CreatedAt": "2019-05-09T17:03:34Z", "Driver": "local", "Labels": {}, "Mountpoint": "/var/lib/docker/volumes/raven/_data", "Name": "raven", "Options": {}, "Scope": "local" } ] To list volumes: docker volume ls IMAGES =-=-=- dock-pg10 uses the official "postgres" docker image dock-pgadmin uses "chorss/docker-pgadmin4" USEFUL COMMANDS ---------------- DUMPING POSTGRES DATA OUT OF IMAGE: docker exec dock-pg10 pg_dumpall -U postgres > dump.sql START A PG SERVER CONTAINER: TEMPORARILY: docker run --rm --name pg -p 5432:5432 -e POSTGRES_PASSWORD=raven -e POSTGRES_DB=AyaNova -d postgres (this will remove it after it closes with the rm switch and the rest is the requirements for ayanova, no idea where the data goes and it won't be kept AFAIK) FANCY WITH VOLUME: docker run --name pg11 -e POSTGRES_PASSWORD=raven -d -p 5432:5432 -v /var/lib/docker/volumes/raven/_data:/var/lib/postgresql/data postgres STATUS: Can view the status of all containers with docker ps -a PGADMIN =-=-=-=- Official docker hub pgadmin 4 image: https://hub.docker.com/r/dpage/pgadmin4/ RUN IT: docker run --rm -p 5050:80 --link pg -e "PGADMIN_DEFAULT_EMAIL=support@ayanova.com" -e "PGADMIN_DEFAULT_PASSWORD=abraxis" -d dpage/pgadmin4 Browse to the site either localhost or if a port specified then another port Login using support@ayanova.com and abraxis Add the server by host name of pg (this is the pg container name and also the --link parameter in the pgadmin run command) Use the credentials specified for postgres which is username postgres and password raven (specified on command line for running pg container) TRANSLATIONS / TRANSLATE / LOCALIZATION / LOCALE Microsoft handy dandy product translation search database site: https://www.microsoft.com/en-us/language/Search Google translate to verify: https://translate.google.com/ POSTGRES STUFF: Mass generate data: query: insert into ametriccpu (t,v) select CURRENT_TIMESTAMP, 58.43239007949476 from generate_series(1, 525600) s(i) insert into ametriccpu ( t,v1,v2,v3,v4,v5,v6,v7,v8,v9,v10 ) select LOCALTIMESTAMP, 58.43239007949476, 0.33006058073955513, 102.44723488288768, 46.078341513002755, 30.23570573933185, 0.000136518543824419, 65.8400891412282, 0.01, 58.43239007949476, 58.43239007949476 from generate_series(1, 525600) s(i) JAVASCRIPT DOWNSAMPLING TOOL //==================== //https://github.com/joshcarr/largest-triangle-three-buckets.js/blob/master/lib/largest-triangle-three-buckets.js 'use strict'; function largestTriangleThreeBuckets( data, threshold, xAccessor, yAccessor ) { var floor = Math.floor, abs = Math.abs, dataLength = data.length, sampled = [], sampledIndex = 0, every = ( dataLength - 2 ) / ( threshold - 2 ), // Bucket size. Leave room for start and end data points a = 0, // Initially a is the first point in the triangle maxAreaPoint, maxArea, area, nextA, i, avgX = 0, avgY = 0, avgRangeStart, avgRangeEnd, avgRangeLength, rangeOffs, rangeTo, pointAX, pointAY; if ( threshold >= dataLength || threshold === 0 ) { return data; // Nothing to do } sampled[ sampledIndex++ ] = data[ a ]; // Always add the first point for ( i = 0; i < threshold - 2; i++ ) { // Calculate point average for next bucket (containing c) avgX = 0; avgY = 0; avgRangeStart = floor( ( i + 1 ) * every ) + 1; avgRangeEnd = floor( ( i + 2 ) * every ) + 1; avgRangeEnd = avgRangeEnd < dataLength ? avgRangeEnd : dataLength; avgRangeLength = avgRangeEnd - avgRangeStart; for ( ; avgRangeStart < avgRangeEnd; avgRangeStart++ ) { avgX += data[ avgRangeStart ][ xAccessor ] * 1; // * 1 enforces Number (value may be Date) avgY += data[ avgRangeStart ][ yAccessor ] * 1; } avgX /= avgRangeLength; avgY /= avgRangeLength; // Get the range for this bucket rangeOffs = floor( ( i + 0 ) * every ) + 1; rangeTo = floor( ( i + 1 ) * every ) + 1; // Point a pointAX = data[ a ][ xAccessor ] * 1; // enforce Number (value may be Date) pointAY = data[ a ][ yAccessor ] * 1; maxArea = area = -1; for ( ; rangeOffs < rangeTo; rangeOffs++ ) { // Calculate triangle area over three buckets area = abs( ( pointAX - avgX ) * ( data[ rangeOffs ][ yAccessor ] - pointAY ) - ( pointAX - data[ rangeOffs ][ xAccessor ] ) * ( avgY - pointAY ) ) * 0.5; if ( area > maxArea ) { maxArea = area; maxAreaPoint = data[ rangeOffs ]; nextA = rangeOffs; // Next a is this b } } sampled[ sampledIndex++ ] = maxAreaPoint; // Pick this point from the bucket a = nextA; // This a is the next a (chosen b) } sampled[ sampledIndex++ ] = data[ dataLength - 1 ]; // Always add last return sampled; } //============================================= COlors colours for charts http://perceptualedge.com/articles/b-eye/choosing_colors.pdf MiniProfiler how to access stats from code // This is how to access the stored profiler stats if I ever need to // var profiler = MiniProfiler.StartNew("My Profiler Name"); // if (profiler != null) // { // var Options = profiler.Options; // var guids = Options.Storage.List(100); // // var lastId = context.Request["last-id"]; // // if (!lastId.IsNullOrWhiteSpace() && Guid.TryParse(lastId, out var lastGuid)) // // { // // guids = guids.TakeWhile(g => g != lastGuid); // // } // var ministats = guids.Reverse() // .Select(g => Options.Storage.Load(g)) // .Where(p => p != null) // .Select(p => new // { // p.Id, // p.Name, // p.ClientTimings, // p.Started, // p.HasUserViewed, // p.MachineName, // p.User, // p.DurationMilliseconds // }).ToList(); // if(ministats.Count>0){ // var v=ministats.Count; // } // } MEMORY LEAK DIAGNOSIS URLS https://docs.microsoft.com/en-us/dotnet/core/diagnostics/debug-memory-leak https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-counters https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-trace Might be doing GC Counts incorrectly, they don't sync up with the dotnet-counters tool which shows them drop back to zero regularly when I stop burn test https://michaelscodingspot.com/ways-to-cause-memory-leaks-in-dotnet/ https://bennettadelson.wordpress.com/2013/04/11/using-perfview-to-diagnose-a-net-memory-leak-2/ HOW TO SEE WHO CALLED A METHOD DURING RUNTIME keywords: caller, call method stack trace stacktrace //System.Diagnostics.Debug.WriteLine((new System.Diagnostics.StackTrace()).GetFrame(1).GetMethod().Name); //System.Diagnostics.Debug.WriteLine((new System.Diagnostics.StackTrace()).ToString().Substring(56,80));