admin/raven
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
341e1fc85af7cd095b329564dfd0c44273a20bd5
raven/devdocs/tools.txt
John Cardinal 589f697e08
2021-12-10 23:12:18 +00:00

782 lines
32 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
## FRONT END DEV TOOLS
LOCALIZATION = MICROSOFT RESOURCE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- They have free access to all translations of all their products searchable
- https://www.microsoft.com/en-us/language/Search?&searchTerm=server&langID=French&Source=true&productid=All%20Products
UI FRAMEWORK = VUE.JS
=-=-=-=-=-=-=-=-=-=
VUE CLI being used
- config: https://github.com/vuejs/vue-cli/blob/dev/docs/config/README.md
INSTALLATION
- https://vuejs.org/v2/guide/installation.html
- https://stackoverflow.com/questions/39478855/how-to-setup-asp-net-core-vue-js#39881050
VUE COMPONENTS AND LIBRARIES
- https://github.com/vuejs/awesome-vue#components--libraries
- NOTE LOOKAT THIS: VETURE supports directly THESE: https://vuejs.github.io/vetur/framework.html
VUE DEV TOOLS FOR BROWSER
- https://github.com/vuejs/vue-devtools#vue-devtools
VSCODE VUE TOOLING
- https://vuejs.github.io/vetur/
VUE WEBPACK TEMPLATE
- https://vuejs-templates.github.io/webpack/
VUE APP ORGANIZING COMPONENTS
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
https://vuejs.org/v2/guide/components.html#Organizing-Components
API ACCESS STRUCTURE / ORGANIZATION
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Something along these lines: https://itnext.io/anyway-heres-how-to-do-ajax-api-calls-with-vue-js-e71e57d5cf12
VUE STYLE GUIDE
=-=-=-=-=-=-=-=-
https://vuejs.org/v2/style-guide/
AJAX LIBRARY = FETCH
=-=-=-=-=-=-=-=-=-=-
Fetch is the built in method in modern browsers for doing ajax and looked at libs like axios but they don't bring much that is needed to the table
- https://scotch.io/tutorials/how-to-use-the-javascript-fetch-api-to-get-data
PROMISES - how to
=-=-=-=-=-=-=-=-=-
https://html5hive.org/how-to-chain-javascript-promises/
CSS PREPROCESSOR = SASS
=-=-=-=-=-=-=-=-=-=-=-=-
Sass is the most widely used, and seems to have what I need
UNIT TESTING = JEST
=-=-=-=-=-=-=-=-=-=-
Not sure how much unit testing I'll be doing as I'm more interested in the end to end perhaps, but JEST has something called snapshot tests
where you don't write the test, just capture the output and that gets made into a test, if it changes or breaks then the test will fail.
This sounds ideal to me.
E2E TESTING = CYPRESS
=-=-=-=-=-=-=-=-=-=-=
Did a quick look, seems to be the best option, there is also Nightwatch but it's older and relies on a lot of tools where cypress is all built in supposedly
LINTER = ESLINT+PRETTIER CONFIG
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
VSCODE will use prettier to prettify my js, eslint will complain about some of it so using eslint with prettier config means
they won't conflict and eslint will focus on errors, not style so much that is handled by prettier
ROUTING = NOT HASHBANG, INSTEAD PATHLOCATIONSTRATEGY
=-=-=-=
Going to attempt pathlocationstrategy for url routing, meaning no hashbang like with pecklist and rockfish so the server will need
to be able to handle direct request and return the index.html instead if it looks like an AyaNova app url.
This page has a snippet showing routing handling at the server, seems to rely on identifying the url having no extension and simply returning index.htm instead
https://www.c-sharpcorner.com/article/single-page-application-using-asp-net-core-angular/
STATE = VUEX
=-=-=-=-=-=-
I'm convinced that a flux state library will be appropriate for RAVEN, mostly because several ui elements / modules will need to share the same data
and VUEX allows that easily, plus it's fairly straightforward to use.
Here is an example of how to structure a non trivial application with VUEX broken into modules:
https://vuex.vuejs.org/guide/structure.html
VUEX STATE PERSISTENCE = VUEX-PERSISTEDSTATE
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- https://github.com/robinvdvleuten/vuex-persistedstate
- Many things need to be persisted and survive a refresh so this is how it will be done
DATE PARSE AND FORMAT = DayJS
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
- https://github.com/iamkun/dayjs/blob/master/docs/en/API-reference.md
SERVICEWORKER = WORKBOX
=-=-=-=-=-=-=-=-=-=-=-=
For offsite PWA use, works well and is simple to implement
https://developers.google.com/web/tools/workbox/guides/advanced-recipes
- Serviceworker push notifications
- INITIAL THINKING: avoid it for now until the dust settles, rely on email notification or other established means for now and in app popups perhaps as well
- This is also part of service workers api but is radically different than above as it's to do with integrating device notifications with web servers.
- requires permission, works with browser closed, messages go from server to intermediary such as google or microsoft or apple who then push it to the device.
- User can block and say no and then app can never after that re-request push notifications (as far as I can tell, maybe there's a workaround but not sure)
- It's main advantage is browser can be closed but still receive timely messages (i.e. new mail in rockfish would be an ideal use-case)
- Alternatives would be user must stay in web page to get notified via polling like how rockfish works now for mail.
- If I enable notifications via email then it kind of roundabout covers this use-case
- as of now 2018-10-22 15:23:49 iOS Safari (mobile apple devices browser) does not support push notifications, only dekstop Macs and you need to jump through many hoops to get approval
- Though, this guy seems to think he's got it working: https://janaks.com.np/sending-push-notification-to-ios-from-asp-net-core/
- This means basically that it would be a pain in the ass for apple devices but easier for others. Hmmm...
- SERVER SIDE: https://www.tpeczek.com/2017/12/push-notifications-and-aspnet-core-part.html
PWA VERIFICATION TOOL = https://developers.google.com/web/tools/lighthouse/
"You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, and more."
This tool will audit a PWA and ensure it meets all the requirements to work well.
(NOT CONFIRMED YET, MAY BE DRIVEN BY VUE CLI TOOLS ANYWAY) ## BUNDLING AND MINIFICATION
- https://docs.microsoft.com/en-us/aspnet/core/client-side/bundling-and-minification?tabs=visual-studio%2Caspnetcore2x
- Gulp seems best for me: https://docs.microsoft.com/en-us/aspnet/core/client-side/using-gulp
VUE UI FRAMEWORK = VUETIFY
=-=-=-=-=-=-=-=-=-=-=-=-=-
- Settled on Vuetify and Material Design look and feel (fuck iOS)
- Vuetify plugin for VUECLI3: https://github.com/vuetifyjs/vue-cli-plugin-vuetify
- Articles on vuetify on medium: https://medium.com/vuetify
VUE DIALOGS / SNACKS / NOTIFICATIONS ETC
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
https://www.npmjs.com/package/vuetify-dialog
VUE NOTES / IMPORTANT INFO
=-=-=-=-=-=-=-=-=-=-=-=-=-
- always use kebab-case for everything including component names, props, events etc.
- this just avoids all manner of potential issues
- If a runtime error occurs during a component’s render, it will be passed to the global Vue.config.errorHandler config function if it has been set. It might be a good idea to leverage this hook together with an error-tracking service like Sentry, which provides an official integration for Vue.
- Style guide: https://vuejs.org/v2/style-guide/
- DATA FETCHING WITH ROUTER: https://router.vuejs.org/guide/advanced/data-fetching.html#data-fetching
ICONS = FONTAWESOME 5 (free)
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
https://fontawesome.com/icons?d=gallery&s=regular,solid&m=free
UTILITY = LODASH
=-=-=-=-=-=-=-=-=-
- Using modularized one by one to save build space : https://www.blazemeter.com/blog/the-correct-way-to-import-lodash-libraries-a-benchmark
- https://lodash.com/custom-builds
LOCALE NUMERIC FORMATTING UTILITY
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
https://github.com/globalizejs/globalize ?
POTENTIALLY USEFUL VUE / GENERAL COMPONENTS AND PLUGINS
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- Code editor component (might be good for report editing): https://www.vuescript.com/vue-prism-code-editor/
- Markdown parser: https://github.com/miaolz123/vue-markdown
- Shortcuts (enable keyboard shortcuts to activate elements like buttons e.g. ctrl-s to save) https://github.com/iFgR/vue-shortkey
- Clickaway (detect user clicking outside an element e.g. to dismiss a popup or something) https://github.com/simplesmiler/vue-clickaway
- Theres actually many on vue-awesome so if use this check them all out for stars and issues
- parse and format currency for display: http://openexchangerates.github.io/accounting.js/
- CONTROLS (note some of these probably have a VUE component wrapper already if not the one already linked to, seek them out)
- Date time picker:
- This one works but is new and a bit flakey but it's based on the built in components so it's getting there and looks correct and seems to work
- https://github.com/darrenfang/vuetify-datetime-picker
- note: can't use the one below because the style is not at all similar
- https://madewithvuejs.com/vue-datetime
- https://github.com/mariomka/vue-datetime
- Input validation component: https://github.com/baianat/vee-validate
- Progress bar (very clean and small, ideal for ajax calls): http://ricostacruz.com/nprogress/
- Charts: https://www.chartjs.org/
- Intro.js Introduction tool to train people in page, AWESOME!!: https://introjs.com/
- ICONS: Font Awesome: https://fontawesome.com/, in rockfish am using Material design icons: https://materialdesignicons.com/
- CSV converter: converts json to csv for download: https://github.com/ynishi/vuecsv
- TAG control for inputing tags: https://github.com/matiastucci/vue-input-tag
- More of them: https://github.com/vuejs/awesome-vue#type-select
- File upload controls: https://github.com/vuejs/awesome-vue#file-upload
- FAB button: https://github.com/PygmySlowLoris/vue-fab
- Image manipulation: https://github.com/vuejs/awesome-vue#image-manipulation
- Markdown: https://github.com/vuejs/awesome-vue#markdown
- Rich text editing: https://github.com/vuejs/awesome-vue#rich-text-editing
- Autocomplete: https://github.com/vuejs/awesome-vue#autocomplete
- QR code generator and readers: https://github.com/vuejs/awesome-vue#qr-code
- BARCODE scanner thing with info on actual scanners: https://github.com/noomerzx/vue-barcode-scanner
- Pull to refresh: https://github.com/vuejs/awesome-vue#pull-to-refresh
- Maps: https://github.com/vuejs/awesome-vue#map
- Notifications / Toast: https://github.com/vuejs/awesome-vue#notification
- Affix (keep sidebars in view or elements that hover): https://github.com/eddiemf/vue-affix
- Query builder: https://dabernathy89.github.io/vue-query-builder/
- Offline / Online detection: https://github.com/vinayakkulkarni/v-offline
- Cookie / Accept / Decline component: https://github.com/promosis/vue-cookie-accept-decline
- CLIPBOARD: https://clipboardjs.com/
## DEPLOYMENT
### DEPLOY TO DIGITAL OCEAN TEST SERVER
PUBLISH TO DEVOPS STEPS
BUILD CLIENT
- Clean out the last build in the server wwwRoot folder at: C:\data\code\raven\server\AyaNova\wwwroot
- Delete all but the \docs subfolder that contains the AyaNova manual unless planning on rebuilding that as well
- Go to client folder: C:\data\code\raven\app\ayanova
- Run npm run build
- Copy build from C:\data\code\raven\app\ayanova\dist to server wwwRoot folder: C:\data\code\raven\server\AyaNova\wwwroot
BUILD SERVER
- Make sure docs are built, use makedocs batch file which will build and put in wwwRoot folder
- Make sure updated version number first!!
- Need to be in C:\data\code\raven\server\AyaNova\
- Then run command:
- dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
- COPY TO SERVER
- Use filezilla to copy files that are new up to server
- Copy to "/home/john/xfer/ayanovadocker/files"
- These two files (and any other changes that are relevant)
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.dll
- C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\AyaNova.pdb
- CONSOLE TO SERVER VIA PUTTY
- Bring down current containers:
- navigate to ~/xfer folder
- execute sudo docker-compose down
- Build new image forcing it to update as it sometimes doesn't
- sudo docker-compose build --force-rm --pull
- Run new image
- sudo docker-compose up -d
- Restart NGINX container (IF NECESSARY) as it seems to sometimes lose it's mind when the AyaNova container is restarted (502 BAD GATEWAY error)
- use the restartnginx.sh script in xfer at the server
- or from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
- Test
- If 502 BAD GATEWAY then AyaNova server is not up so the NGINX config bombs because it's proxying to it.
- Actually, it just happened and what needs to be done is AyaNova container needs to be running BEFORE nginx container or it seems to get stuck
- Check logs with sudo docker logs [containerID] to find out what happened
- Or in some cases (once) Digital Ocean fucked up something
- ERASE DB, FETCH LICENSE, GENERATE DATA
- ERASE DB:
- Stop container if not already stopped: execute sudo docker-compose down
- Edit docker-compose.yml, uncomment line with erase db environment variable and re-start to erase db
- sudo docker-compose up -d
- Stop the container again, use nano to edit docker-compose.yml and re-comment the erase db environment variable
- Start the container again with the up command
- FETCH TEST KEY:
- Go into the api explorer, authenticate then
- select the POST to license Trial route first { "registeredTo": "TestCo", "emailAddress": "cardjohn@ayanova.com"}
- This seems to setup the db to accept a trial key when fetching the regular key next
- select the POST to license route (not the TRIAL one), this will fetch a test key and install it
- SEED DB:
- Go to trial route and pick seed level (HUGE for proper testing) and activate
- NOTE: as of today 2018-10-9 it takes 8 minutes at the Devops server to generate the HUGE dataset
### Publish command line:
Windows 64 bit:
dotnet publish -o /home/john/Documents/raven/dist/server/win-x64/ -r win-x64 -c Release --self-contained
dotnet publish -o C:\data\code\raven\dist\server\win-x64\ -r win-x64 -c Release --self-contained
Linux 64 bit:
Normal build without all the .net files (not self contained)
This is appropriate for docker based distribution since another image will contain the .net runtime:
#### DEFAULT BUILD COMMAND
dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
(linux)
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ayanovadocker/files/ -c Release
Self contained (this is appropriate for non containerized distribution, but still requires some Linux native requirements - see below):
dotnet publish -o C:\data\code\raven\dist\server\linux-x64\ -r linux-x64 -c Release --self-contained
dotnet publish -o ~/Documents/raven/dist/server/linux-x64/ -r linux-x64 -c Release --self-contained
Needed to change permissions on the AyaNova file to make it executable and also it requires these requirements and probably more:
apt-get install libunwind8
apt-get install libcurl3
//.net core 2.x linux native requirements
https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
Windows 32 bit:
dotnet publish -o /home/john/Documents/raven/dist/server/win-x86/ -r win-x86 -c Release --self-contained
Self contained Windows 10 x64:
dotnet publish -o /home/john/Documents/raven/dist/server/win10x64/ -r win10-x64 -c Release --self-contained
PORTABLE RID's:
win-x64
win-x86
linux-x64
//D.O. Linux
ubuntu.16.04-x64 //<--- ends up being the same size as portable linux 64 so not really necessary
- https://docs.microsoft.com/en-us/dotnet/core/deploying/index
- https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/index?tabs=aspnetcore2x
- https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish?tabs=netcore2x
- https://docs.microsoft.com/en-us/dotnet/core/rid-catalog
### DOCKER
- Build containers:
- john@debian9John:~/Documents/raven/dist/docker/linux-x64$ docker-compose build
- Run it:
- :~/Documents/raven/dist/docker/linux-x64$ docker-compose up -d
- Build it in prep for running it:
- dotnet publish -o C:\data\code\raven\dist\docker\linux-x64\ayanovadocker\files\ -c Release
- john@debian9John:~/Documents/raven/server/AyaNova$ dotnet publish -o ~/Documents/raven/dist/docker/linux-x64/ayanovadocker/files -c Release
- OPTIONAL SAVING IMAGES (probably will never use this again but keeping for the info)
- Save image:
- docker image save -o .\image\ay-alpha2 gztw/ayanova
- Note: if you use a tag name or repo name it's preserved but if you use an image id it loses the tags
- Not compressed, can be compressed about 60% smaller
- Load image:
- docker image load -i saved_image_file_name_here
####
- Running docker at our D.O. server
- run AyaNova container FIRST sudo docker-compose up -d at ~/xfer/
- To update:
- run a publish command to publish to my local dist/linux-x64/ayanovadocker/files
- Then use Filezilla to copy up to the server at ~/xfer/ayanovadocker/files
- Optionally, update the ~/xfer/docker-compose to set a new version number for the image name ("alpha-5" etc or maybe remove the name in future)
- If necessary do a docker-compose build to rebuild
- run Nginx server:
- from /docker/letsencrypt-docker-nginx/src/production run sudo docker-compose up -d
- If necessary can switch to root with command: sudo su -
- documented here: https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
## TESTING
- DATA SEEDING: https://github.com/bchavez/Bogus (a port of faker.js)
### DOCKER NGINX Let's Encrypt CERTBOT
- https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
- https://github.com/humankode/letsencrypt-docker-nginx/blob/master/src/production/production.conf
INITIALLY FETCH CERTIFICATES (MUST START LETSENCRYPT NGINX CONTAINER FIRST AND STOP ALL OTHERS)
#### STAGING
sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
certbot/certbot \
certonly --webroot \
--email support@ayanova.com --agree-tos --no-eff-email \
--webroot-path=/data/letsencrypt \
--staging \
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
#### PRODUCTION
sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
-v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
certbot/certbot \
certonly --webroot \
--email support@ayanova.com --agree-tos --no-eff-email \
--webroot-path=/data/letsencrypt \
-d helloayanova.com -d www.helloayanova.com -d v8.helloayanova.com -d test.helloayanova.com
#### SAMPLE OUTPUT:
john@ubuntu-s-1vcpu-1gb-sfo2-01:/docker/letsencrypt-docker-nginx/src/letsencrypt$ sudo docker run -it --rm \
> -v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
> -v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
> -v /docker/letsencrypt-docker-nginx/src/letsencrypt/letsencrypt-site:/data/letsencrypt \
> -v "/docker-volumes/var/log/letsencrypt:/var/log/letsencrypt" \
> certbot/certbot \
> certonly --webroot \
> --email support@ayanova.com --agree-tos --no-eff-email \
> --webroot-path=/data/letsencrypt \
> -d helloayanova.com -d www.helloayanova.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for helloayanova.com
http-01 challenge for www.helloayanova.com
Using the webroot path /data/letsencrypt for all unmatched domains.
Waiting for verification...
Cleaning up challenges
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/helloayanova.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/helloayanova.com/privkey.pem
Your cert will expire on 2018-06-10. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew *all* of your certificates, run
"certbot renew"
- Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
=-=-=-=-=-=-=-=-
GRAFANA / INFLUXDB / DOCKER
Container to run the whole shebang:
- https://github.com/philhawthorne/docker-influxdb-grafana
docker run -d \
--name docker-influxdb-grafana \
-p 3003:3003 \
-p 3004:8083 \
-p 8086:8086 \
-p 22022:22 \
-v /path/for/influxdb:/var/lib/influxdb \
-v /path/for/grafana:/var/lib/grafana \
philhawthorne/docker-influxdb-grafana:latest
NOTE: you can leave out the paths and it works and the name is a little verbose
Dashboard for Grafana and app.metrics:
- https://grafana.com/dashboards/2125
MSBUILD reference for csproj file
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild#BKMK_ProjectFile
LARGE FILE GENERATION
=-=-=-=-=-=-=-=-=-=-=
Quickly generate large files in windows: http://tweaks.com/windows/62755/quickly-generate-large-test-files-in-windows/
Never download another 100mb test file or waste time searching for a large file. Sometimes you need a large file fast to test data transfers or disk performance. Windows includes a utility that allows you to quickly generate a file of any size instantly.
Open an administrative level command prompt.
Run the following command:
fsutil file createnew <file> <size in bytes>
For example, this command will create a 1GB file called 1gb.test on my desktop:
fsutil file createnew c:\users\steve\desktop\1gb.test 1073741824
The key is to input the size of the file in bytes so here are some common file sizes to save you from math:
1 MB = 1048576 bytes
100 MB = 104857600 bytes
1 GB = 1073741824 bytes
10 GB = 10737418240 bytes ** Takes just under 300 seconds on the same machine to migrate a 10gb assigneddoc file to the v8 server 2021-08-30 beta 128
100 GB =107374182400 bytes
1 TB = 1099511627776 bytes
10 TB =10995116277760 bytes
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
//DATA TYPES .net to postgres map
//http://www.npgsql.org/doc/types/basic.html
ICON GENERATOR
=-=-=-=-=-=-=-=-
- Used to generate pwa icons from source svg: http://cthedot.de/icongen/
seems to be broken now in 2020
- 2020 https://realfavicongenerator.net/
- 2020 LOTS OF APPLE STUFF https://github.com/onderceylan/pwa-asset-generator
=-=-=-=-=-=-=-=-=-=-
POSTGRES SQL PORTABLE:
https://github.com/rsubr/postgresql-portable/releases/tag/v12.1
Also this:
https://www.postgresonline.com/journal/archives/172-Starting-PostgreSQL-in-windows-without-install.html
HOW I DO IT FOR WINDOWS
=-=-=-=-=-=-=-=-=-=-=-=-
See win-x64 folder structure.
Basically download the binary for w64, extract and keep only the folders bin, lib and share
in the win-x64 data folder need to init a new database using this command:
..\postgres\bin\initdb -U postgres -D .\database
That will init a new fresh db with postgres user, no password and in a folder called database to sit beside the other data folders being disted (backup files, logs, user files)
DOCKER POSTGRES DEVELOPMENT STATION COMMANDS AND USEFUL INFO
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
OFFICIAL DOCKER POSTGRES IMAGE
https://hub.docker.com/_/postgres/
DEV SETUP
Using two containers dock-pg10 and dock-pgadmin
Command to list them all:
docker container ls
Command to see the detailed configuration of them:
docker container inspect [CONTAINER_NAME OR ID]
They use a volume in docker called "raven"
C:\data\code\raven\tools>docker volume inspect raven
[
{
"CreatedAt": "2019-05-09T17:03:34Z",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/raven/_data",
"Name": "raven",
"Options": {},
"Scope": "local"
}
]
To list volumes:
docker volume ls
IMAGES
=-=-=-
dock-pg10 uses the official "postgres" docker image
dock-pgadmin uses "chorss/docker-pgadmin4"
USEFUL COMMANDS
----------------
DUMPING POSTGRES DATA OUT OF IMAGE:
docker exec dock-pg10 pg_dumpall -U postgres > dump.sql
START A PG SERVER CONTAINER:
TEMPORARILY: docker run --rm --name pg -p 5432:5432 -e POSTGRES_PASSWORD=raven -e POSTGRES_DB=AyaNova -d postgres
(this will remove it after it closes with the rm switch and the rest is the requirements for ayanova, no idea where the data goes and it won't be kept AFAIK)
FANCY WITH VOLUME: docker run --name pg11 -e POSTGRES_PASSWORD=raven -d -p 5432:5432 -v /var/lib/docker/volumes/raven/_data:/var/lib/postgresql/data postgres
STATUS:
Can view the status of all containers with
docker ps -a
PGADMIN
=-=-=-=-
Official docker hub pgadmin 4 image:
https://hub.docker.com/r/dpage/pgadmin4/
RUN IT:
docker run --rm -p 5050:80 --link pg -e "PGADMIN_DEFAULT_EMAIL=support@ayanova.com" -e "PGADMIN_DEFAULT_PASSWORD=abraxis" -d dpage/pgadmin4
Browse to the site either localhost or if a port specified then another port
Login using support@ayanova.com and abraxis
Add the server by host name of pg (this is the pg container name and also the --link parameter in the pgadmin run command)
Use the credentials specified for postgres which is username postgres and password raven (specified on command line for running pg container)
TRANSLATIONS / TRANSLATE / LOCALIZATION / LOCALE
Microsoft handy dandy product translation search database site: https://www.microsoft.com/en-us/language/Search
Google translate to verify: https://translate.google.com/
POSTGRES STUFF:
Mass generate data:
query: insert into ametriccpu (t,v) select CURRENT_TIMESTAMP, 58.43239007949476 from generate_series(1, 525600) s(i)
insert into ametriccpu (
t,v1,v2,v3,v4,v5,v6,v7,v8,v9,v10
)
select
LOCALTIMESTAMP,
58.43239007949476,
0.33006058073955513,
102.44723488288768,
46.078341513002755,
30.23570573933185,
0.000136518543824419,
65.8400891412282,
0.01,
58.43239007949476,
58.43239007949476
from generate_series(1, 525600) s(i)
JAVASCRIPT DOWNSAMPLING TOOL
//====================
//https://github.com/joshcarr/largest-triangle-three-buckets.js/blob/master/lib/largest-triangle-three-buckets.js
'use strict';
function largestTriangleThreeBuckets( data, threshold, xAccessor, yAccessor ) {
var floor = Math.floor,
abs = Math.abs,
dataLength = data.length,
sampled = [],
sampledIndex = 0,
every = ( dataLength - 2 ) / ( threshold - 2 ), // Bucket size. Leave room for start and end data points
a = 0, // Initially a is the first point in the triangle
maxAreaPoint,
maxArea,
area,
nextA,
i,
avgX = 0,
avgY = 0,
avgRangeStart,
avgRangeEnd,
avgRangeLength,
rangeOffs,
rangeTo,
pointAX,
pointAY;
if ( threshold >= dataLength || threshold === 0 ) {
return data; // Nothing to do
}
sampled[ sampledIndex++ ] = data[ a ]; // Always add the first point
for ( i = 0; i < threshold - 2; i++ ) {
// Calculate point average for next bucket (containing c)
avgX = 0;
avgY = 0;
avgRangeStart = floor( ( i + 1 ) * every ) + 1;
avgRangeEnd = floor( ( i + 2 ) * every ) + 1;
avgRangeEnd = avgRangeEnd < dataLength ? avgRangeEnd : dataLength;
avgRangeLength = avgRangeEnd - avgRangeStart;
for ( ; avgRangeStart < avgRangeEnd; avgRangeStart++ ) {
avgX += data[ avgRangeStart ][ xAccessor ] * 1; // * 1 enforces Number (value may be Date)
avgY += data[ avgRangeStart ][ yAccessor ] * 1;
}
avgX /= avgRangeLength;
avgY /= avgRangeLength;
// Get the range for this bucket
rangeOffs = floor( ( i + 0 ) * every ) + 1;
rangeTo = floor( ( i + 1 ) * every ) + 1;
// Point a
pointAX = data[ a ][ xAccessor ] * 1; // enforce Number (value may be Date)
pointAY = data[ a ][ yAccessor ] * 1;
maxArea = area = -1;
for ( ; rangeOffs < rangeTo; rangeOffs++ ) {
// Calculate triangle area over three buckets
area = abs( ( pointAX - avgX ) * ( data[ rangeOffs ][ yAccessor ] - pointAY ) -
( pointAX - data[ rangeOffs ][ xAccessor ] ) * ( avgY - pointAY )
) * 0.5;
if ( area > maxArea ) {
maxArea = area;
maxAreaPoint = data[ rangeOffs ];
nextA = rangeOffs; // Next a is this b
}
}
sampled[ sampledIndex++ ] = maxAreaPoint; // Pick this point from the bucket
a = nextA; // This a is the next a (chosen b)
}
sampled[ sampledIndex++ ] = data[ dataLength - 1 ]; // Always add last
return sampled;
}
//=============================================
COlors colours for charts
http://perceptualedge.com/articles/b-eye/choosing_colors.pdf
MiniProfiler how to access stats from code
// This is how to access the stored profiler stats if I ever need to
// var profiler = MiniProfiler.StartNew("My Profiler Name");
// if (profiler != null)
// {
// var Options = profiler.Options;
// var guids = Options.Storage.List(100);
// // var lastId = context.Request["last-id"];
// // if (!lastId.IsNullOrWhiteSpace() && Guid.TryParse(lastId, out var lastGuid))
// // {
// // guids = guids.TakeWhile(g => g != lastGuid);
// // }
// var ministats = guids.Reverse()
// .Select(g => Options.Storage.Load(g))
// .Where(p => p != null)
// .Select(p => new
// {
// p.Id,
// p.Name,
// p.ClientTimings,
// p.Started,
// p.HasUserViewed,
// p.MachineName,
// p.User,
// p.DurationMilliseconds
// }).ToList();
// if(ministats.Count>0){
// var v=ministats.Count;
// }
// }
MEMORY LEAK DIAGNOSIS URLS
https://docs.microsoft.com/en-us/dotnet/core/diagnostics/debug-memory-leak
https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-counters
https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump
https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-trace
Might be doing GC Counts incorrectly, they don't sync up with the dotnet-counters tool which shows them drop back to zero regularly when I stop burn test
https://michaelscodingspot.com/ways-to-cause-memory-leaks-in-dotnet/
https://bennettadelson.wordpress.com/2013/04/11/using-perfview-to-diagnose-a-net-memory-leak-2/
HOW TO SEE WHO CALLED A METHOD DURING RUNTIME
keywords: caller, call method stack trace stacktrace
//System.Diagnostics.Debug.WriteLine((new System.Diagnostics.StackTrace()).GetFrame(1).GetMethod().Name);
//System.Diagnostics.Debug.WriteLine((new System.Diagnostics.StackTrace()).ToString().Substring(56,80));
Profiling / perf testing / performance testing
How to peformance test
get process id for AyaNova
C:\temp\xfer\perf>dotnet-trace ps
12048 AyaNova C:\temp\xfer\v8test\win-x64\ayanova\AyaNova.exe
collect data with dotnet trace:
dotnet-trace collect --process-id 12048 --format Speedscope
Use speedscope or built in to Chrome flame graph
https://www.hanselman.com/blog/dotnettrace-for-net-core-tracing-in-perfview-speedscope-chromium-event-trace-profiling-flame-graphs-and-more
how to read a flame graph:
https://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html
Reference in New Issue View Git Blame Copy Permalink