raven/devdocs/todo.txt

# TODO (J.F.C. - Just fucking code it already)

Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOiIxNTM1NDc4Mzc5IiwiZXhwIjoiMTUzODA3MDM3OSIsImlzcyI6IkF5YU5vdmEiLCJpZCI6IjEifQ.VP0amo0xXRaXrMSwkiowjJh-u3tWJVnxvzB5PxJIeiM


## IMMEDIATE ITEMS

CHOPPY DAY WORK
+++++++++++++++

https://severalnines.com/blog/performance-cheat-sheet-postgresql
https://medium.com/@wesharehoodies/immensely-upgrade-your-development-environment-with-these-visual-studio-code-extensions-9cd790478530
Changes needed to routes??
	- http://www.talkingdotnet.com/actionresult-t-asp-net-core-2-1/

	

CODING WORK
+++++++++++

CURRENT DEVELOPMENT OBJECTIVES: 

CLIENT SHELL: anything standing in the way of making the initial client shell UI needs to be done first, everything else can wait
UI Shell is required for driving development processes of backend, once have client can make up initial test forms and then determine best way to write backend biz objects.
Once that is done then can steam ahead on the biz objects but until I have the client I won't know the best way to code them so to avoid re-working shit do it in this order.


IMMEDIATE ITEMS:
================
	
	
	- Tag groups (modify tags already coded)
		- X Modify tag to bring combined list of groups with tag picklist (NOPE, let the UI query both separately and integrate them at the UI level)
		- Modify tag to remove itself from tagroupmap if deleted
	- Retag: code to tag an item should take into account if it's already tagged and not make a new record, just return existing
	- Localized text
		- ** DEVISE a system to ensure no unused keys are brought forward to raven
	- Search and search text indexing
		- Add to widget tests
	- Auto visible id number assigning code
		- Give widgets a visible ID number scheme and add to tests
	

	

Ensure all modern best practice security is properly enabled on helloayanova.com so testing is valid
	- https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security#Deployment_best_practices


************************************************************************************
FUTURE ITEMS:
=============

CLIENT SHELL

Once I can make the client I need to get into that and make the shell and initial interface with enough stuff to do basic testing initially
	- Make sure to see the vue.js stuff in tools and below and contemplate it fully before committing to it
		- VUE was chosen some time ago and there are likely other things out now
	- Look and layout, graphics, logo, anything that is shell only 
	- Menu system	
		- Help link
		- search 
	- Login , logout
	- License
	- Security / rights
	- See localized text / change locale 

	later
	- Widget CRUD and lists
		- All input controls and date localization etc etc
	- LOAD TESTER see below...

LOAD TESTER
-----------
After very initial client stuff but before getting too deep into anything, need a load tester that drives the client and makes hundreds of thousands of objects and really exercises the client
and backend.  I want to know what happens under typical heavy usage simulating months to years of work in just a few minutes.
I'm guessing something that autodrives a browser or simulates one to access via client UI for full end to end testing.
	- Should be configurable to simulate various loads:
		- Smoke test (Quick bug test): excercises one of everything once only looking for fundamental problems
		- Load / scale / assurance test (Slower longer test to look for scaling issues related to a huge, busy office): exercises typical daily load of very large office but configurable to run in simulated days, so
			can run it as a simulated week with 7 or month with 30 or years by entering in the number of days to simulate.  
			Would be nice if it did month end type stuff or delayed then billed out etc.
			Should report stats about timing etc from the client perspective, we already have metrics for the backend perspective.
				I'd like to know immediately if something is weirdly slow, not just in error.
			Should not erase data at end so we can then manually login and check things out with a full load of at scale data




### ALL ITEMS

	
	- LOCALIZED TEXT
		- Localized text keys would be nice if they are understandable as is for API direct users so you don't need to be in the client to understand
			what's happening
		- Need a locale indepedent locale so that server errors without a corresponding user are localized to default english
			- Also maybe a locale can be chosen at the server for error messages since we'll go by code numbers anyway.
		- Starting to get to the point where I'll need this, i.e. error messages and logs that are part of core ops but need to be displayed in the UI
		- Need to go through the api and find all the plain text messages returned and convert to locale text keys
			- Ensure every error message has an error number of one kind or another and that they are not conflicted and easy to sort out if coming from server or api or etc
		- Need to suck out our paid for translations and convert them into new locale text format
			- DataDump??

	

	- SEARCH TEXT
		- See spec docs
	
	- TECH SUPPORT
		- Investigate how I can look at a customers DB with RAVEN
		- Data masking for dumps is a start
			- special "tech support" dump with masked customer information??
			- Customer has a "key" that they can see which customer is the substituted masked one so we don't know the customer name but they can reference it themselves
		- What about live looking at data through some feature?
			- What would I need to look at or what information would I need?
			- Be able to run a query directly and view results??
		- Be able to run a query provided
			- As a fix might need to enable customer to run a provided query.
		- Be able to view all the meta information about the postgres instance
			- Collation, sort order, languages, anything the user can set that could fuck up RAVEN
	

	- MODIFICATION / CHANGE LOG (see case 79)
	
	- Visible ID number generator case 3544
	
	- CUSTOM FIELDS (case 3426)

	
	
	- Notification / generator / event of interest stuff (case 3491) BIG ONE
		- Need interface, code for triggering notifications in biz objects ITriggerable :)
		- Would it be more efficient to just process all notifications into the modification log regardless of subscribers	

	- CHILD objects need to point to their parent and be readable in code for searching and for opening objects based on child object
		- so all child objects need a typeandid of the immediate parent
		- Not sure where to document this so putting it here for reference
		- Required for opening a search result of a descendent that is not directly openable an
		- Need parent AyaType as an ENUM ATTRIBUTE in the AyaType table for easy traversal

	
	- Import V7
		- Tags - any type that is moving to tag can be coded now

	- TESTING
		- Longevity test on the DO server I can have up and running see core-testing.txt doc

CLIENT	
			
	- WHEN HAVE CLIENT - Localization (see core-localization.md) 
		- Time zone stuff (case 1912 related)
	- WHEN HAVE CLIENT - Layout / Form user setttings
	- WHEN HAVE CLIENT - Default form filling settings handling (case 3485)
	- WHEN HAVE CLIENT - Push notification to client
		- PUSH / POLL notification: determine and implement a system that can send notifications to client for things like 
		- change of localized text (invalidate cache)
		- Server shutting down (log out asap)
		- business object notifications (new workorder, status change etc)
	- WHEN HAVE CLIENT Report route for widget
	- WHEN HAVE CLIENT - test my PickList, is it sufficient? (pageable, alpha pageable (A-D, E-G kind of thing?))
		- Case 1692 
		- search by tags plus text plus maybe pageable or...???
	- WHEN HAVE CLIENT - ACTION / UI WIDGETS case 3460, 1729 UI as a collection of widgets stuff


- When widget is completely done, go over it and see if anything can be made easier or better before proceeding
- Generate seed data for tags
- Better to do this when the above core items are done as it touches on them
- Time zone	
	- This is not specced anywhere, but here, not sure where to put it at the moment, hopefully by the time I get here I will know (global settings? User settings?)
	- Do not rely on the server's time zone setting, for example a docker container will be utc even if the server hosting it is pacific time
	- Instead, use UTC for everything and have configurable value for timezone offset

### NFR	

	- UPDATE SWASHBUCKLE / SWAGGER to support testing file upload if not too onerous
		- https://github.com/domaindrivendev/Swashbuckle/issues/280
		- http://www.talkingdotnet.com/how-to-upload-file-via-swagger-in-asp-net-core-web-api/

	- IMPORT / EXPORT
		- biz object should import from v7, make a dummy import for widgets from something I've already exported in v7 (units? something with at least a name)	
		- Make a route for import to upload an import file?  Then it runs the import via the biz objects and the correct ordering?
		- OPS Functionality?? (maybe just biz admin only since it's dealing with actual biz data)
		- OPS can import ops related stuff??(notification email server settings etc)

	- BACKUP and RESTORE and COPY automatically to storage offsite
		- Backup and restore data (widget, users etc)
		- Close AyaNova server, erase db(optionally?? maybe user wants to combine two separate db's), restore the data
		- See Discourse, they have some kind of AWS thing
		- Also maybe this is handy: http://www.talkingdotnet.com/webhooks-with-asp-net-core-dropbox-and-github/
		- Download backup, upload backup file
		- FTP automatically?

	
	- Need api speed test route (to independently of any particular object know how fast the connection is, for choosing a host site and troubleshooting)
		- Some kind of static test list that is perfectly reproducible on demand
		- Maybe a set routine of items to generate and return but in a way to disambiguate between slow server and slow connection
		- don't re-invent the wheel
		- What to test:
			- compute performance
			- DB performance
			- thoughput? Speed of network

	- Need some way to know if AyaNova is taking longer than it should to process requests so it can be an alert of some kind
		- research how to time api avg running total or something, graph it for ops
		- keep data by class of operation or tag it somehow
		- Don't want it to actually slow performance
		- Maybe have a benchmark time for various ops gathered during debugging tests, then hard code in that benchmark and if it takes longer then it logs it
		- http://www.neekgreen.com/2017/11/06/easy-way-measure-execution-time-aspnetcore-action-method/
		- https://weblogs.asp.net/jeff/asp-net-core-middleware-to-measure-request-processing-time


- SSL / TLS
	- Need to look into how to support this
	- Look into how the 2.1 dotnet will work with ssl so I do something relatively compatible 


- REPORTING
	- NOTE TO SELF: Don't report off Biz objects, make report specific objects. Better to have a reportclient list object and a selection client list object and etc than just a single client list doing duty as a selection box filler and a reporting object
	And also biz object interfaces ideas:
	ITaggable, ICustomFields, ISearchable, IExportable,IBizAction, IReportable (with sub interfaces for paging, format, report name and biz object for single and list etc,report stuff), ILocaleFields?, Etc
	Your welcome!🤘😎


- CLIENT / UI DEVELOPMENT
	- CLIENT UI "WIDGETS" ("COMPONENTS")
	- Have UI testing scripts for developing UI. Scenario and then I can manually walk through it and see how ui responds to iterate from rough skeletal UI.
	- Make a script for top X scenarios in the work day if each role.  That way can try early rough designs with neutral expectations and reiterate until adequate.
	- This way I won't design out of my ass without good input to riff off of.
	- Orient express is some good shit for the shabs!


- Client: Start initial front end vue.js shell
	- Need way to shut down clients gracefully (added value in api return? Polling [can't recall what the decision was in polling])

- Implement unlicensed server mode in client
	- Request trial key from client, server fetches and installs

- Implement trial mode in client 
	- Seed data, erase db etc


- AFTER APRIL 1st 2018 - Dotnet 2.1 changes I must look into:
	- will be rtm this summer-ish
	- some swagger and webapi affecting changes
	- efcore group by and lazy loading thing, might be relevant, nto sure
	- HTTPS by default
	- Look into it, see if something will be so huge that I should use the beta now for dev.



BUNDLING
- ONCE there is any front end code worthwhile then - Automatic build process Bundling and minification
- Parcel is coming on strong and requires supposedly zero configuration: https://parceljs.org/getting_started.html
- READ THIS: https://docs.microsoft.com/en-us/aspnet/core/client-side/using-gulp
- THEN SET IT UP
- Need automatic file copy script or whatever to copy docs to wwwroot folder somewhere so it can be served by the ayanova server
- Also need to package front end stuff for deployment as well with versioning etc, not webpack but along those lines




- Think about hostname being included with license, maybe a requirement?
	- Localhost only or domain?
	- Or would it be too much hassle with non-domain sites
- Fail2ban?  Will we need that kind of thing incorporated into AyaNova?
	- See again how it works and then look into application level ideas for that or what is smart for hardening, throttling etc

- 2FA Two factor authentication
	- How hard is this to support in AyaNova?
	- What about apps like Authy?
	- It might be important to enable this for ops and biz accounts? Or at least be an option?

- LETS ENCRYPT
	- https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
	- https://weblog.west-wind.com/posts/2017/Sep/09/Configuring-LetsEncrypt-for-ASPNET-Core-and-IIS
	- https://stackoverflow.com/questions/48272373/how-do-i-get-letsencrypt-working-in-asp-net-core-razor-pages
	- NGINX: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04
		- Review again if need NGINX in front of kestrel still and if so then go this route




- DO WE NEED TO BE ABLE TO admin db from within raven ops route even if can't connect to db?
	- Don't want users to have to use a db admin tool for anything, so should have ability to do whatever is necessary from ops route with db
	- REQUIRED OPS:  See if db exists


DEPLOYMENT AND TESTING
- DOCKER As soon as viable make an automatic build to a docker image for testing and deployment
	- https://docs.microsoft.com/en-us/dotnet/core/docker/building-net-docker-images
	- remote server online testing
		- Better product will come from running it as it will be used as early and often as possible
		- Look into renting the cheapest server on linode or digital ocean for dev testing
		- set it up to pull the latest from repo so it automatically updates (or a docker image maybe)
		- possibly set up integration test that goes off the remote server 
- WINDOWS Automatic installer for testing
	- Need a windows test bed and regular testing on it to confirm multiplatform interoperability
	- Maybe a windows installer or maybe a docker image

- Integration test that can be pointed at any location to run a series of tests



MANUAL
- Add how to use swagger UI and authentication


BOOTSTRAPPING AUTHENTICATION

- Manager account can only login from localhost? - HMM...think on it

- What if can only create new users if manager account is changed from default credentials?
	- that way you start your setup with one account, change it and then it's safe to do remote work


- by default manager account is only one with rights to configure server or user accounts.
	- Doesn't have any rights to business config, only server config and CRUD user accounts
	- So at least one admin user needs to be created locallhy before it can be used remotely to set up users
	- manager account cannot be changed in any way, so always has default password and login
	- JWT token check must check if local when it's the manager account user id 1
		- This is so a user can't copy the creds from browser and use them remotely
		- Test that shit from host
- What if can't run a browser in host for some reason???
 	- need an override that does allow remote manager account

- devise a way to bootstrap with no user accounts and a way to reset back to that




MAKE MVP 

- Has the following features:
	Alpha-0
	- installer for windows and docker container
	- VUE.js Front end that supports at minimum a login / logout and empty shell
		- Shows server and client versions (about)
		- https://vuejs.github.io/vetur/
	- swagger docs and way to view them via the api
	- User manual docs
	- See the tools.txt section search for vue
	
	Alpha-1
	- Can do some minimal config like seed data, erase db etc 
	- Ops interface showing status and can view log etc

	etc



LONG TERM::::::

## MVP and iterate