# SERVER TODO (J.F.C. - Just fucking code it already)

Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOiIxNTQ3NTgwMzg2IiwiZXhwIjoiMTU1MDE3MjM4NiIsImlzcyI6ImF5YW5vdmEuY29tIiwiaWQiOiIxIiwiYXlhbm92YS9yb2xlcyI6IjMyNzY3In0.duYSuTC2v9dCD7POPTvRNB2B4tGCkNQ1UDkgaSrNSYo


## IMMEDIATE ITEMS

		

Need a sprint to get to a fully testable client with entry form, list and as much as possible all features from COMMON-* specs list 
Do the stuff in the Client todo first then back to the server as required.

-----------------------

DO CLIENT STUFF NOW COME BACK TO THIS STUFF LATER

 ### SERVER WORK NEEDED FIRST....
			- NOT done correctly at the server NEED TO CHANGE THIS SHIT FIRST:
				- is returning a 401 (not authenticated) for rights issues that should return 403 (not authorized)
				- Before can do below rights stuff need to go back to server and change that
				- https://stackoverflow.com/questions/3297048/403-forbidden-vs-401-unauthorized-http-responses#6937030
				- I know it works when the user SubContractorLimited logs in and force to fetch widget gets a 403 instead of a 401 and instead of logging off redirects to home or back or something instead
				- in GZAPI handleError has this: ErrorUserNotAuthenticated  error string, I also need to check server and docs for the corresponding ErrorUserNotAuthorized which may need to be added and documented
					- Also need a localized text for it in all languages apparently and also document it properly and add it as a type of error returned in those circumstances
				- May be faster to just try to fetch the object and have rights checked that way and react accordingly in the client rather than try to pre-check before hand
					- This is because need the actual object to check if self owned and can still edit, let the server handle that shit and just act accordingly
				- If server returns a read only copy of an object due to read full record but not due to allow edit then perhaps the server can also tag it with a READONLY flag so client can adjust accordingly and not need to do the checking with a double request
				- SO...SERVER Should return on request of an object one of these:
					- Not authenticated at all 401
					- Not authorized for this object 403 (could be due to not own or whatever, we don't care, server handles that shit, client just knows not to show it)
					- Object...BUT with READONLY flag of some kind present (in outer wrapper??), so client knows to show read only and not allow editing
					- Object without readonly flag present so fully editable!!! WOOT!
	- FINDINGS / TODO
		- So early on the server is returning 401 not authenticated before the route is hit, so really, in any route my own rights check would always be 403, not 401, that's handled already by authentication middleware



### TODO FILTER ISSUE: SEE integration tests search for BUGBUG there are two tests failing in DataFilterFilteringLists tests that might be DST related

=-=-=-=-=-=-
Less urgent things...

DOCS / MANUAL 
	- Disable google fonts for manual / docs generator:
		- first need to upgrade to latest mkdocs AND material theme for mkdocs
			- https://www.mkdocs.org/#installation
			- https://github.com/squidfunk/mkdocs-material
		- https://squidfunk.github.io/mkdocs-material/compliance/
	- Once docs no longer need google fonts can adjust the devops nginx config and remove the overrides for CORS and all that shit.

DEVOPS 
	- Right now all that really matters is being able to test it and I can do that so the rest can be bumped.
		- Wipe the server, create new at v 18 (or try the upgrade first just for kicks)
			- Upgrade in place process: https://www.digitalocean.com/community/tutorials/how-to-upgrade-to-ubuntu-18-04
		- Make devops AyaNova barebones without docker?
			- Look into pros and cons because ultimately we'll want to run a bunch, maybe on demand, how to do that automatically? (Cadillac problem)
				- Actually, won't they just be separate droplets?  So really what I want is a very efficient droplet that I can clone?
				- But then people want to just sign up and start working, not rent a whole server etc, it's up to us where we put the droplets or apps, but they don't want to see that
				- Several scenarios: 
					- User signs up and starts working immediately, all through a subdomain, i.e. https://abcelectric.ayanova.com 
					- User wants to host it themselves on their equipment (nothing for us to do but sell the license)
					- User wants their own host and domain i.e. https://abcelectric.com
					- User wants to run it all locally on their own computer, docker image?  Do we really want a windows installer route or...?

			- initially probably manually set up so really not an affect either way as will just need to dedicate a port to each one regardless if dockerized or not
			- We do want fastest perf though and no aggravation or cross exposure





- WIKI PAGE
		- Doesn't require docs support as is now changed to a standard file attachment


- SERVER landing page
	- I know TTM and all that but the server landing page should look nicer.  Maybe a logo and better layout or something.

- If the Database connection is lost while AyaNova is running there are a lot of errors triggered by the generator etc.
	- It should more gracefully handle this by perhaps closing down the api and generator should respect a closed API
		- Set the reason to "Lost databases connection"

=-=-=-
some shit I probably don't need anymore:

CHOPPY DAY WORK
+++++++++++++++

https://severalnines.com/blog/performance-cheat-sheet-postgresql
https://medium.com/@wesharehoodies/immensely-upgrade-your-development-environment-with-these-visual-studio-code-extensions-9cd790478530
Changes needed to routes??
	- http://www.talkingdotnet.com/actionresult-t-asp-net-core-2-1/

Ensure all modern best practice security is properly enabled on helloayanova.com so testing is valid
	- https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security#Deployment_best_practices



CODING
++++++
refs:
	- https://vuetifyjs.com/en/components/navigation-drawers
	- https://vuetifyjs.com/en/examples/layouts/googleContacts
	- Read this: https://qiita.com/nickhall/items/d1043f3f9874c90b6f8e
	- And this: https://logrocket.com/blog/an-imperative-guide-to-forms-in-vue-js-2/



// eslint-disable-next-line

NEXT UP / CURRENTLY WORKING ON:
=-=-=-

IMMEDIATE ITEMS:
================


*** CLIENT SHELL ***

Go through the TOOLS.txt file VUE section and install

ServiceWorker - learn it, know it, love it
	- Make Pecklist work with a service worker for offline use!!!!
		- Do it "manually" to learn about SW better
		- Use the VUE CLI generated stuff as a base

Do some tutorials to get a feel, read all the following 
VUE PWA 
	- https://naturaily.com/blog/pwa-vue-cli-3
	- https://dev.to/developerayo_58/vue-pwa-tutorial-how-to-build-efficient-progressive-web-apps-using-vuejs-5fdo

UI LIBS
	

OTHER
	- Think about spa features
	- Webpack (how can it help, what parts to use)
	- Read about Single File Components https://vuejs.org/v2/guide/single-file-components.html
	- Skim and have at hand ES2015 guide: https://babeljs.io/docs/en/learn




Once I can make the client I need to get into that and make the shell and initial interface with enough stuff to do basic testing initially
	- Make sure to see the vue.js stuff in tools and below and contemplate it fully before committing to it
		- VUE was chosen some time ago and there are likely other things out now
	- Look and layout, graphics, logo, anything that is shell only 
	- Menu system	
		- Help link
		- search 
	- Login , logout
	- License
	- Security / rights
	- See localized text / change locale 

	later
	- Widget CRUD and lists
		- All input controls and date localization etc etc
	- LOAD TESTER see below...








************************************************************************************
FUTURE ITEMS:
=============




LOAD TESTER
-----------
After very initial client stuff but before getting too deep into anything, need a load tester that drives the client and makes hundreds of thousands of objects and really exercises the client
and backend.  I want to know what happens under typical heavy usage simulating months to years of work in just a few minutes.
I'm guessing something that autodrives a browser or simulates one to access via client UI for full end to end testing.
	- Should be configurable to simulate various loads:
		- Smoke test (Quick bug test): excercises one of everything once only looking for fundamental problems
		- Load / scale / assurance test (Slower longer test to look for scaling issues related to a huge, busy office): exercises typical daily load of very large office but configurable to run in simulated days, so
			can run it as a simulated week with 7 or month with 30 or years by entering in the number of days to simulate.  
			Would be nice if it did month end type stuff or delayed then billed out etc.
			Should report stats about timing etc from the client perspective, we already have metrics for the backend perspective.
				I'd like to know immediately if something is weirdly slow, not just in error.
			Should not erase data at end so we can then manually login and check things out with a full load of at scale data


MISC FUTURE ITEMS NOT SURE ABOUT THAT CAME UP DURING CODING / TESTING
---------------------------------------------------------------------

- Should server show uptime somewhere?
- Log route: Add a route to download *all* log files as a single zip archive which will make life much easier for analysis





### ALL ITEMS

	
	- LOCALIZED TEXT
		- Localized text keys would be nice if they are understandable as is for API direct users so you don't need to be in the client to understand
			what's happening
		- Need a locale indepedent locale so that server errors without a corresponding user are localized to default english
			- Also maybe a locale can be chosen at the server for error messages since we'll go by code numbers anyway.
		- Starting to get to the point where I'll need this, i.e. error messages and logs that are part of core ops but need to be displayed in the UI
		- Need to go through the api and find all the plain text messages returned and convert to locale text keys
			- Ensure every error message has an error number of one kind or another and that they are not conflicted and easy to sort out if coming from server or api or etc
		- Need to suck out our paid for translations and convert them into new locale text format
			- DataDump??

	- REPORTING 
		- Huge and unknown at this point Oct. 2018

	- SEARCH TEXT
		- See spec docs
	
	- TECH SUPPORT
		- Investigate how I can look at a customers DB with RAVEN
		- Data masking for dumps is a start
			- special "tech support" dump with masked customer information??
			- Customer has a "key" that they can see which customer is the substituted masked one so we don't know the customer name but they can reference it themselves
		- What about live looking at data through some feature?
			- What would I need to look at or what information would I need?
			- Be able to run a query directly and view results??
		- Be able to run a query provided
			- As a fix might need to enable customer to run a provided query.
		- Be able to view all the meta information about the postgres instance
			- Collation, sort order, languages, anything the user can set that could fuck up RAVEN
	

	- MODIFICATION / CHANGE LOG (see case 79)
	
	- Visible ID number generator case 3544
	
	

	
	
	- Notification / generator / event of interest stuff (case 3491) BIG ONE
		- Need interface, code for triggering notifications in biz objects ITriggerable :)
		- Would it be more efficient to just process all notifications into the modification log regardless of subscribers	

	- CHILD objects need to point to their parent and be readable in code for searching and for opening objects based on child object
		- so all child objects need a typeandid of the immediate parent
		- Not sure where to document this so putting it here for reference
		- Required for opening a search result of a descendent that is not directly openable an
		- Need parent AyaType as an ENUM ATTRIBUTE in the AyaType table for easy traversal

	
	- Import V7
		- Tags - any type that is moving to tag can be coded now

	- TESTING
		- Longevity test on the DO server I can have up and running see core-testing.txt doc

CLIENT	
			
	- WHEN HAVE CLIENT - Localization (see core-localization.md) 
		- Time zone stuff (case 1912 related)
	- WHEN HAVE CLIENT - Layout / Form user setttings
	- WHEN HAVE CLIENT - Default form filling settings handling (case 3485)
	- WHEN HAVE CLIENT - Push notification to client
		- PUSH / POLL notification: determine and implement a system that can send notifications to client for things like 
		- change of localized text (invalidate cache)
		- Server shutting down (log out asap)
		- business object notifications (new workorder, status change etc)
	- WHEN HAVE CLIENT Report route for widget
	- WHEN HAVE CLIENT - test my PickList, is it sufficient? (pageable, alpha pageable (A-D, E-G kind of thing?))
		- Case 1692 
		- search by tags plus text plus maybe pageable or...???
	- WHEN HAVE CLIENT - ACTION / UI WIDGETS case 3460, 1729 UI as a collection of widgets stuff


- When widget is completely done, go over it and see if anything can be made easier or better before proceeding
- Generate seed data for tags
- Better to do this when the above core items are done as it touches on them
- Time zone	
	- This is not specced anywhere, but here, not sure where to put it at the moment, hopefully by the time I get here I will know (global settings? User settings?)
	- Do not rely on the server's time zone setting, for example a docker container will be utc even if the server hosting it is pacific time
	- Instead, use UTC for everything and have configurable value for timezone offset

### NFR	

	- UPDATE SWASHBUCKLE / SWAGGER to support testing file upload if not too onerous
		- https://github.com/domaindrivendev/Swashbuckle/issues/280
		- http://www.talkingdotnet.com/how-to-upload-file-via-swagger-in-asp-net-core-web-api/

	- IMPORT / EXPORT
		- biz object should import from v7, make a dummy import for widgets from something I've already exported in v7 (units? something with at least a name)	
		- Make a route for import to upload an import file?  Then it runs the import via the biz objects and the correct ordering?
		- OPS Functionality?? (maybe just biz admin only since it's dealing with actual biz data)
		- OPS can import ops related stuff??(notification email server settings etc)

	- BACKUP and RESTORE and COPY automatically to storage offsite
		- Backup and restore data (widget, users etc)
		- Close AyaNova server, erase db(optionally?? maybe user wants to combine two separate db's), restore the data
		- See Discourse, they have some kind of AWS thing
		- Also maybe this is handy: http://www.talkingdotnet.com/webhooks-with-asp-net-core-dropbox-and-github/
		- Download backup, upload backup file
		- FTP automatically?

	
	- Need api speed test route (to independently of any particular object know how fast the connection is, for choosing a host site and troubleshooting)
		- Some kind of static test list that is perfectly reproducible on demand
		- Maybe a set routine of items to generate and return but in a way to disambiguate between slow server and slow connection
		- don't re-invent the wheel
		- What to test:
			- compute performance
			- DB performance
			- thoughput? Speed of network

	- Need some way to know if AyaNova is taking longer than it should to process requests so it can be an alert of some kind
		- research how to time api avg running total or something, graph it for ops
		- keep data by class of operation or tag it somehow
		- Don't want it to actually slow performance
		- Maybe have a benchmark time for various ops gathered during debugging tests, then hard code in that benchmark and if it takes longer then it logs it
		- http://www.neekgreen.com/2017/11/06/easy-way-measure-execution-time-aspnetcore-action-method/
		- https://weblogs.asp.net/jeff/asp-net-core-middleware-to-measure-request-processing-time


- SSL / TLS
	- Need to look into how to support this
	- Look into how the 2.1 dotnet will work with ssl so I do something relatively compatible 


- REPORTING
	- NOTE TO SELF: Don't report off Biz objects, make report specific objects. Better to have a reportclient list object and a selection client list object and etc than just a single client list doing duty as a selection box filler and a reporting object
	And also biz object interfaces ideas:
	ITaggable, ICustomFields, ISearchable, IExportable,IBizAction, IReportable (with sub interfaces for paging, format, report name and biz object for single and list etc,report stuff), ILocaleFields?, Etc
	Your welcome!🤘😎


- CLIENT / UI DEVELOPMENT
	- CLIENT UI "WIDGETS" ("COMPONENTS")
	- Have UI testing scripts for developing UI. Scenario and then I can manually walk through it and see how ui responds to iterate from rough skeletal UI.
	- Make a script for top X scenarios in the work day if each role.  That way can try early rough designs with neutral expectations and reiterate until adequate.
	- This way I won't design out of my ass without good input to riff off of.
	- Orient express is some good shit for the shabs!


- Client: Start initial front end vue.js shell
	- Need way to shut down clients gracefully (added value in api return? Polling [can't recall what the decision was in polling])

- Implement unlicensed server mode in client
	- Request trial key from client, server fetches and installs

- Implement trial mode in client 
	- Seed data, erase db etc


- AFTER APRIL 1st 2018 - Dotnet 2.1 changes I must look into:
	- will be rtm this summer-ish
	- some swagger and webapi affecting changes
	- efcore group by and lazy loading thing, might be relevant, nto sure
	- HTTPS by default
	- Look into it, see if something will be so huge that I should use the beta now for dev.



BUNDLING
- ONCE there is any front end code worthwhile then - Automatic build process Bundling and minification
- Parcel is coming on strong and requires supposedly zero configuration: https://parceljs.org/getting_started.html
- READ THIS: https://docs.microsoft.com/en-us/aspnet/core/client-side/using-gulp
- THEN SET IT UP
- Need automatic file copy script or whatever to copy docs to wwwroot folder somewhere so it can be served by the ayanova server
- Also need to package front end stuff for deployment as well with versioning etc, not webpack but along those lines




- Think about hostname being included with license, maybe a requirement?
	- Localhost only or domain?
	- Or would it be too much hassle with non-domain sites
- Fail2ban?  Will we need that kind of thing incorporated into AyaNova?
	- See again how it works and then look into application level ideas for that or what is smart for hardening, throttling etc

- 2FA Two factor authentication
	- How hard is this to support in AyaNova?
	- What about apps like Authy?
	- It might be important to enable this for ops and biz accounts? Or at least be an option?

- LETS ENCRYPT
	- https://www.humankode.com/ssl/how-to-set-up-free-ssl-certificates-from-lets-encrypt-using-docker-and-nginx
	- https://weblog.west-wind.com/posts/2017/Sep/09/Configuring-LetsEncrypt-for-ASPNET-Core-and-IIS
	- https://stackoverflow.com/questions/48272373/how-do-i-get-letsencrypt-working-in-asp-net-core-razor-pages
	- NGINX: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04
		- Review again if need NGINX in front of kestrel still and if so then go this route




- DO WE NEED TO BE ABLE TO admin db from within raven ops route even if can't connect to db?
	- Don't want users to have to use a db admin tool for anything, so should have ability to do whatever is necessary from ops route with db
	- REQUIRED OPS:  See if db exists


DEPLOYMENT AND TESTING
- DOCKER As soon as viable make an automatic build to a docker image for testing and deployment
	- https://docs.microsoft.com/en-us/dotnet/core/docker/building-net-docker-images
	- remote server online testing
		- Better product will come from running it as it will be used as early and often as possible
		- Look into renting the cheapest server on linode or digital ocean for dev testing
		- set it up to pull the latest from repo so it automatically updates (or a docker image maybe)
		- possibly set up integration test that goes off the remote server 
- WINDOWS Automatic installer for testing
	- Need a windows test bed and regular testing on it to confirm multiplatform interoperability
	- Maybe a windows installer or maybe a docker image

- Integration test that can be pointed at any location to run a series of tests



MANUAL
- Add how to use swagger UI and authentication


BOOTSTRAPPING AUTHENTICATION

- Manager account can only login from localhost? - HMM...think on it

- What if can only create new users if manager account is changed from default credentials?
	- that way you start your setup with one account, change it and then it's safe to do remote work


- by default manager account is only one with rights to configure server or user accounts.
	- Doesn't have any rights to business config, only server config and CRUD user accounts
	- So at least one admin user needs to be created locallhy before it can be used remotely to set up users
	- manager account cannot be changed in any way, so always has default password and login
	- JWT token check must check if local when it's the manager account user id 1
		- This is so a user can't copy the creds from browser and use them remotely
		- Test that shit from host
- What if can't run a browser in host for some reason???
 	- need an override that does allow remote manager account

- devise a way to bootstrap with no user accounts and a way to reset back to that




MAKE MVP 

- Has the following features:
	Alpha-0
	- installer for windows and docker container
	- VUE.js Front end that supports at minimum a login / logout and empty shell
		- Shows server and client versions (about)
		- https://vuejs.github.io/vetur/
	- swagger docs and way to view them via the api
	- User manual docs
	- See the tools.txt section search for vue
	
	Alpha-1
	- Can do some minimal config like seed data, erase db etc 
	- Ops interface showing status and can view log etc

	etc



LONG TERM::::::

## MVP and iterate

