@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@ ROADMAP STAGE 4 - REPORTING / DASHBOARD / KPI @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ todo: look at proper settings for rendering in docker on linux and try to make it resilient to crashing even if set too low or too little resources IDEAS: check guidelines for running in docker from link below check command line params for best fit see about re-using same or whatever check cpu, memory usage upon render request, reject immediately if the system is overburdened check for running chromium processes or some way to determine if in the middle of the last render for someone Try a method to zap all chromium processes as a test to see if they *can* be killed whil stuck this way To test use the "custom date time format helpers" with 5k records, that reliably freezes everything even when others all seem to run ok (this in itself is curious but whatever) What I'm seeing is on a long render it returns a 504 gateway timeout to the client but it's still churning away in the background 5k records and "custom date time format helpers" report will take about 3 minutes to render, maybe less and it will crash out for the client return 504 but then it will complete at some point Confirmed it *does* complete because I was able to download the 4.12mb 2144 page pdf manually from the server temp folder via filezilla once I saw the cpu go down again and chrome process stop 90% memory is the max used in do graph panel, probably docker is not letting it take all or something isn't maybe swapping out is what's happening, peak memory usage seems to be half a gig or so but c 97% cpu usage is max I'm thinking it's a memory issue more than a cpu issue because the cpu is hardly pegged at all while it's rendering right up until it appears to run out of memory then it starts swapping and all hell breaks loose and the cpu pegs on the swap daemon Need to test by moving up to 2gb of memory witha resize but keep the single vcpu and see what's what also increase the timeout in nginx for the reverse proxy to wait todo: take a look at front end cypress testing etc again, it's going to be the best way to assure quality going forward and indespensible for smoke / load testing todo: triage this stuff: todo: Client is *NOT* automatically updating no matter what version numbers are set Need to look into this and fix it for certain as it will be a huge issue when beta / release generally Find the notes already about this, maybe there is a link about serviceworkers etc todo: Default sample logos are fugly, make them smaller and cleaner and less ugly, maybe just a stylized text and faint background, classier looking todo: Direct report view URL doesn't work if already logged in due to code only being in login form need to hijack the router before navigation to intercept special urls https://test.helloayanova.com/widgets/499 localhost:8080/viewreport?oid=20000&rid=9 todo: wiki image helper, if select ok but haven't selected a attachment it errors out, ok should not be available until there is a valid image selected todo: attach files drag and drop helper should be expanded to include the selection tab, not just the list tab todo: .webp type not available to wiki image helper when attached this is because it's not identified as an image when it's a .webp extension add to image types list for MIME todo: DataListFilter UI pretty shitty error on duplicate how to make a new filter? todo: make our logo smaller or move it when a biz logo is displaying on the login form todo: hide swagger logo and branding in api explorer todo: Email notification sb link to edit subscription? Might be handy to directly click and turn off a subscription from teh notification? todo: chunk-vendors has fa-brands in it which is fucking huge, what else is in there I don't need? eradicate fa-brands! Looks like I'm doing it wrong, I was using a generic NPM module when there is a Vue specific one that also has guidance to limit to just the icons I actually use: https://github.com/FortAwesome/vue-fontawesome This could be a big breaking change so need to do it carefully and all at once as it's own project Definitely need to do though, it will save a lot of size to just include the actual icons I'm using which is a handful todo: Why is the delete button pushed to be exposed all the time, surely it's not that commonly used? todo: LOGOUT UI button is still fucked on mobile needs a re-think, positioning it at the bottom is not working TODO: ON UPDATE TO NEW version NOTE: This might all be due to package.json needing to have it's version updated it might be pulling that into the serviceworker for pwa or consulting it Right now it doesn't always refresh, have to manually do so, it should reload and ditch the cache on update, not sure why it isn't to be honest Try a test of it here locally and with release build, check network traffic, is it in fact actually updating and it's just the login page that isn't or...? have an url that opens automatically or a notification and link to one after a new version has been detected just like visual studio does in order to show what is new in this version maybe ideally it opens to a new document page "whats new in version x.xx" this way no need to go beyond the local server or hit our site unnecessarily todo: attachment control drag and drop should work on both file list and also attach side of things todo: wiki drag and drop image should automatically attach the image and do the linkup etc todo: notification subscription form tags control looks like ordinary tags control might confuse people that they are tagging that subscription Maybe change the text to "Automatic filter tags" or "Only if tagged with" todo: export from grid to xls, csv etc. I'm thinking perhaps also a secondary ability to make report templates that are actually intended for export to xls or whatever as an additional feature but the easiest thing for people would be a straight export of grid data from grid, perhaps using the reportdata system since it pulls in all the bits needed for full data rather than just id links https://www.npmjs.com/package/json-2-csv From extensions? (because it's a single or list thing) DASHBOARD / KPI / BIZ METRICS todo: DASHBOARD - Joyce kpi / charts and graphs - these cases: https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/2024 https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1974 - Printable, need to test render these same metrics graphs in reports todo: decimal-control and currency-control: Fix currency and decimal input controls since after vue-currency update latest version get error ".$ci is undefined" when open a form with them on it proposal: remove all need for vue-currency-input, see if can roll my own using intl code or: fix the stupid error that has now crept in due to unneeded changes they made ## I'm downgrading to the last working version for now and will look into rewriting this to remove the dependency since it just uses Intl.NumberFormat behind the scenes just like I'm doing with date and time inputs NOTE: Last working version is "vue-currency-input": "1.20.3", Plan: figure out if can ditch it as a dependency and if not then figure out how to work with it now as it's all different =-=-=-=- REPORTING STUFF THAT IS FUTURE OR ON HOLD MOVE THIS TO A v.next case or into reporting SPEC doc for future reference if not a coding issue now todo: bizrule for report scale value at server and maybe other shit as well todo: Need a setting that warns people about printing too much data, i.e. "That's a lot of data, are you sure you want to render that, it will be slow" or something to that effect todo: REPORT BACKEND - report delete throws server exception about db context re-use or something to that effect No exact steps top repro, but happens for Joyce with win64 release build if go in and edit same template a few times from widget list then attempt to delete it try to repro on debug build figure it out, could be a big issue if it's not specifically a report issue but wider biz object issue COULD NOT REPRO HERE todo: (On hold pending testing) pdf options UI and passthrough OUTSTANDING Docs DOCS reference pages https://stackoverflow.com/questions/49943479/puppeteer-header-and-footertemplate-doesnt-work#49996999 https://pptr.dev/#?product=Puppeteer&version=v5.3.0&show=api-pagepdfoptions todo: document about the troubleshooting section items here if applicable: https://jsreport.net/learn/chrome-pdf which may or may not apply in our case Testing DONE basically need to be able to select every option and send it through options: http://www.puppeteersharp.com/api/PuppeteerSharp.PdfOptions.html http://www.puppeteersharp.com/api/PuppeteerSharp.Page.html#PuppeteerSharp_Page_PdfAsync_System_String_PuppeteerSharp_PdfOptions_ https://pptr.dev/#?product=Puppeteer&version=v5.3.0&show=api-pagepdfoptions page numbers control Test this, it might do what we need as it has a template for pdf footer and page number is part of it http://www.puppeteersharp.com/api/PuppeteerSharp.PdfOptions.html look at jsreport what do they include in their pdf post processing parameters and capabilities need to add pdfkit or whatever it's called at the front. todo: I have console logging capture code now in backend, but it's doing nothing really, just logs if exception thrown it might be handy if it returned the log value and any other diagnostic info with render return data when user is in designer i.e. have a further property Diagnostic bool and if set then returns diagnostic data and return property with pdf name would be in an object with additional properties for diagnosis etc But only if it proves helpful or necessary mainly this would allow a console log or error or trace to flow back to the user from the script being run at the server //before getting into timeouts and shit make sure it's running as well as can be in docker todo: look at guidance for running puppeteer (js) on alpine docker here: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-on-alpine how jsreport is launching headless chrome, i.e. which settings and flags etc https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#puppeteerlaunchoptions https://jsreport.net/learn/chrome-pdf https://github.com/jsreport/jsreport-chrome-pdf/blob/master/lib/conversion.js (after looking at it, it's still a bit unclear, maybe not relevant as they seem to do a lot differently for that) todo: look over this: https://github.com/puppeteer/puppeteer/issues/1834 todo: REPORT BACKEND - more timeouts for report rendering, hard kill after 30 seconds tops but adjustable interval maybe? each process takes time in part, need to see which is taking which time the most and killable i.e. run a huge report and see which exact step takes which time for each we have the pre-render timing out maybe need more for each step where it's vulnerable to crashing / timing out Devops droplet is overwhelmed by 10k records of widgets using the customfields example report symptom is super high cpu usage (100%) pegged and probably virtual memory usage as well timeout is kind of a hard core way to work around this issue, maybe instead it should be looking at excessive cpu and memory usage? The metrics don't catch it because it happens too quickly for the metrics lifecycle I want it to be able to handle it gracefully without crashing Looks like jsreports has this issue too and they hard kill the process I think if necessary, but they actually re-use the same instance for reporting I think. If it's not an issue then no need to resolve at the moment UPDATE: No way to timeout the pdf generation built into puppeteer sharp so far, maybe some workarounds I think the nuclear option would be to start timer once we know the process id of the chormium instance and that time automatically kills that process ID if still found around XX seconds later (or in the form of a clean up job maybe like the temp file deleter job) todo: reporting load test test locally with 20k widgets, make it crash then determine what limits to set on it and properly return error when it's exceeded right now it just says something about puppeteer and "crash" NOT ABLE TO REPRO HERE =-=-=-=-=- @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@ ROADMAP STAGE 5 - FINALIZE ALL NON BIZ OBJECT SPECIFIC FUNCTIONALITY @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ todo: Much of this stage below needs TRIAGING, do that first. Any real (corebizobject) shit goes to stage 7 todo: FRONT END ROADMAP RELEASES (Sept 23 2020 was put on hold as it appears vuetify is way behind so looking like post release but check again here) look at what's coming on the radar before release in the big libs I'm using at front end as well, i.e. new vue or vuetify major release etc I would really like to release without a huge new migration looming over my head post release with potential breaking changes Vue 3.x is about ready now or very shortly: https://v3.vuejs.org/guide/migration/introduction.html#overview "The Composition API is purely additive and does not affect / deprecate any existing 2.x APIs. I" looks like I can switch to vue 3 without code changes, at least when it comes to composition api Vuetify 3.x is lagging way behind vue 3.x and doesn't appear to be even close so Put front end newification on hold until it all shakes out, looks like post release` todo: "lighthouse" score? PWA test tool for certification? todo: it would be *extremely* useful if we could run automated tests against the release ui with devops server todo: See if this has a use: as an aside there is a relative time formatter available which might be handy does locale aware things like "5 days ago" or "8 years from now" etc https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/RelativeTimeFormat todo: MAPPING getting a *lot* of request about this https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1816 maybe stage 7 or I guess could fake it for now, it's going to be known what will be needed https://news.ycombinator.com/item?id=23722133 https://blog.mapbox.com/openstreetmap-a-global-map-for-worldwide-insight-4e041cbf1ec1 todo: can I support keycodes for saving in AyaNova and other shit that are the same as in v7 or as much as possible, i.e. ctrl-s to save (or whatever was defined) watch out the report editor uses a bunch of hotkeys pre-defined What v7 used to support: f1 - help, case (Keys.Alt | Keys.X) //Close form, alt-w new workorder, alt-m new pm workorder, alt-q new quote, alt-c new client, alt-u new unit, alt-p new part, ctrl-alt-g grid criteria for development, IMPORTANT / DO THIS: insert date and time (localized) as text anywhere with a key combo https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1514 todo: back and forward buttons when running without browser controls in application mode? - wait and see on this one, as it will be likely outside of any particular form so not something to be baked in early necessarily todo: GUIDED TOUR - This is an important feature and at least get a basic one in there for starters and initial release - This is a replacement for the tutorials and videos in v7 a - Need to add that auto-pilot thingy that allows for guided tours in HTML apps - Specifically it should at least have an ONBOARDING walk through of how to move around, enter data, get help etc. Not feature specific but usage specfic. - Later I'll add feature specfic tutorials like how to make a workorder etc todo: clickable urls https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1738 ############### MAYBE STUFF OR STUFF ON HOLD / FUTURE ############### (If get here and not doing it put it into a v.next case where appropriate or spec doc for future reference or just ditch it) todo: consider feature to set server to always use a pre-set browser locale settings and not the ones in the browse itself Scenario is user in another country but needs to login and work with central server? server - wide User specific YAGNI / TTM? REPORTING bits and pieces - page breaks properly EXAMPLE NEEDED this is the current CSS property, not the one jsreport was using: https://developer.mozilla.org/en-US/docs/Web/CSS/break-before There are many options including ones for columns and some to prevent page breaks splitting up blocks.