From fb370940569a2eec0c621f87345e13b9c012f52d Mon Sep 17 00:00:00 2001 From: John Cardinal Date: Fri, 25 Sep 2020 21:06:39 +0000 Subject: [PATCH] --- devdocs/specs/core-reporting.txt | 130 +++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) diff --git a/devdocs/specs/core-reporting.txt b/devdocs/specs/core-reporting.txt index abb3d3b1..c0916f07 100644 --- a/devdocs/specs/core-reporting.txt +++ b/devdocs/specs/core-reporting.txt @@ -1,5 +1,135 @@ REPORTING SPECS +################################## +2020-09-25 OUTSTANDING REPORTING STUFF +This is stuff that may or may not be relevant in future, keeping here from my notes +REPORTING STUFF THAT IS FUTURE OR ON HOLD +MOVE THIS TO A v.next case or into reporting SPEC doc for future reference if not a coding issue now + + +todo: bizrule for report scale value at server + and maybe other shit as well + +todo: Need a setting that warns people about printing too much data, i.e. "That's a lot of data, are you sure you want to render that, it will be slow" or something to that effect + + +todo: REPORT BACKEND - report delete throws server exception about db context re-use or something to that effect + No exact steps top repro, but happens for Joyce with win64 release build if go in and edit same template a few times from widget list then attempt to delete it + try to repro on debug build figure it out, could be a big issue if it's not specifically a report issue but wider biz object issue + COULD NOT REPRO HERE + +todo: (On hold pending testing) pdf options UI and passthrough + OUTSTANDING + Docs + DOCS reference pages + https://stackoverflow.com/questions/49943479/puppeteer-header-and-footertemplate-doesnt-work#49996999 + https://pptr.dev/#?product=Puppeteer&version=v5.3.0&show=api-pagepdfoptions + todo: document about the troubleshooting section items here if applicable: + https://jsreport.net/learn/chrome-pdf + which may or may not apply in our case + Testing + + DONE + basically need to be able to select every option and send it through + options: + http://www.puppeteersharp.com/api/PuppeteerSharp.PdfOptions.html + http://www.puppeteersharp.com/api/PuppeteerSharp.Page.html#PuppeteerSharp_Page_PdfAsync_System_String_PuppeteerSharp_PdfOptions_ + https://pptr.dev/#?product=Puppeteer&version=v5.3.0&show=api-pagepdfoptions + page numbers control + Test this, it might do what we need as it has a template for pdf footer and page number is part of it + http://www.puppeteersharp.com/api/PuppeteerSharp.PdfOptions.html + look at jsreport what do they include in their pdf post processing parameters and capabilities + need to add pdfkit or whatever it's called at the front. + +todo: I have console logging capture code now in backend, but it's doing nothing really, just logs if exception thrown + it might be handy if it returned the log value and any other diagnostic info with render return data when user is in designer + i.e. have a further property Diagnostic bool and if set then returns diagnostic data + and return property with pdf name would be in an object with additional properties for diagnosis etc + But only if it proves helpful or necessary + mainly this would allow a console log or error or trace to flow back to the user from the script being run at the server + +//before getting into timeouts and shit make sure it's running as well as can be in docker +todo: look at guidance for running puppeteer (js) on alpine docker here: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-on-alpine + how jsreport is launching headless chrome, i.e. which settings and flags etc + https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#puppeteerlaunchoptions + https://jsreport.net/learn/chrome-pdf + https://github.com/jsreport/jsreport-chrome-pdf/blob/master/lib/conversion.js + (after looking at it, it's still a bit unclear, maybe not relevant as they seem to do a lot differently for that) + + +todo: look over this: https://github.com/puppeteer/puppeteer/issues/1834 + +todo: REPORT BACKEND - more timeouts for report rendering, hard kill after 30 seconds tops but adjustable interval maybe? + each process takes time in part, need to see which is taking which time the most and killable + i.e. run a huge report and see which exact step takes which time for each + we have the pre-render timing out maybe need more for each step where it's vulnerable to crashing / timing out + + Devops droplet is overwhelmed by 10k records of widgets using the customfields example report + symptom is super high cpu usage (100%) pegged and probably virtual memory usage as well + timeout is kind of a hard core way to work around this issue, maybe instead it should be looking at excessive cpu and memory usage? + The metrics don't catch it because it happens too quickly for the metrics lifecycle + + I want it to be able to handle it gracefully without crashing + Looks like jsreports has this issue too and they hard kill the process I think if necessary, but they actually re-use the same instance for reporting I think. + If it's not an issue then no need to resolve at the moment + + UPDATE: No way to timeout the pdf generation built into puppeteer sharp so far, maybe some workarounds + I think the nuclear option would be to start timer once we know the process id of the chormium instance and that time automatically kills that process ID if still found around XX seconds later + (or in the form of a clean up job maybe like the temp file deleter job) + +todo: reporting load test + test locally with 20k widgets, make it crash then determine what limits to set on it and properly return error when it's exceeded + right now it just says something about puppeteer and "crash" + NOT ABLE TO REPRO HERE + + + +todo: MEMORY / PERFORMANCE / CRASHING + BOTTOM LINE: + It *sometimes* can't render at 5k records lots of pages on devops but mainly it does around 2.5 to 3 minutes avg, same on dev ws takes 1.5 m avg but no crash + uses a lot of ram, also cpu does get pegged by chrome process at 95% so it's a bit of both I guess + If users have issue with server during reporting they should check ram and cpu + a minimal droplet appears fine for normal workload, it's the big report rendering that's the most inefficient + + look at proper settings for rendering in docker on linux and try to make it resilient to crashing even if set too low or too little resources + IDEAS: + check guidelines for running in docker from link below + check command line params for best fit + see about re-using same or whatever + check cpu, memory usage upon render request, reject immediately if the system is overburdened + check for running chromium processes or some way to determine if in the middle of the last render for someone + + Try a method to zap all chromium processes as a test to see if they *can* be killed whil stuck this way + To test use the "custom date time format helpers" with 5k records, that reliably freezes everything even when others all seem to run ok + (this in itself is curious but whatever) + + What I'm seeing is on a long render it returns a 504 gateway timeout to the client but it's still churning away in the background + 5k records and "custom date time format helpers" report will take about 3 minutes to render, maybe less and it will crash out for the client return 504 but then it will complete at some point + Confirmed it *does* complete because I was able to download the 4.12mb 2144 page pdf manually from the server temp folder via filezilla once I saw the cpu go down again + and chrome process stop + 90% memory is the max used in do graph panel, probably docker is not letting it take all or something isn't + maybe swapping out is what's happening, peak memory usage seems to be half a gig or so but c + 97% cpu usage is max + + I'm thinking it's a memory issue more than a cpu issue because the cpu is hardly pegged at all while it's rendering right up until it appears to run out of memory + then it starts swapping and all hell breaks loose and the cpu pegs on the swap daemon + Need to test by moving up to 2gb of memory witha resize but keep the single vcpu and see what's what + also increase the timeout in nginx for the reverse proxy to wait + A temporary rendering caused memory shortage should not cause the system to come to a standstill + + +todo: memory usage and timeout is directly related to the amount of space taken up physically on the page + it's NOT related to the helpers as just putting static text on the page causes teh same issue + it's memory taken to render to pdf probably and a byte is a byte even if it's blank white page + =-=-=-=-=- + + + + + + + +########## ORIGINAL SPECS ################### CASES 1734 - REPORTS:GRIDS: - grid filter name and summary of filter criteria available as fields to print on report