This commit is contained in:
@@ -2,54 +2,65 @@
|
||||
@@@@@@@@@@@@@@@ ROADMAP STAGE 4 - REPORTING / DASHBOARD / KPI
|
||||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||||
|
||||
todo: reporting "ZOMBIE" chrome processes on linux
|
||||
open the shell into the docker container:
|
||||
docker run -it [container_id] /bin/ash
|
||||
run the "top" command and then print a report
|
||||
each run generates 3 chrome processes
|
||||
Many discussions about workarounds and stuff here:
|
||||
https://github.com/hardkoded/puppeteer-sharp/issues/1489
|
||||
https://github.com/puppeteer/puppeteer/issues/1825
|
||||
https://github.com/Yelp/dumb-init
|
||||
|
||||
Ok, fixed it, but now it's a little slower and can crash if it hits the limits of the droplet i.e.
|
||||
Not sure what to do about that, maybe more timeouts in more places?
|
||||
What are the specs for the droplet?
|
||||
|
||||
todo: figure out how jsreport is launching headless chrome, i.e. which settings and flags etc
|
||||
https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#puppeteerlaunchoptions
|
||||
https://jsreport.net/learn/chrome-pdf
|
||||
https://github.com/jsreport/jsreport-chrome-pdf/blob/master/lib/conversion.js
|
||||
|
||||
todo: look over this: https://github.com/puppeteer/puppeteer/issues/1834
|
||||
todo: look at guidance for running puppeteer (js) on alpine docker here: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-on-alpine
|
||||
todo: test with generated huge data locally here in Windows, see what limits get hit and how it handles it
|
||||
todo: document (or tell Joyce) about the troubleshooting section items here:
|
||||
https://jsreport.net/learn/chrome-pdf
|
||||
which may or may not apply in our case
|
||||
|
||||
todo: more timeouts for report rendering, Devops droplet is overwhelmed by 10k records of widgets using the customfields example report
|
||||
symptom is super high cpu usage (100%) pegged and probably virtual memory usage as well
|
||||
timeout is kind of a hard core way to work around this issue, maybe instead it should be looking at excessive cpu and memory usage?
|
||||
The metrics don't catch it because it happens too quickly for the metrics lifecycle
|
||||
|
||||
I want it to be able to handle it gracefully without crashing
|
||||
|
||||
todo: look at metrics snapshot lifecycle, perhaps it should be shorter as it's missing 100% cpu pegging during big rendering on devops
|
||||
|
||||
todo: pdf page numbers control
|
||||
todo: pdf options UI and passthrough
|
||||
basically need to be able to select every option and send it through
|
||||
options:
|
||||
http://www.puppeteersharp.com/api/PuppeteerSharp.PdfOptions.html
|
||||
http://www.puppeteersharp.com/api/PuppeteerSharp.Page.html#PuppeteerSharp_Page_PdfAsync_System_String_PuppeteerSharp_PdfOptions_
|
||||
https://pptr.dev/#?product=Puppeteer&version=v5.3.0&show=api-pagepdfoptions
|
||||
page numbers control
|
||||
Test this, it might do what we need as it has a template for pdf footer and page number is part of it
|
||||
http://www.puppeteersharp.com/api/PuppeteerSharp.PdfOptions.html
|
||||
look at jsreport what do they include in their pdf post processing parameters and capabilities
|
||||
need to add pdfkit or whatever it's called at the front.
|
||||
|
||||
|
||||
|
||||
|
||||
todo: document (or tell Joyce) about the troubleshooting section items here:
|
||||
https://jsreport.net/learn/chrome-pdf
|
||||
which may or may not apply in our case
|
||||
|
||||
|
||||
//before getting into timeouts and shit make sure it's running as well as can be in docker
|
||||
todo: look at guidance for running puppeteer (js) on alpine docker here: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-on-alpine
|
||||
how jsreport is launching headless chrome, i.e. which settings and flags etc
|
||||
https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#puppeteerlaunchoptions
|
||||
https://jsreport.net/learn/chrome-pdf
|
||||
https://github.com/jsreport/jsreport-chrome-pdf/blob/master/lib/conversion.js
|
||||
(after looking at it, it's still a bit unclear, maybe not relevant as they seem to do a lot differently for that)
|
||||
|
||||
|
||||
todo: look over this: https://github.com/puppeteer/puppeteer/issues/1834
|
||||
|
||||
todo: reporting load test
|
||||
test locally with 20k widgets, make it crash then determine what limits to set on it and properly return error when it's exceeded
|
||||
right now it just says something about puppeteer and "crash"
|
||||
|
||||
|
||||
todo: more timeouts for report rendering, hard kill after 30 seconds tops but adjustable interval maybe?
|
||||
each process takes time in part, need to see which is taking which time the most and killable
|
||||
i.e. run a huge report and see which exact step takes which time for each
|
||||
we have the pre-render timing out maybe need more for each step where it's vulnerable to crashing / timing out
|
||||
|
||||
Devops droplet is overwhelmed by 10k records of widgets using the customfields example report
|
||||
symptom is super high cpu usage (100%) pegged and probably virtual memory usage as well
|
||||
timeout is kind of a hard core way to work around this issue, maybe instead it should be looking at excessive cpu and memory usage?
|
||||
The metrics don't catch it because it happens too quickly for the metrics lifecycle
|
||||
|
||||
I want it to be able to handle it gracefully without crashing
|
||||
Looks like jsreports has this issue too and they hard kill the process I think if necessary, but they actually re-use the same instance for reporting I think.
|
||||
If it's not an issue then no need to resolve at the moment
|
||||
|
||||
|
||||
todo: Need a setting that warns people about printing too much data, i.e. "That's a lot of data, are you sure you want to render that, it will be slow" or something to that effect
|
||||
|
||||
|
||||
todo: look at metrics snapshot lifecycle, perhaps it should be shorter as it's missing 100% cpu pegging during big rendering on devops
|
||||
|
||||
|
||||
|
||||
todo: Take a serious look at .net5 migration, it will almost certainly be better to do it now than down the road after a release
|
||||
it also has many performance enhancements and nifty features like code analysis and warnings which might save me a lot of hassle
|
||||
|
||||
@@ -61,6 +72,18 @@ todo: look at what's coming on the radar before release in the big libs I'm usin
|
||||
|
||||
todo: triage this stuff:
|
||||
|
||||
todo: Direct report view URL doesn't work if already logged in due to code only being in login form
|
||||
need to hijack the router before navigation to intercept special urls
|
||||
https://test.helloayanova.com/widgets/499
|
||||
localhost:8080/viewreport?oid=20000&rid=9
|
||||
|
||||
todo: wiki image helper, if select ok but haven't selected a attachment it errors out, ok should not be available until there is a valid image selected
|
||||
todo: attach files drag and drop helper should be expanded to include the selection tab, not just the list tab
|
||||
todo: .webp type not available to wiki image helper when attached
|
||||
this is because it's not identified as an image when it's a .webp extension
|
||||
add to image types list for MIME
|
||||
|
||||
|
||||
todo: DataListFilter UI pretty shitty
|
||||
error on duplicate
|
||||
how to make a new filter?
|
||||
|
||||
Reference in New Issue
Block a user