From 8670ea588343ba4526a90d604571b184eedd15a4 Mon Sep 17 00:00:00 2001 From: John Cardinal Date: Wed, 20 Oct 2021 17:33:42 +0000 Subject: [PATCH] performance, index tuning etc --- ayanova/devdocs/todo.txt | 50 +++++++++++++++++++++++++------ ayanova/src/views/ay-evaluate.vue | 2 +- 2 files changed, 42 insertions(+), 10 deletions(-) diff --git a/ayanova/devdocs/todo.txt b/ayanova/devdocs/todo.txt index ecbe5f73..6f8f7ea3 100644 --- a/ayanova/devdocs/todo.txt +++ b/ayanova/devdocs/todo.txt @@ -17,24 +17,52 @@ Prioritize items to get to beta fast as possible - -select * from atag order by refcount +todo: huge data erasedatabase to reseed failing with timeout on aunit delete due to requiring indexes + added the indexes, now need to re-test: genhuge over lunch or something will take 30 minutes //performance overall sanity check before beta release hotspots or potential ones in small sample generation: - updatetagsinrepository seems slower than search index!? - 6.00s (22%) 5.52ns (<0.01%) AyaNova!AyaNova.Biz.TagBiz+d__5.MoveNext() + log job / log status async during seeding unitbiz.validateasync MOVE NEXT?? - UnitBiz keeps coming up in several threads weirdly, it seems to trigger things all over the place, is it heavier than most in some way?? - 24.28s (91%) 172.03ns (<0.01%) Npgsql!Npgsql.Internal.NpgsqlReadBuffer+<g__EnsureLong|41_0>d.MoveNext() - AyaNova!AyaNova.Biz.EventLogProcessor+d__1.MoveNext() comes up a lot too, event logging maybe could be improved?? - Get job status from client checking repeatedly on seeding - should change this to check less frequently todo: perf, test installed postgres vs standalone jimmied one for same op + YES, very important to know this, is my cobbled together standalone postgres as good as installed or huge diff?? + +perf: UNIT unique indexes on names and serial numbers to speed up validation on create?? + Needs enough data to be a test, here's the check in sql: + --after adding index on serial +explain (analyze,buffers,timing) +SELECT EXISTS ( + SELECT 1 + FROM aunit AS a + WHERE ((a.serial = 'wqwerqwe') AND (a.unitmodelid = 7)) AND (a.id <> 0)) + + Execution Time: 0.087 ms + + CREATE INDEX IF NOT EXISTS idx_aunit_serial + ON public.aunit USING btree + (serial COLLATE pg_catalog."default" ASC NULLS LAST) + TABLESPACE pg_default; + +CREATE INDEX IF NOT EXISTS idx_aunit_modelid + ON public.aunit USING btree + (unitmodelid ASC NULLS LAST) + TABLESPACE pg_default; + + + +SEEDING PERF LEVELS: +2021-10-20 10:06:12.8367|INFO|Seeder|Small level sample data seeded in 16 seconds +2021-10-20 10:29:52.5918|INFO|Seeder|MEDIUM level sample data seeded in 1 minute and 44 seconds +#Added indexes to speed up validation on aunit for serial and modelid + + + + +todo: ops document where is the query timeout setting and how to adjust it for timeouts in long running queries? todo: hard cap on appointments brought back controlled by client in settings otherwise unusable ideally centered around current date @@ -66,6 +94,7 @@ todo: firefox schedule fix: TRACK: + load and concurrency testing for confident release Customer work order form / view / open??? Customer CSR form has a bunch of todo in the template, WTF? Dashboard / widgets @@ -89,6 +118,9 @@ TRACK: V8Migrate completion All stock reports (and schedule ones) Seeder / evaluation system worked out fully and implemented + do not specify by size but rather scenario + we care about size for load testing but don't want to give impression we think HUGE is just a few 10s of thousands of workorders because it's a db issue at a certain point + Also NO time estimates in the UI, just make sure they are all a reasonable length of time to generate, like no more than 15 minutes or something, ideally less than 5 minutes Load testing Smoke test for every post release build that I can add problems to as I fix them to avoid regressions Beta testing diff --git a/ayanova/src/views/ay-evaluate.vue b/ayanova/src/views/ay-evaluate.vue index 5525058a..ac630a64 100644 --- a/ayanova/src/views/ay-evaluate.vue +++ b/ayanova/src/views/ay-evaluate.vue @@ -205,7 +205,7 @@ export default { vm.seedingJobActive = true; let jobStatus = 1; while (vm.seedingJobActive == true) { - await window.$gz.util.sleepAsync(2000); + await window.$gz.util.sleepAsync(5000); jobStatus = await window.$gz.api.get( `job-operations/status/${jobId}` );