performance, index tuning etc

ayanova/devdocs/todo.txt

@@ -17,24 +17,52 @@ Prioritize items to get to beta fast as possible
 
 
 
-
-select * from atag order by refcount 
+todo: huge data erasedatabase to reseed failing with timeout on aunit delete due to requiring indexes
+	added the indexes, now need to re-test: genhuge over lunch or something will take 30 minutes
 
 
 //performance overall sanity check before beta release
 
 hotspots or potential ones in small sample generation:
-	updatetagsinrepository seems slower than search index!?
-		6.00s (22%)	5.52ns (<0.01%)	AyaNova!AyaNova.Biz.TagBiz+<ProcessUpdateTagsInRepositoryAsync>d__5.MoveNext()
+	
 	log job / log status async during seeding
 	unitbiz.validateasync MOVE NEXT??
-	UnitBiz keeps coming up in several threads weirdly, it seems to trigger things all over the place, is it heavier than most in some way??
-	24.28s (91%)	172.03ns (<0.01%)	Npgsql!Npgsql.Internal.NpgsqlReadBuffer+<<Ensure>g__EnsureLong|41_0>d.MoveNext()
-	AyaNova!AyaNova.Biz.EventLogProcessor+<LogEventToDatabaseAsync>d__1.MoveNext() comes up a lot too, event logging maybe could be improved??
-	Get job status from client checking repeatedly on seeding
-		should change this to check less frequently
 
 todo: perf, test installed postgres vs standalone jimmied one for same op
+	YES, very important to know this, is my cobbled together standalone postgres as good as installed or huge diff??
+
+perf: UNIT unique indexes on names and serial numbers to speed up validation on create??
+	Needs enough data to be a test, here's the check in sql:
+	--after adding index on serial
+explain (analyze,buffers,timing)
+SELECT EXISTS (
+    SELECT 1
+    FROM aunit AS a
+    WHERE ((a.serial = 'wqwerqwe') AND (a.unitmodelid = 7)) AND (a.id <> 0))
+
+	Execution Time: 0.087 ms
+
+	CREATE INDEX IF NOT EXISTS idx_aunit_serial
+    ON public.aunit USING btree
+    (serial COLLATE pg_catalog."default" ASC NULLS LAST)
+    TABLESPACE pg_default;
+
+CREATE INDEX IF NOT EXISTS idx_aunit_modelid
+    ON public.aunit USING btree
+    (unitmodelid ASC NULLS LAST)
+    TABLESPACE pg_default;
+
+
+
+SEEDING PERF LEVELS:
+2021-10-20 10:06:12.8367|INFO|Seeder|Small level sample data seeded in 16 seconds
+2021-10-20 10:29:52.5918|INFO|Seeder|MEDIUM level sample data seeded in 1 minute and 44 seconds
+#Added indexes to speed up validation on aunit for serial and modelid
+
+
+
+
+todo: ops document where is the query timeout setting and how to adjust it for timeouts in long running queries?
 
 todo: hard cap on appointments brought back controlled by client in settings otherwise unusable
 	ideally centered around current date
@@ -66,6 +94,7 @@ todo: firefox schedule fix:
 
 
 TRACK:
+	load and concurrency testing for confident release
 	Customer work order form / view / open???
 	Customer CSR form has a bunch of todo in the template, WTF?
 	Dashboard / widgets
@@ -89,6 +118,9 @@ TRACK:
 	V8Migrate completion
 	All stock reports (and schedule ones)	
 	Seeder / evaluation system worked out fully and implemented
+		do not specify by size but rather scenario
+		we care about size for load testing but don't want to give impression we think HUGE is just a few 10s of thousands of workorders because it's a db issue at a certain point
+		Also NO time estimates in the UI, just make sure they are all a reasonable length of time to generate, like no more than 15 minutes or something, ideally less than 5 minutes
 	Load testing
 	Smoke test for every post release build that I can add problems to as I fix them to avoid regressions
 	Beta testing

ayanova/src/views/ay-evaluate.vue

@@ -205,7 +205,7 @@ export default {
         vm.seedingJobActive = true;
         let jobStatus = 1;
         while (vm.seedingJobActive == true) {
-          await window.$gz.util.sleepAsync(2000);
+          await window.$gz.util.sleepAsync(5000);
           jobStatus = await window.$gz.api.get(
             `job-operations/status/${jobId}`
           );