fixed slow customers and units list

ayanova/devdocs/todo.txt

@@ -35,78 +35,8 @@ Case 3999
 	but what implications at front and back??
 	
 
-TIMEOUT ISSUES ON HUGE LISTS
-	appears to happen with customer list on getting count, not data query
-	todo: 
-		compare timing within AyaNova server and in pgAdmin for same query
-		can a timing report be added into the return data?
-			so we can see at the client when diagnosing how long each bit actually took?
-			or log that if it's excessive beyond xx seconds??
-		Get a query plan and examine what's happening
-		See if it's just a badly written query in the first place and all else is well before getting into the weeds of alternate counting etc
-		research faster ways to handle count queries for all records, see if can speed it up		
-			possibly need an alternative count method if it gets too slow or a timeout of some kind
-			maybe a separate query in datalist for known potentially slow ones?
-			but at the end of the day maybe the query is just a bad one and can be improved
-				the lateral stuff could maybe be a "with" type query using a chain instead of all inside as a subquery which is undoubtedly slower
-		
-		make a "mega" level of data generation that can be put up on devops to replicate test here
-			or, probably more practical, just up the HUGE level to crazy proportions beyond what any of our customers now likely have i.e. millions of records
-			who cares if it takes all night to generate if it provides useful testing
-			This is very important, we need to be confident it can work and perform well with huge amounts of data, i.e. millions of workorders and customers
-
-
-/////////////////////////////////////
-SLOW QUERY WORK AREA
-
-Baseline:
-2021-10-14 10:24:11.2302|INFO|AyaNova.Api.Controllers.DataListController|(debug) DataListFetcher:GetResponse COUNT query took 49859ms to execute: SELECT COUNT(*) FROM ACUSTOMER
-LEFT JOIN AHEADOFFICE ON (ACUSTOMER.HEADOFFICEID = AHEADOFFICE.ID)
-LEFT JOIN ACONTRACT ON (ACUSTOMER.CONTRACTID = ACONTRACT.ID)
-LEFT JOIN LATERAL
-	(SELECT serial AS LASTWORKORDERSERIAL,
-			SERVICEDATE AS LASTWORKORDERSERVICEDATE,
-			AWORKORDER.ID AS LASTWORKORDERID
-		FROM AWORKORDER
-		LEFT JOIN AWORKORDERSTATUS ON AWORKORDER.LASTSTATUSID = AWORKORDERSTATUS.ID
-		WHERE AWORKORDERSTATUS.COMPLETED = TRUE
-			AND AWORKORDER.CUSTOMERID = ACUSTOMER.ID
-		ORDER BY AWORKORDER.ID DESC
-		LIMIT 1) AS LWO ON TRUE 
-
-LATERAL join is *all* the slowness exactly.  It's ok in the data query because the limit is really saving our asses, but I bet a report on that would suck.  So the fix needs to be done for data even though it's not slow
-
-rewrite as a WITH chain type query??
-{WITH qr AS (SELECT asearchkey.atype, asearchkey.objectid, COUNT(*) FILTER (WHERE asearchdictionary.word = 'fish') AS st1 FROM asearchdictionary INNER JOIN asearchkey ON asearchdictionary.id = asearchkey.wordid GROUP BY asearchkey.objectid, asearchkey.atype) SELECT atype, objectid FROM qr WHERE st1 > 0 }
-NOPE same problem, runs once per entire query so returns last
-
-as a function as originally considered:
-CREATE OR REPLACE FUNCTION public.aycustomerlastclosedworkorder(
-	customerid bigint,
-out lastid bigint,
-out lastserial text,
-out lastdate timestamp)
-    LANGUAGE 'plpgsql'
-AS $$
-BEGIN	
-	
-	SELECT serial,
-			SERVICEDATE,
-			AWORKORDER.ID
-		INTO lastserial,lastdate,lastid
-		FROM AWORKORDER
-		LEFT JOIN AWORKORDERSTATUS ON AWORKORDER.LASTSTATUSID = AWORKORDERSTATUS.ID
-		WHERE AWORKORDERSTATUS.COMPLETED = TRUE
-			AND AWORKORDER.CUSTOMERID = customerid
-		ORDER BY AWORKORDER.ID DESC
-		LIMIT 1		
-END;$$;
-
-
-
-
-//////////////////////////////////////
-
+todo: make the huge generation be actually huge and let's see how far we can push it within overnight bounds of generation, i.e. set and come back next morning so should take less than 8 hours ideally
+current changes to huge take 2.5 hours, want way more of every table essentially so....
 
 	security jwt tokens and expiration, can a user just keep working if they are set to inactive because their token hasn't expired?
 		should tokens refresh periodically and have a super short life?