diff --git a/devdocs/todo.txt b/devdocs/todo.txt index fd844c03..ee393227 100644 --- a/devdocs/todo.txt +++ b/devdocs/todo.txt @@ -11,26 +11,60 @@ Get stress test running, need to be sure 2022-09-13 17:26:59 60 users 86% cpupeak, 5.76 loadpeak, 3.85 endload, cpu 85%!, 78% mem peaked at end ** First Errors in the test output from jmeter, priors did not error out Peak errors (87%!) 4 minutes into test then drops off, caching? This is too many errors to be reliable. 2022-09-13 19:06:14 10 users 78% cpupeak, 1.39 loadpeak, .1 end load, cpu 15%, mem 58% 2022-09-13 19:26:33 50 users 75% cpupeak, 4.48 loadpeak but at end weirdly, peakmem 48% *Very minor errors 1.7%error was most then dropped, found at early peak load. All load ramped up smoothly over time unlike other runs that were opposite. also watching plex maybe affecting things - 2022-09-13 19:47:04 40 users 0 errors (trying to find error range) + 2022-09-13 19:47:04 40 users 0 errors (trying to find error range) worst appdex .63 (quote), average appdex 0.910 2022-09-13 21:30:07 40 users LONG LOAD 8 hours (28,800seconds) crash test see if it crashes or keeps chugging no crash keeps chugging, super minimal errors looking like maybe when server rebooted spontaneously a few times which is weird, down for about 20 seconds each time maybe the reboot is not the server booting but linux systemctl or dotnet restarting the service -Stress test todo: - Realism - I fear that the way the load seems to peak then drop off as the test progresses indicates it's caching the similar requests and not reflecting true random load - Check over tests are they just simple consistent reads or are there enough writes in there? - Only one write to Vendors, sb more writes, particularly wo more complex objects like wo but at least customers, memos etc - Are the get routes just getting the same records over and over or is it randomizing? - Is the schedule view hit enough? I feel like that's somethign people will be in a lot - Searching in picklists by entered value simulation but randomized? - note doesn't have to actually return values, just needs to do the work - Download or fake download report files, wait on report jobs? +TODO: Stress test todo: + - Parameterize the options, + SEEDLEVEL + USERCOUNT + SERVERADDRESS + DURATION_SECONDS + + - Add wider range of random record counts to fetch suitable to large data + - Add customer creation with a full set of data if possible generate random notes text (jmeter lorem ipsum) + - Add more types of objects fetched, look for objects not fetched before and add them + - Create a memo to a random user + - search picklist randomly doesn't have to return results bonus if random list type too + - download reports? (or would this eat up resources on my test machine, a download isn't really atest of any horsepower is it?) + +TODO: Get minimum server size vs user count so can sell it properly + stress test to get maximum users per server config before consistent errors creep in + Try with minimal server, then go up a level and try again and see what the magic numbers are for usability and reliability + To gather: + APPDEX average ( 0.85 or higher is the goal) + APPDEX worst / route + CPU peak percentage during test + MEM peak percentage during test + LOAD peack 5 minute load value during test (not super relevant but maybe in conjunction with other values a good representative) + +TODO: Get teh amount of space a HUGE seeded database consumes + get the rough stats i.e. how many wo, how many quotes, how many customers users etc, just to get a feel for it + Test with this size data once have the stress test fixed up to be more realistic + 11k customers + 1814 outside users + 500 headoffices + 3k workorders + 1k projects + 200 csr + 10k parts + 1k PO's + 48k inventory transactions + 500 part assemblies + 200 warehouses + 5k vendors + 200 service rates + 100 travel rates + 750 internal users -SUBSCRIPTION CHANGES + +TODO: SUBSCRIPTION CHANGES ATTACHMENT / DATA CAP have a maximum attachment / data cap size that can be configured for subscriptions + Settable from license or defaults in subscription to a known value, i.e. basically 20gb but can be overriden if paid for more! STRIP OUT METRICS (maybe some of them for both types, just not working or accurate, the db and file size stuff is useful if working properly) OPS - subscriptions don't need almost all of ops nor should they have it, is any info leaking that shouldn't go through it remove with a machete (tied to build type) DOCS - ops section of docs not relevant to subscribers, add "PERPETUAL BUILD ONLY" maybe at top or something?