From b55818ea6372f221d345128bdee19d0bf47dc014 Mon Sep 17 00:00:00 2001 From: John Cardinal Date: Tue, 21 Dec 2021 20:10:19 +0000 Subject: [PATCH] --- devdocs/deploy.txt | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/devdocs/deploy.txt b/devdocs/deploy.txt index f38a9854..eb4f62a0 100644 --- a/devdocs/deploy.txt +++ b/devdocs/deploy.txt @@ -42,14 +42,16 @@ BASELINE WHAT TO CHECK: DASHBOARD PAGE is the main page with the main stats: "REquests summary" The most telling overall stat is the "Requests summary" pie chart pass/fail ratio on the main page -As of today 2021-12-21 release 8-beta.06 the results are PASS: 99.04%, FAIL: 0.96% -(however they are all in the report render so need to look into that something wrong in the test) +As of today 2021-12-21 release 8-beta.06 the results are PASS: 92.87%, FAIL: 7.13% +Errors *are* expected, it's a stress / load test but there shouldn't be an entirely failing route, just a scattered set of failures, +typically unauthorized or bad gateway "Appdex" - skipping this for now, not really relevant at the moment "STATISTICS" table - this is the important one as it shows failures -- Scroll down and look in "executions" -> "FAIL" column and make sure they are not all in one line item or more than a tiny pct. - if see a lot of failures need to manually look into that exact issue and route and examine the jmeter test for that route / run it manually perhaps +- Scroll down and look in "executions" -> "FAIL" column and make sure they are not all in one line item + if see a lot of failures in a single route need to manually look into that exact issue and route and examine the jmeter test for that route / run it manually perhaps + it's normal to see unauthorized or bad gateway errors when the server is super stressed out but other failures might need examination or all of it if less than 90% success @@ -58,7 +60,7 @@ NOTES: - this also catches reporting as it generates some reports report-render test has explicit report id number for workorder, currently it's 15 for the dispatch report, if all fail it may have changed the id number would be better deterministically but for now this works. - + - To test local or devops edit the project file "C:\data\code\raven-test\load\stressdevops.jmx" There are two configuration sections near top for local or devops and it's switched by setting either enabled or disabled: