67 lines
4.5 KiB
Plaintext
67 lines
4.5 KiB
Plaintext
TEST RESULTS OVER TIME
|
|
|
|
Stats gathered currently (note, seeding hammers the server badly so data is gathered as a representative eyeball test level after seeding has completed when it has levelled out and is stable):
|
|
APPDEX average ( 0.85 or higher is the goal)
|
|
APPDEX worst / route
|
|
PASS %
|
|
CPU avg percentage during test phase not seeding
|
|
MEM avg representative during actual test phase
|
|
LOAD avg representative 5 minute load value during test (not super relevant but maybe in conjunction with other values a good representative)
|
|
RESTARTS number of times ayanova was restarted due to dotnet exit w error
|
|
|
|
|
|
Test parameters, user count only change between runs, 20 minutes each run. 5 minutes "soak" time between data generation and start of test run
|
|
./jmeter/bin/jmeter.bat -f -n -t stressdevops.jmx -l ./results/results.jtl -e -o ./results -JayServer=3312.onayanova.com -JaySeedLevel=large -JayUserCount=20 -JayDuration=1200 -JayProtocol=https
|
|
|
|
|
|
GOALS
|
|
Don't crash the server (no restarts of AyaNova)
|
|
Don't run out of memory
|
|
Average appdex of .85 or better
|
|
REasonable cpu overhead (which corelates to desired appdex around max )
|
|
True (test not seeding) pass rate of 98% or higher
|
|
|
|
|
|
All tests were on same droplet: 3312.onayanova.com in NYC1 data center - Ubuntu 22.04 x64
|
|
Sized it up and ran through each level of testing
|
|
Tests are a what I hope is a fairly representative simulation of users hitting it pretty hard in a real world manner (e.g. random brief pauses between page loads, not just serially hammering it in a machine like manner).
|
|
|
|
TYPE CPU TYPE VCPUS MEMORY SSD TRANSFER PRICE
|
|
=============================================
|
|
|
|
BASIC SHARED CPU 1 VCPU 1 GB 25 GB 1 TB $6/MO
|
|
2022-09-14 17:55:22 30 users cpu 47%, mem 83%, load 1.15, apdex avg .920, appdex worst quote .556, pass 98.98%, restarts 0
|
|
2022-09-14 18:53:24 50 users cpu 73%, mem 86%, load 2.3, apdex avg .872 worst quote .400, pass 99.95%, restarts 0
|
|
2022-09-14 19:56:51 75 users cpu 98%, mem 94%, load 12, apdex avg .577 worst quote .023, pass 86%, restarts 2
|
|
2022-09-14 20:58:35 SUPERLOAD huge data, 8 hours run time 20 users cpu 42%, mem 82%, load 1.2, apdex avg .917 worst .489 (makes sense lots of records, quote was .712), pass 99.86%, restarts 0
|
|
|
|
BASIC - PREMIUM AMD SHARED CPU 1 VCPU 1 GB 25 GB 1 TB $7/MO
|
|
2022-09-15 07:32:50 resize droplet to basic premium AMD only change then rerun
|
|
2022-09-15 09:23:31 30 users cpu 35%, mem 85%, load 1.0, appdex avg .955 worst quote .821, pass 100%, restarts 0
|
|
2022-09-15 10:47:17 50 users cpu 60%, mem 89%, load 2.0, appdex avg .925 worst pm .587, pass 99.98%, restarts 0
|
|
2022-09-15 12:50:43 75 users cpu 87%, mem 95%, load 6.0, appdex avg .775 worst qutoe .237, pass 94.08%, restarts 1
|
|
|
|
BASIC PREMIUM AMD SHARED CPU 1 VCPU 2 GB 25 GB 2 TB (2gb ram is only relevant change from previous)
|
|
2022-09-15 15:38:59 30 users cpu 43%, mem 48%, load .97, appdex avg .949 worst quote .772, pass 99.99%, restarts 0
|
|
2022-09-15 16:16:16 50 users cpu 69%, mem 57%, load 2.3, appdex avg .913 worst quote .594, pass 99.97%, restarts 0
|
|
2022-09-15 16:52:42 75 users cpu 87%, mem 67%, load 6.0, appdex avg .813 worst quote .241, pass 99.2%, restarts 1
|
|
|
|
BASIC SHARED CPU 2 VCPUS 2 GB 25 GB 3 TB $18/MO
|
|
2022-09-15 17:33:06 30 users cpu 38%, mem 44%, load 1.47, appdex avg .941 worst quote .623, pass 99.99%, restarts 0
|
|
2022-09-15 18:11:14 50 users cpu 46%, mem 51%, load 1.5, appdex avg .956 worst pm .765, pass 99.99%, restarts 0
|
|
2022-09-15 18:53:46 75 users cpu 74%, mem 65%, load 4.4, appdex avg .913 worst quote .439, pass 99.93%, restarts 0
|
|
|
|
|
|
BASIC - PREMIUM AMD SHARED CPU 2 VCPUS 2 GB 25 GB 3 TB $21/MO
|
|
30 users cpu 20%, mem 65%, load .45, appdex avg .986 worst data-list/CustomerDataList .943, pass 100%
|
|
50 users cpu 32%, mem 67%, load 1.0, appdex avg .983 worst data-list/CustomerDataList .917, pass 99.99%
|
|
75 users cpu 43%, mem 73%, load 1.5, appdex avg .981 worst custdatalist .917 , pass 99.99% , restarts 0
|
|
100 users cpu 57%, mem 77%, load 2.3, appdex avg .959 worst quote .815 , pass 99.97 , restarts 0
|
|
|
|
|
|
CPU-OPTIMIZED *DEDICATED* CPU 2 VCPUS 4 GB 25 GB 4 TB $42/MO
|
|
30 users cpu 10%, mem 26%, load .35 , appdex avg .969 worst customerdatalist .858 , pass 99.98% , restarts 0
|
|
50 users cpu 15%, mem 29%, load .48, appdex avg .969 worst customerdatalist .868 , pass 99.98 , restarts 0
|
|
75 users cpu 21%, mem 35%, load .78, appdex avg .970 worst customerdatalist .843 , pass 100% , restarts 0
|
|
100 users cpu 30%, mem 46%, load 1.2, appdex avg .967 worst customerdatalist .854 , pass 99.98% , restarts 0
|