admin/raven
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Browse Source
This commit is contained in:
John Cardinal
2020-05-18 23:19:21 +00:00
parent fc95aedaac
commit ec20650cb0
2 changed files with 22 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
devdocs/todo.txt
View File

@@ -12,6 +12,15 @@ todo: OPS routes (SERVER AND CLIENT)
BACKUP & related stuff that should be in v.initial release https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/3369
todo: OPS notification created for failed jobs
also maybe direct immediate email bypassing generator?
todo: Search indexing performance improvement and exception avoidance (Search.cs 828)
ON CONFLICT IDEA
https://www.postgresql.org/docs/current/sql-insert.html#SQL-ON-CONFLICT
Idea: do the insert manually with the clause "on conflict do nothing"
if detect it hasn't inserted (conflict) trigger a fetch instead
like what is being done now but won't have the exception to deal with!!
todo: (BREAK THIS OUT INTO LATER/NOW/CASES) there are several outstanding AUTHENTICATION related cases in rockfish for RAVEN
e.g. https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1924
https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1835

17
server/AyaNova/biz/Search.cs
View File

@@ -792,6 +792,11 @@ Now going to try the opposite, a *lot* of text 10 paragraphs in both c2 and note
So the quantity of text directly affects the performance, so it's not just some overhead from the query being run, it's the amount of work it needs to do in the queries
cache or provide directly the translation to save time repeatedly fetching it when doing bulk ops!!!
-After doing this 500=21 That's as fast as when I stripped out all the text, what a huge overhead saving right there!:
2020-01-24 12:00:41.2547|INFO|Seeder|Seeding 500 Widgets....
2020-01-24 12:00:51.9138|INFO|Seeder|500 Widgets seeded in 10649 ms
THINGS TO TRY:
Completely alternate methods:
- https://stackoverflow.com/a/15089664/8939 Store a Digest of each record with that record then can just search the digests (would mean a search has to traverse all records of every table possibly)
@@ -804,10 +809,14 @@ Async the keyword processing
Removing use of EF entirely in search indexing processing in favor of direct sql queries
cache or provide directly the translation to save time repeatedly fetching it when doing bulk ops!!!
-After doing this 500=21 That's as fast as when I stripped out all the text, what a huge overhead saving right there!:
2020-01-24 12:00:41.2547|INFO|Seeder|Seeding 500 Widgets....
2020-01-24 12:00:51.9138|INFO|Seeder|500 Widgets seeded in 10649 ms
Transaction locked faster than exception method now used?
Make indexing a job that happens periodically behind the scenes based on last indexing date and event log?
ON CONFLICT IDEA
https://www.postgresql.org/docs/current/sql-insert.html#SQL-ON-CONFLICT
Idea: do the insert manually with the clause "on conflict do nothing"
if detect it hasn't inserted (conflict) trigger a fetch instead
like what is being done now but won't have the exception to deal with!!
///////////////////////////////////////////////////////////////////////////////