This commit is contained in:
223
devdocs/todo.txt
223
devdocs/todo.txt
@@ -5,229 +5,6 @@ Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOiIxNTQ0NTU5NzAwIiwiZXhwIjoi
|
||||
|
||||
## IMMEDIATE ITEMS
|
||||
|
||||
See below todo to figure out what is wrong with the filter query next month (for starters)
|
||||
It's nothing obvious that's for sure.
|
||||
|
||||
|
||||
|
||||
==================
|
||||
DEVOPS - PUT UP A DEVOPS COPY AFTER THE ABOVE IS DONE AND RUN ALL UNIT TESTS TO CONFIRM DATE FILTER COD
|
||||
FIRST RUN WITH HUGE DATASET LOG:
|
||||
Starting extension
|
||||
Finding projects for pattern c:/data\code\raven\test\raven-integration/
|
||||
Found 1 matches for pattern in folder c:\data\code\raven\test\raven-integration
|
||||
Evaluating match c:/data/code/raven/test/raven-integration/
|
||||
Adding directory c:/data/code/raven/test/raven-integration/
|
||||
Executing dotnet test -t -v=q in c:/data/code/raven/test/raven-integration/
|
||||
Discoverd tests with xunit. Found 135 in 1 directories
|
||||
Executing dotnet build in c:/data/code/raven/test/raven-integration/
|
||||
Executing dotnet test --no-build --logger "trx;LogFileName=C:\Users\John\AppData\Local\Temp\test-explorer\0.trx" in c:/data/code/raven/test/raven-integration/
|
||||
Process 12920 started
|
||||
Test run for c:\data\code\raven\test\raven-integration\bin\Debug\netcoreapp2.1\raven-integration.dll(.NETCoreApp,Version=v2.1)
|
||||
Microsoft (R) Test Execution Command Line Tool Version 15.7.0
|
||||
Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
|
||||
Starting test execution, please wait...
|
||||
Failed raven_integration.DataFilterFilteringLists.TextOpNotEqualFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be 2, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.Be(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 49
|
||||
at raven_integration.DataFilterFilteringLists.TextOpNotEqualFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 5162
|
||||
Failed raven_integration.DataFilterFilteringLists.DecimalNotEqualToFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be greater or equal to 1, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterOrEqualTo(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 239
|
||||
at raven_integration.DataFilterFilteringLists.DecimalNotEqualToFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 7142
|
||||
Failed raven_integration.DataFilterFilteringLists.DateOpNotEqualToFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be greater or equal to 1, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterOrEqualTo(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 239
|
||||
at raven_integration.DataFilterFilteringLists.DateOpNotEqualToFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 694
|
||||
Failed raven_integration.DataFilterFilteringLists.TextOpLessThanOrEqualToFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be 2, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.Be(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 49
|
||||
at raven_integration.DataFilterFilteringLists.TextOpLessThanOrEqualToFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 5041
|
||||
Failed raven_integration.DataFilterFilteringLists.IntegerOpLessThanFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be greater or equal to 1, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterOrEqualTo(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 239
|
||||
at raven_integration.DataFilterFilteringLists.IntegerOpLessThanFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 6068
|
||||
Failed raven_integration.DataFilterFilteringLists.IntegerNotEqualToFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be greater or equal to 1, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterOrEqualTo(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 239
|
||||
at raven_integration.DataFilterFilteringLists.IntegerNotEqualToFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 6271
|
||||
Failed raven_integration.DataFilterFilteringLists.DecimalOpLessThanFilterWorks
|
||||
Error Message:
|
||||
Expected InclusiveMatchCount to be greater or equal to 1, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterOrEqualTo(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 239
|
||||
at raven_integration.DataFilterFilteringLists.DecimalOpLessThanFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 6939
|
||||
Failed raven_integration.DataFilterFilteringLists.DateTokenInTheLast3MonthsFilterWorks
|
||||
Error Message:
|
||||
Expected ((JArray)a.ObjectResponse["data"]).Count to be greater than 0, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterThan(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 217
|
||||
at raven_integration.DataFilterFilteringLists.DateTokenInTheLast3MonthsFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 2209
|
||||
Failed raven_integration.DataFilterFilteringLists.DateTokenInTheLast6MonthsFilterWorks
|
||||
Error Message:
|
||||
Expected ((JArray)a.ObjectResponse["data"]).Count to be greater than 0, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterThan(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 217
|
||||
at raven_integration.DataFilterFilteringLists.DateTokenInTheLast6MonthsFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 2310
|
||||
Failed raven_integration.DataFilterFilteringLists.DateTokenNextMonthFilterWorks
|
||||
Error Message:
|
||||
Expected ExclusiveMatchCount to be 0, but found 1.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.Be(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 49
|
||||
at raven_integration.DataFilterFilteringLists.DateTokenNextMonthFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 1606
|
||||
Failed raven_integration.DataFilterFilteringLists.DateTokenPast90DaysFilterWorks
|
||||
Error Message:
|
||||
Expected ((JArray)a.ObjectResponse["data"]).Count to be greater than 0, but found 0.
|
||||
Stack Trace:
|
||||
at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\XUnit2TestFramework.cs:line 32
|
||||
at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Execution\AssertionScope.cs:line 215
|
||||
at FluentAssertions.Numeric.NumericAssertions`1.BeGreaterThan(T expected, String because, Object[] becauseArgs) in C:\projects\fluentassertions-vf06b\Src\FluentAssertions\Numeric\NumericAssertions.cs:line 217
|
||||
at raven_integration.DataFilterFilteringLists.DateTokenPast90DaysFilterWorks() in c:\data\code\raven\test\raven-integration\DataFilter\DataFilterFilteringLists.cs:line 3818
|
||||
Results File: C:\Users\John\AppData\Local\Temp\test-explorer\0.trx
|
||||
|
||||
Total tests: 135. Passed: 124. Failed: 11. Skipped: 0.
|
||||
Test execution time: 53.2319 Seconds
|
||||
|
||||
Process 12920 finished
|
||||
|
||||
=-=-=
|
||||
WITH SMALL DATASET MANY MORE PASSED EXCEPT THESE:
|
||||
DateTokenPast90DaysFilterWorks
|
||||
DateTokenNextMonthFilterWorks
|
||||
DateTokenInTheLast6MonthsFilterWorks
|
||||
DateTokenInTheLast3MonthsFilterWorks
|
||||
|
||||
Which failed repeatedly, I wonder if the tests are flawed or there's a weird date issue of some kind?
|
||||
=-=-=-
|
||||
Further data: Next Month filter - This is failing at devops only with the exclusive date being the first day of the month -1 seconds as written
|
||||
However if I back it by 8 hours and 1 second then it passes, which is utc difference (7 hours fails still)
|
||||
So what appears to be happening is the server at UTC is not correctly building the sql filter dates, Need to log that shit to see what is actually happening
|
||||
|
||||
TODO:
|
||||
Put a debug log on the FilterSqlCriteriaBuilder to log the input and the output in debug mode so can see what is actually happening
|
||||
Put it up to the server and check to see what is actually happening
|
||||
I suspect that the issue is the criteria building code dates are starting out in UTC on devops then are further converted to UTC when building the final sql fragment
|
||||
and somehow getting fucked in that process even though they should just be staying in the same UTC the whole time.
|
||||
|
||||
Here at this server locally they are starting out in pacific time and then converting to UTC properly, maybe?
|
||||
- Nope, did a test and it's all good at the server, something else going on
|
||||
|
||||
Will need the debug log of the query built to see what the what next
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Working (localhost) next month filter log:
|
||||
2018-12-10 08:39:49.3954|INFO|WidgetBiz::GetManyAsync|Filter criteria:
|
||||
2018-12-10 08:39:49.3954|INFO|WidgetBiz::GetManyAsync|[
|
||||
{
|
||||
"fld": "name",
|
||||
"op": "%-",
|
||||
"value": "DateTokenNextMonthFilterWorks"
|
||||
},
|
||||
{
|
||||
"fld": "startdate",
|
||||
"op": "=",
|
||||
"value": "{[nextmonth]}"
|
||||
}
|
||||
]
|
||||
2018-12-10 08:39:49.3954|INFO|WidgetBiz::GetManyAsync|Generated SQL:
|
||||
2018-12-10 08:39:49.4126|INFO|WidgetBiz::GetManyAsync|SELECT *, xmin FROM AWIDGET where name Like 'DateTokenNextMonthFilterWorks%' AND startdate >'2019-01-01T07:59:59.0000000Z' AND startdate <'2019-02-01T08:00:00.0000000Z'
|
||||
2018-12-10 08:39:49.4248|WARN|Microsoft.EntityFrameworkCore.Query|Query: '(from Widget <generated>_4 in DbSet<Widget> select [<generated>_4]).Skip(__p_1).Take(__p_2)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
|
||||
|
||||
SERVER DEVOPS VERSION:
|
||||
2018-12-10 16:46:25.5657|INFO|WidgetBiz::GetManyAsync|Filter criteria:
|
||||
2018-12-10 16:46:25.5657|INFO|WidgetBiz::GetManyAsync|[
|
||||
{
|
||||
"fld": "name",
|
||||
"op": "%-",
|
||||
"value": "DateTokenNextMonthFilterWorks"
|
||||
},
|
||||
{
|
||||
"fld": "startdate",
|
||||
"op": "=",
|
||||
"value": "{[nextmonth]}"
|
||||
}
|
||||
]
|
||||
2018-12-10 16:46:25.5657|INFO|WidgetBiz::GetManyAsync|Generated SQL:
|
||||
2018-12-10 16:46:25.5657|INFO|WidgetBiz::GetManyAsync|SELECT *, xmin FROM AWIDGET where name Like 'DateTokenNextMonthFilterWorks%' AND startdate >'2018-12-31T23:59:59.0000000Z' AND startdate <'2019-02-01T00:00:00.0000000Z'
|
||||
2018-12-10 16:46:25.5733|WARN|Microsoft.EntityFrameworkCore.Query|Query: '(from Widget <generated>_4 in DbSet<Widget> select [<generated>_4]).Skip(__p_1).Take(__p_2)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
|
||||
|
||||
|
||||
//AFTER UPDATE
|
||||
2018-12-10 19:17:47.3300|INFO|WidgetBiz::GetManyAsync|Filter criteria:
|
||||
2018-12-10 19:17:47.3300|INFO|WidgetBiz::GetManyAsync|[
|
||||
|
||||
{
|
||||
|
||||
"fld": "name",
|
||||
|
||||
"op": "%-",
|
||||
|
||||
"value": "DateTokenNextMonthFilterWorks"
|
||||
|
||||
},
|
||||
|
||||
{
|
||||
|
||||
"fld": "startdate",
|
||||
|
||||
"op": "=",
|
||||
|
||||
"value": "{[nextmonth]}"
|
||||
|
||||
}
|
||||
|
||||
]
|
||||
2018-12-10 19:17:47.3300|INFO|WidgetBiz::GetManyAsync|Generated SQL:
|
||||
2018-12-10 19:17:47.3300|INFO|WidgetBiz::GetManyAsync|SELECT *, xmin FROM AWIDGET where name Like 'DateTokenNextMonthFilterWorks%' AND startdate >'2018-12-31T23:59:59.0000000Z' AND startdate <'2019-02-01T00:00:00.0000000Z'
|
||||
|
||||
|
||||
|
||||
WHAT IS HAPPENING
|
||||
what is happening is that the data is converted to UTC before sending to the server, the server is supposed to be a dumb date unaware repository
|
||||
However if I want "next month" then that's relative to my personal location, I can't go by the server's next month because that is in UTC and I'm not.
|
||||
So for the server to process *my* next month it needs to know what my relative time offset is.
|
||||
I've decided to never trust the client for current time zone information and use an independent setting for the user account.
|
||||
So the relative time query builders need to use the relative offset in the production of the NOW date that everything keys off of.
|
||||
SOLUTION: provide the relative time offset to the sql criteria builder for CURRENT USER, sql criteria builder will factor it into all the NOW values determined.
|
||||
|
||||
=================
|
||||
|
||||
|
||||
|
||||
TESTING TODO - Are all tests viable with a huge dataset, try locally and rejig the tests as necessary to make them work properly with huge data if necessary
|
||||
|
||||
Reference in New Issue
Block a user