LIST / GRAPH FILTERING SORTING AND COLUMN RETURNED

DATALIST
A dataList is a raw source of data for returning to client
All below is based on a datalist which is a predefined list of fields, rights and some query info for joins etc
There are specific datalists made for picklists, viewing all grid data, KPI shit, whatever is required during development
DataLists are hard coded, only a DataListView can be customized to limit, filter, sort what is returned from a pre-defined DataList
DataLists have roles associated with them, only a user of one of those roles can fetch that list (or see it as an option when doing reporting etc)

DATALISTVIEW
A DataListView is a filtered, sorted, reduced version of a DataList
It can be saved and set to private or public
They can be made on the fly in the Client and don't need to be saved as the Client will always send the datalistview json string with the request 
DataListView objects are used to filter and sort and determine display order and presence of columns in order to feed these client needs:
DataTables (was grids in v7)
Reports
Select boxes PickLists (on forms like selecting a part or a client)

When the DataList route is hit it's used to get all of the above types of data.
A dataListView JSON string is sent with the request in order to have the server sort/filter/return 

PICKLISTS
Picklists are based on a particular DataList with rights for ANYONE
Client will be hard coded to specifiy the exact dataLIst
Users can select a datalistview for a picklist DataList which will in turn affect how it appears and what's returned
    - This is like the PartDisplayFormat feature of v7 only for everything!
Users will get the default hard coded DataListView built into the DataList that is specified for that picklist unless it's changed to an alternative datalistview
There is a default PickList format and a default DataTable format pre-defined and hard coded at the server with each dataList definition object

If the client isn't using a particular dataListView it MUST send the ListView json as empty string or omit it entirely.
This will instruct the server to use the pre-defined format instead
	
JSON DataListView format:
ONE string of json, not separate
Contains all columns that are relevant each as an object with various props desired to be returned in order
If a column isn't filtered or sorted or included then it isn't in the collection
Order and presence determines sort, filter, return and display order
Each column object contains it's sorted, sort order and filters collection which can be empty
Sort and filter properties are optional and can be omitted
"any" property is optional in a filter and can be omitted

UI: shows all available columns, user sets order by re-arranging and whether to include or not with a checkbox Include which defaults to true if there is any conditions on that column

Example:

DataListView JSON:
[{key:"COLUMN UNIQUE KEY ID",sort:"-" or "+",filter:{any:true/false,items:[{FILTER OBJECT SEE BELOW}]} }, {key:"second column unique key"},{...etc...}]

Sort property definition
ID VS NAME
if it's an name *and* id field it won't matter because it will sort by name only 
I've thought about it and can't see a use case for needing to sort by the underlying ID value, but if people want that they can just not sort the list to get
the most recent records at the top (for example)

Filter object definition:
Has an "any" boolean property which if true means make an OR query of the conditions if more than one, if nore present or false it means AND each condition (if more than one)


filter:{any:true/false,items:[
	{op:"OPERATOR",value:One of an array of strings, single string or single value of any supported type, i.e. int, decimal, bool iso8601 date etc }
	]}

ID VALUE vs NAME VALUE
    Some colums represent both an ID and a text name, in those cases if the operator is Equals or NotEquals and the "value" property of the filter is a number then it's a specific ID
    Otherwise if it's a string it's always filtering by the display name instead

AUTHORIZATION:
    Authorization is controlled via roles set on underlying DataList object itself
    We provide a DataList suitable for picklists with ANY role access and anything with sensitive info has it's DataList Required roles to higher level 
    This way can still supply for picklists but hide sensitive infor for tables etc
    Rights to DataListView follow rights to underlying object and also rights to make a datalistview itself

Filter operators:


        public const string OpEquality = "=";
        public const string OpGreaterThan = ">";
        public const string OpGreaterThanOrEqualTo = ">=";
        public const string OpLessThan = "<";
        public const string OpLessThanOrEqualTo = "<=";
        public const string OpNotEqual = "!=";
        public const string OpNotLike = "!%";
        public const string OpStartsWith = "%-";
        public const string OpEndsWith = "-%";
        public const string OpContains = "-%-";
        public const string OpNotContains = "!-%-";


TAGS - At server if equality compare value is an array of strings then it's assumed to be tags, if it's a single value then it's regular



Also relative date filters are supported and their implementation is entirely a CLIENT responsibility as it touches on Locale issues:

public const string Null = "*NULL*";
//         public const string Yesterday = "{[yesterday]}";
//         public const string Today = "{[today]}";
//         public const string Tomorrow = "{[tomorrow]}";
//         public const string LastWeek = "{[lastweek]}";
//         public const string ThisWeek = "{[thisweek]}";
//         public const string NextWeek = "{[nextweek]}";
//         public const string LastMonth = "{[lastmonth]}";
//         public const string ThisMonth = "{[thismonth]}";
//         public const string NextMonth = "{[nextmonth]}";
//         public const string FourteenDayWindow = "{[14daywindow]}";
//         public const string Past = "{[past]}";
//         public const string Future = "{[future]}";
//         public const string LastYear = "{[lastyear]}";
//         public const string ThisYear = "{[thisyear]}";
//         public const string InTheLast3Months = "{[last3months]}";
//         public const string InTheLast6Months = "{[last6months]}";
//         public const string InTheLastYear = "{[lastcalendaryear]}";

//         //Months THIS year
//         public const string January = "{[january]}";
//         public const string February = "{[february]}";
//         public const string March = "{[march]}";
//         public const string April = "{[april]}";
//         public const string May = "{[may]}";
//         public const string June = "{[june]}";
//         public const string July = "{[july]}";
//         public const string August = "{[august]}";
//         public const string September = "{[september]}";
//         public const string October = "{[october]}";
//         public const string November = "{[november]}";
//         public const string December = "{[december]}";

Client calculates dates and sends with DataListView when requesting a list














******************************** DEPRECATED MOSTLY BELOW ********************************************************


This all applies equally to a report or a list or a graph so all work basically the same but will refer only to "list" for simplicity:

At client user can view a list and select from a dropdown of saved filters for that type of list which corresponds to an api route.
User creates filters in separate UI from the actual list, i.e. clicks on the filter button above a list to make or edit filter.
Two types of filters Named or Default:
    User can save a filter with a name for later selection or it will always default to "Default" (localized) if not edited but will always require saving to server.
    Generic "default" filter is always personal, not public / shared
    - This is a client issue, Server doesn't care what the filters are called adn has no special processing for default filters
Named filters can be made public or personal.  If public then all users with rights to that object can see them, personal are always only personal.
Filter is constructed from an FILTEROPTIONS object fetched from the server list route that has a list type name which is unique to that list route 
and also lists all the fields filterable, their type and the translation key to display it
    - e.g.: {ListKey:"widget",fields:[{fld:"name",lt:"WidgetName",type:"text"},{fld:"dollarAmount",lt:"WidgetDollarAmount",type:"currency"}]}
    - List key 
Certain types have extended abilities, for example dates have the classic floating AyaNova date ranges pre-defined or specific dates
Filters are saved to the database:
    - Filter: Name, UserId, Public, ListKey, Filter (Json string) 
        - ListKey is always lower case to match biz object list key name
        - Filter format:
            - Array [{fld:"fieldname",op:"See filtercomparisonoperator class",value:"compareValue"},...] these are all AND in sql, no OR 
            - fld=name of field to filter by
                - always all lower case 
                - Straight up field name like "name"
                - Could be compound for joins like "table.name" (no a prepends the table name)
                - Special indirect values such as "[TAGS]" which means cross filter with tags
            - op=one of the values specified in the FilterComparisonOperator class in Biz namespace
                - Note: no Like on purpose, that would require input, better to hard code a starts with, ends with, contains and not bothering with the negative of those three as I don't see it being widely used
            - value = string version of direct comparison value or could be a special token meaning more
                - Never an empty string, empty string is invalid value
                - All Tokens are a value surrounded by this fragment: "{[XXX]}" where XXX is the token
                - if empty or null then will be a token "*NULL*" 
                - If string then a string fragment, case is sensitive
                - If date then iso style date //RAVEN NOTE: it is my intention that dates come in iso8601 UTC format from the client
                - could be whole number or decimal number
                - Could be a special "macro" filter value like "{[THIS_MONTH]}" 
                - Could be a series of id values like this "[23,45,56,123]" as in tag id's or something related to that case.
   
Upon user selecting a filter to use the list query string has the regular paging info but also the filter id as a query parameter 
    - Server loads the filter if it's public or has the user ID if it's personal only
    - If list not found then it will return a 404 instead of the list    
    - server loads the filter, generates the sql based on the stored filter, adds the sort by and offset / page, runs the query and then returns the data as json collection to the client

--------------------------------------------------------
NOTES ABOUT WHY I DID THE FILTEROPTIONS LIKE I DID:
//Need a collection of fields and types and translation keys for
    //  - Client fetching filteroptions via list class
    //      - Just a static list of items localized and sent to the client
    //  - DataFilter validation via list class
    //      - static list of items to compare against
    //  - ToSql from DataFilter validation and query building via list class
    //      - Static list of filter options so that it can build sql fragment (needs to know type, possibly the field name [or that is inferred probably])
    //      - Ideally each list object will have it's own chunk of code to handle it's shit with tosql just making the fragments of sql like in v7
    //      - Again, a static list of items to check against



    //Where to store that?
    //all cases are via list class so really the whole thing is self contained and no need for an interface at all
--------------------------------------------------


SORTING
=-=-=-=-
Sorting is done same as filtering but property on datafilter is called Sort and it's also a json Array
Fields are: "fld" which is the same column name as in a datafilter and "dir" for direction which will contain a single character either a "-" or a "+" for descending and ascending respectively
If no Sort is defined the default will be by ID number descending meaning the most recent records in that table will be first

