admin/raven
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Browse Source
This commit is contained in:
John Cardinal
2020-06-18 23:34:27 +00:00
parent 1e9c3a83fb
commit 1e1553773c
12 changed files with 85 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.vscode/launch.json vendored
View File

@@ -40,11 +40,12 @@
"env": {
"ASPNETCORE_ENVIRONMENT": "Development",
"AYANOVA_JWT_SECRET": "UNLICENSED5G*QQJ8#bQ7$Xr_@sXfHq4",
"AYANOVA_SET_SUPERUSER_PW": "l3tm3in",
"AYANOVA_LOG_LEVEL": "Info",
//"AYANOVA_LOG_LEVEL": "Debug",
"AYANOVA_DEFAULT_TRANSLATION": "en",
//TRANSLATION MUST BE en for Integration TESTING
"AYANOVA_PERMANENTLY_ERASE_DATABASE": "true",
//"AYANOVA_PERMANENTLY_ERASE_DATABASE": "true",
"AYANOVA_DB_CONNECTION": "Server=localhost;Username=postgres;Password=raven;Database=AyaNova;",
"AYANOVA_USE_URLS": "http://*:7575;",
"AYANOVA_FOLDER_USER_FILES": "c:\\temp\\RavenTestData\\userfiles",

6
devdocs/todo.txt
View File

@@ -96,11 +96,17 @@ TODO: restrict server so randos can't login since the client now has all the log
not sure how to do that and still support phone via cellular network or other people's wifi from logging in
Firewall settings I guess of some kind or maybe require a manual edit to the password, like add a 1 to the end of all of them or something?
todo: Notification look for and implement //TODO: notify OPSNOTIFY
When notification system is in place
todo: OPS notification created for failed jobs
also maybe direct immediate email bypassing generator?
Add backup fail to this will stub out for now
todo: Look for the comment //todo in the server source code and in each case turn into a todo here instead or in addition or remove if no longer an isue
todo: (BREAK THIS OUT INTO LATER/NOW/CASES) there are several outstanding AUTHENTICATION related cases in rockfish for RAVEN
e.g. https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1924
https://rockfish.ayanova.com/default.htm#!/rfcaseEdit/1835

11
docs/8.0/ayanova/docs/ops-config-environment-variables.md
View File

@@ -34,13 +34,6 @@ These values can all be specified as an environment variable or as a command lin
- [AYANOVA_FOLDER_USER_FILES](ops-config-folder-user-files.md)
- [AYANOVA_FOLDER_BACKUP_FILES](ops-config-folder-backup-files.md)
## METRICS
## SECURITY
- [AYANOVA_METRICS_USE_INFLUXDB](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_BASEURL](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_DBNAME](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_CONSISTENCY](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_USERNAME](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_PASSWORD](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_RETENTION_POLICY](ops-metrics.md)
- [AYANOVA_METRICS_INFLUXDB_CREATE_DATABASE_IF_NOT_EXISTS](ops-metrics.md)
- [AYANOVA_SET_SUPERUSER_PW](ops-config-set-superuser-pw.md)

35
docs/8.0/ayanova/docs/ops-config-set-superuser-pw.md Normal file
View File

@@ -0,0 +1,35 @@
# SUPER USER PASSWORD OVERRIDE
In the event that the SuperUser password is lost it can be reset using this setting.
Upon booting up the server will reset the current SuperUser User account password to this value.
It should only be used to temporarily reset the password and never left in place.
## Default
This setting is not used by default.
## Overriding
AyaNova expects the override password to be provided by an environment variable or command line parameter named:
`AYANOVA_SET_SUPERUSER_PW`
The value specified should be a string used to temporarily override the super user password e.g.:
`Th3RainInSpainFallsMainlyOnTh3Pla1n`
Example command line parameter
`dotnet run --AYANOVA_SET_SUPERUSER_PW="Th3RainInSpainFallsMainlyOnTh3Pla1n"`
Example environment variable
Windows
`set "AYANOVA_SET_SUPERUSER_PW=Th3RainInSpainFallsMainlyOnTh3Pla1n"`
Linux / MAC
`export AYANOVA_SET_SUPERUSER_PW="Th3RainInSpainFallsMainlyOnTh3Pla1n"`
If both a command line parameter and an environment variable are set the command line parameter takes precedence.

53
docs/8.0/ayanova/docs/ops-metrics.md
View File

@@ -1,4 +1,4 @@
# METRICS
# METRICS (INCOMPLETE)
AyaNova 8+ automatically tracks server metrics for ongoing server maintenance, monitoring and troubleshooting.
@@ -9,55 +9,6 @@ This information is typically useful to the Operations staff who are responsible
When the AyaNova server is booted it starts gathering snapshots of statistical data during regular intervals that can be viewed to observe the current state of the server and some historical data from the point it was last rebooted.
Some examples of the metrics gathered include:
- Performance per API endpoint routes
- Error rates per HTTP error code and API endpoint route
- Transactions per endpoint
- Database records per table of significance
- Count and size of user files (attachments) stored at the server
- Count and size of operations files (backups, import/export etc) stored at the server
- Job operations data about background process jobs (notifications, backups, maintenance etc) running, succeeded and failed
- Memory usage of the server
- And more
todo: how to view metrics, what they mean what to use them for
## ROLES AND RIGHTS
Metrics are available to users with the `OPS - full` or `OPS - limited` roles.
## INFORMATION SECURITY AND PRIVACY
By design and policy no personally identifiable information is gathered for metrics. The data about API routes consists of consolidated information gathered over multiple users and does not track per IP address.
## VIEWING SNAPSHOT METRICS
View a current metrics snapshot directly on the server via the [API Explorer](api-console.md) tool:
![API Explorer](img/v8ServerMetricsSnapshotText.png)
TODO: VIEW METRICS IN AYANOVA CLIENT UI
## TAKING IT TO THE NEXT LEVEL - STORING METRICS AND VIEWING GRAPHICALLY
AyaNova has built in support to send metrics snapshots automatically to the open source time series database [InfluxDB](https://www.influxdata.com/) and can be viewed with the open source analytics and monitoring tool [Grafana](https://grafana.com/)
Example of a testing run of AyaNova during development visualized with Grafana and InfluxDB hosted in a Docker container:
![Grafana in Docker](img/v8ServerMetricsDashboard.png)
### Configuration settings for InfluxDB
Use of InfluxDB for metrics is controlled with [environment variables](ops-config-environment-variables.md) read during startup of the AyaNova server:
- `AYANOVA_METRICS_USE_INFLUXDB` true / false value, default is `false` set to `true` to turn on metrics reporting to InfluxDB
- `AYANOVA_METRICS_INFLUXDB_BASEURL` string value uri to your InfluxDB server default value is `http://127.0.0.1:8086`
- `AYANOVA_METRICS_INFLUXDB_DBNAME` string value name of database to use with InfluxDB server default value is `AyaNova`
- `AYANOVA_METRICS_INFLUXDB_CONSISTENCY` string value name of InfluxDB consistency policy to use with InfluxDB server default value is empty and not set
- `AYANOVA_METRICS_INFLUXDB_USERNAME` string value user name of account to connect to database default value is `root`
- `AYANOVA_METRICS_INFLUXDB_PASSWORD` string value password of account to connect to database default value is `root`
- `AYANOVA_METRICS_INFLUXDB_RETENTION_POLICY` string value name of InfluxDB retention policy to use with InfluxDB server default value is empty and not set
- `AYANOVA_METRICS_INFLUXDB_CREATE_DATABASE_IF_NOT_EXISTS` true / false value, default is `true` set to `true` to automatically create database in InfluxDB if it doesn't exist
### Setting up a Grafana dashboard
TODO: dashboard setup and mention of docker

3
server/AyaNova/Controllers/AttachmentController.cs
View File

@@ -468,7 +468,8 @@ namespace AyaNova.Api.Controllers
var filePath = FileUtil.GetPermanentAttachmentFilePath(dbObj.StoredFileName);
if (!System.IO.File.Exists(filePath))
{
//TODO: this should trigger some kind of notification to the ops people
//TODO: notify OPSNOTIFY
//TODO: notify this should trigger some kind of notification to the ops people
//and a red light on the dashboard
var errText = $"Physical file {dbObj.StoredFileName} not found despite attachment record, this file is missing";

2
server/AyaNova/Program.cs
View File

@@ -165,7 +165,7 @@ namespace AyaNova
//log configuration
try
{
var AyaNovaConfig = config.AsEnumerable().Where(z => z.Key.StartsWith("AYANOVA") && z.Key != "AYANOVA_JWT_SECRET").Select(z => z.Key + "=" + z.Value).ToList();
var AyaNovaConfig = config.AsEnumerable().Where(z => z.Key.StartsWith("AYANOVA") && z.Key != "AYANOVA_JWT_SECRET"&& z.Key != "AYANOVA_SET_SUPERUSER_PW").Select(z => z.Key + "=" + z.Value).ToList();
var DiagConfig = string.Join(",", AyaNovaConfig);
DiagConfig = DbUtil.PasswordRedactedConnectionString(DiagConfig);
logger.Info($"Config {DiagConfig}");

12
server/AyaNova/Startup.cs
View File

@@ -602,8 +602,18 @@ namespace AyaNova
_newLog.LogInformation($"License - [{AyaNova.Core.License.LicenseInfoLogFormat}]");
//Check for SuperUser password override
if (!string.IsNullOrWhiteSpace(ServerBootConfig.AYANOVA_SET_SUPERUSER_PW))
{
_newLog.LogWarning($"### AYANOVA_SET_SUPERUSER_PW IS PRESENT - RESETTING SUPERUSER PASSWORD NOW... ###");
AyaNova.Biz.UserBiz.ResetSuperUserPassword();
_newLog.LogWarning($"### AYANOVA_SET_SUPERUSER_PW HAS BEEN USED TO RESET SUPER USER PASSWORD YOU CAN REMOVE THIS SETTING NOW ###");
}
//Boot lock for generator
ServerGlobalOpsSettingsCache.BOOTING=false;
ServerGlobalOpsSettingsCache.BOOTING = false;
//Open up the server for visitors
_newLog.LogDebug("Setting server state open");

2
server/AyaNova/biz/JobsBiz.cs
View File

@@ -249,7 +249,7 @@ namespace AyaNova.Biz
catch (Exception ex)
{
log.LogError(ex, "JobsBiz::ProcessJobsAsync unexpected error during processing");
//TODO:OPSNOTIFY
//TODO: notify OPSNOTIFY
}
finally
{

14
server/AyaNova/biz/UserBiz.cs
View File

@@ -42,7 +42,7 @@ namespace AyaNova.Biz
}
//Called by license processor when use downgrades to lesser amount of techs
internal static async Task DeActivateExcessiveTechs(long KeepThisManyActiveTechs, ILogger _log)
internal static async Task DeActivateExcessiveTechs(long KeepThisManyActiveTechs, ILogger _log)
{
var TotalActiveTechs = await ActiveCountAsync();
int CountOfTechsToSetInactive = (int)(TotalActiveTechs - KeepThisManyActiveTechs);
@@ -64,6 +64,18 @@ namespace AyaNova.Biz
}
internal static void ResetSuperUserPassword()
{
using (AyContext ct = ServiceProviderProvider.DBContext)
{
User dbObj = ct.User.FirstOrDefault(z => z.Id == 1);
dbObj.Password = Hasher.hash(dbObj.Salt, ServerBootConfig.AYANOVA_SET_SUPERUSER_PW);
ct.SaveChanges();
//TODO: notify OPSNOTIFY
}
}
internal static UserBiz GetBiz(AyContext ct, Microsoft.AspNetCore.Http.HttpContext httpContext = null)
{
if (httpContext != null)

4
server/AyaNova/generator/CoreJobBackup.cs
View File

@@ -74,7 +74,7 @@ namespace AyaNova.Biz
{
await JobsBiz.LogJobAsync(Guid.Empty, $"Error during data backup \"{Result}\"");
log.LogError($"BACKUP ERROR: {Result}");
//TODO:OPSNOTIFY
//TODO: notify OPSNOTIFY
}
//DO FILE BACKUP IF ATTACHMENTS BACKED UP
@@ -97,7 +97,7 @@ namespace AyaNova.Biz
await JobsBiz.LogJobAsync(Guid.Empty, "Backup failed with errors:");
await JobsBiz.LogJobAsync(Guid.Empty, ExceptionUtil.ExtractAllExceptionMessages(ex));
log.LogError(ex, "Backup failed");
//TODO:OPSNOTIFY
//TODO: notify OPSNOTIFY
throw ex;
}
finally

11
server/AyaNova/util/ServerBootConfig.cs
View File

@@ -51,7 +51,9 @@ namespace AyaNova.Util
internal static string AYANOVA_LOG_LEVEL { get; set; }
internal static bool AYANOVA_LOG_ENABLE_LOGGER_DIAGNOSTIC_LOG { get; set; }
//SECURITY
internal static string AYANOVA_SET_SUPERUSER_PW { get; set; }
/// <summary>
/// Populate the config from the configuration found at boot
@@ -154,11 +156,14 @@ namespace AyaNova.Util
AYANOVA_FOLDER_BACKUP_FILES = config.GetValue<string>("AYANOVA_FOLDER_BACKUP_FILES");
//pgdump backup utility path
AYANOVA_BACKUP_PG_DUMP_PATH = config.GetValue<string>("AYANOVA_BACKUP_PG_DUMP_PATH");
AYANOVA_BACKUP_PG_DUMP_PATH = config.GetValue<string>("AYANOVA_BACKUP_PG_DUMP_PATH");
//backdoor back door password superuser reset
AYANOVA_SET_SUPERUSER_PW = config.GetValue<string>("AYANOVA_SET_SUPERUSER_PW");
#endregion server BASICS
}