diff --git a/CLAUDE.md b/CLAUDE.md index 982ab7aa..986e1029 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -25,10 +25,8 @@ hardcode these paths: do not move or rename the folders. | `raven-test-integration` | API integration tests against a running server | xUnit, .NET 8 | | `raven-launcher` | Windows standalone launcher | .NET 8, self-contained, trimmed | | `ravenqbi` | QuickBooks Desktop integration | .NET Framework 4.8 WinForms, x86, Visual Studio only | -| `sockeye` | Internal licensing server, a near-copy of this codebase, **already ported to .NET 10** (tag `8.1.0`) **[unverified: folder name]** | same as `raven` | - -Cypress E2E tests (Cypress 9.7.0, targeting remote devtest servers) also exist -**[unverified: which folder]**. +| `raven-test` | Cypress E2E suite in `e2e/` (Cypress 9.7.0, targets remote devtest servers); also JMeter (`load/`) and Locust (`loadlocust/`) load tests | Node, Cypress | +| `sockeye` | Internal licensing server, a near-copy of this codebase, **already ported to .NET 10** (tag `8.1.0`, baseline tag `net6-baseline`) | same as `raven` | ## Layout of this repo @@ -67,7 +65,7 @@ The rest of the modernization plan: `devdocs/modernization/ROADMAP.md`. ## Environment - Windows 11, PowerShell, VS Code. Visual Studio only for `ravenqbi`. -- .NET 10 SDK installed alongside the .NET 8 runtime **[unverified: check `dotnet --info`]**. +- .NET SDK 10.0.103 only (no 8 SDK; it still builds `net8.0`). Runtimes 8.0.24 and 10.0.3 (also 6.0.36). - Local server: `http://localhost:7575`, API base `/api/v8/`. - Compile-time constant `SUBSCRIPTION_BUILD` (`-p:SUBSCRIPTION_BUILD=true`) distinguishes the hosted build; use `#if SUBSCRIPTION_BUILD`. diff --git a/devdocs/build/BUILD.md b/devdocs/build/BUILD.md index 14f46209..f9fc2838 100644 --- a/devdocs/build/BUILD.md +++ b/devdocs/build/BUILD.md @@ -31,7 +31,7 @@ To set up a machine that can run this from scratch, see `DEV-VM-SETUP.md`. | MkDocs + mkdocs-material | current | Customer manual (`pip install mkdocs mkdocs-material`) | | Inno Setup | 6.x | Windows installers | | 7-Zip | current | Linux zips | -| Visual Studio | **[unverified: 2022 or 18/2026]** | QBI only | +| Visual Studio | 18 (2026) Community on the dev machine; no VS 2022 installed | QBI only | The scripts hardcode these paths: @@ -119,8 +119,9 @@ Set in five places for every release: ## Docker builds -`Dockerfile.frontend` and `Dockerfile.docs` build the front end and the manual in containers -**[unverified: where they live]**. The Linux server packages and the Windows +`raven-client\ayanova\Dockerfile.frontend` and `raven\Dockerfile.docs` build the front end +and the manual in containers. A runtime image and compose file for the Linux server are in +`raven\dist\docker\linux-x64\`. The Linux server packages and the Windows framework-dependent build could also build in Docker. Windows self-contained builds, Inno Setup, QBI and the embedded PostgreSQL need Windows. diff --git a/devdocs/build/DEV-VM-SETUP.md b/devdocs/build/DEV-VM-SETUP.md index 4a65139e..607921c6 100644 --- a/devdocs/build/DEV-VM-SETUP.md +++ b/devdocs/build/DEV-VM-SETUP.md @@ -636,7 +636,7 @@ msbuild C:\data\code\ravenqbi\AyaNovaQBI\AyaNovaQBI.csproj /p:Configuration=Rele |------|------| | 7-Zip | `C:\Program Files\7-Zip\7z.exe` | | Inno Setup | `C:\Program Files (x86)\Inno Setup 6\ISCC.exe` | -| MSBuild | `C:\Program Files\Microsoft Visual Studio\18\Community\MSBuild\Current\Bin\MSBuild.exe` **[unverified]**: 6.3 was changed to `18` (Visual Studio 2026) while 4.6 still says install VS 2022. Confirm which is installed | +| MSBuild | `C:\Program Files\Microsoft Visual Studio\18\Community\MSBuild\Current\Bin\MSBuild.exe`. Confirmed on the dev machine (VS 18 / 2026; no VS 2022). Section 4.6 still says install VS 2022: install VS 2026 instead | --- diff --git a/devdocs/modernization/CODEBASE-REFERENCE.md b/devdocs/modernization/CODEBASE-REFERENCE.md index 3082bad4..a19bb3de 100644 --- a/devdocs/modernization/CODEBASE-REFERENCE.md +++ b/devdocs/modernization/CODEBASE-REFERENCE.md @@ -18,7 +18,7 @@ truth**. Anything marked **[unverified]** has not been checked against the code. | MailKit | 4.5.0 | Email | | Swashbuckle.AspNetCore | 6.5.0 | OpenAPI | | NLog.Web.AspNetCore | 5.3.11 | Logging | -| Asp.Versioning.Mvc | — | API versioning **[unverified]** | +| Asp.Versioning.Mvc | 8.1.0 | API versioning (with `Asp.Versioning.Mvc.ApiExplorer`) | ### Structure @@ -26,14 +26,21 @@ truth**. Anything marked **[unverified]** has not been checked against the code. raven/server/AyaNova/ ├── AyaNova.csproj ├── Program.cs, Startup.cs +├── biz/ business objects (…Biz.cs), report rendering (ReportBiz.cs) +├── ControllerHelpers/ +├── Controllers/ API controllers +├── DataList/ data-list definitions and the SQL filter builder +├── generator/ background jobs (CoreJob…: backup, licence, sweeper …) +├── kpi/ +├── models/ EF Core context and entities +├── PickList/ +├── util/ ServerBootConfig, DbUtil, License, FileUtil, Mailer … ├── resource/ report templates and static resources, copied to output │ └── rpt/ report templates (.ayrt) └── wwwroot/ built front end and customer manual, copied in by the build ``` -Older notes also list `Controllers/`, `Models/`, `Services/` and `Data/` folders -**[unverified]**: that list reads like a generic template, not the actual layout. -Data-list classes live in the `AyaNova.DataList` namespace, and their class names are the +There are no `Services/` or `Data/` folders. Data-list classes live in the `AyaNova.DataList` namespace, and their class names are the list keys (`WorkOrderDataList`, `CustomerDataList` …). ### Hosted vs self-hosted @@ -50,19 +57,24 @@ list keys (`WorkOrderDataList`, `CustomerDataList` …). ### Reports -PuppeteerSharp renders PDFs in headless Chrome, downloaded on first run. A customer -setting holds extra browser launch parameters, and an environment variable can point at -an existing browser instead of downloading one (`REPORT_RENDER_BROWSER_PATH` in older -notes **[unverified]**). +PuppeteerSharp renders PDFs in headless Chrome, downloaded on first run +(`biz/ReportBiz.cs`). Two configuration keys control it: `AYANOVA_REPORT_RENDER_BROWSER_PATH` +points at an existing browser instead of downloading one, and +`AYANOVA_REPORT_RENDER_BROWSER_PARAMS` replaces the default launch arguments +(`util/ServerBootConfig.cs:84-88`, `:185-191`). ### Configuration -Older notes list `ConnectionStrings__DefaultConnection`, `JWT_SECRET` and -`ASPNETCORE_ENVIRONMENT` as the server's settings **[unverified]**. Sockeye, the near-copy, -reads `SOCKEYE_*` keys from a `config.json` beside the app (`SOCKEYE_DB_CONNECTION`, -`SOCKEYE_USE_URLS`, `SOCKEYE_JWT_SECRET`, `SOCKEYE_DATA_PATH` …), so AyaNova likely does the -same with its own prefix. The Linux packages ship a `config.json` from -`dist/assets/linux-server/` or `dist/assets/linux-desktop/`, which supports that. +`Program.cs:38` reads `config.json` beside the app, then environment variables, then the +command line (later sources win). `util/ServerBootConfig.cs` reads `AYANOVA_*` keys: +`AYANOVA_DB_CONNECTION`, `AYANOVA_USE_URLS`, `AYANOVA_JWT_SECRET`, `AYANOVA_DATA_PATH`, +`AYANOVA_LOG_PATH`, `AYANOVA_LOG_LEVEL`, `AYANOVA_ATTACHMENT_FILES_PATH`, +`AYANOVA_BACKUP_FILES_PATH`, `AYANOVA_TEMP_FILES_PATH`, `AYANOVA_BACKUP_PG_DUMP_PATH`, +`AYANOVA_DEFAULT_TRANSLATION`, `AYANOVA_SET_SUPERUSER_PW`, `AYANOVA_REPORT_RENDER_*`, plus +boot flags such as `AYANOVA_PERMANENTLY_ERASE_DATABASE` and `AYANOVA_REMOVE_LICENSE_FROM_DB`. +The names in older notes (`ConnectionStrings__DefaultConnection`, `JWT_SECRET`) were wrong. +The Linux packages ship a `config.json` from `dist/assets/linux-server/` or +`dist/assets/linux-desktop/`; the LAN installer uses `dist/assets/lan-install-config.json`. ### API conventions diff --git a/devdocs/modernization/NET10-PORT.md b/devdocs/modernization/NET10-PORT.md index 9028145e..ce048fc1 100644 --- a/devdocs/modernization/NET10-PORT.md +++ b/devdocs/modernization/NET10-PORT.md @@ -11,7 +11,7 @@ under "Deferred", not into the branch. | Front end | Minimal or no change. Rebuild only if needed (open question) | | Branch | `net10-port` in `raven` (and `raven-client` and `raven-test-integration` and `raven-launcher`) | | Baseline tag | `net8-baseline`, on the last commit before the port | -| Release version | Not yet decided (open question) | +| Release version | 8.3.0 | ## Where this plan comes from @@ -45,60 +45,66 @@ them. Items touching those three areas carry more weight here than they did in S - Version bump first. One concern per commit. - Prove each gate on the **unmodified** code before changing anything, so a later failure points at the port, not at the test. -- Packages that must move together: framework-matched `Microsoft.*` packages all go to 10.x in one step. `BouncyCastle.Cryptography` 2.x goes with MailKit/MimeKit 4.x, because MimeKit 3.x pulls in `Portable.BouncyCastle` and the assemblies clash. +- Packages that must move together: framework-matched `Microsoft.*` packages all go to 10.x in one step. (The BouncyCastle/MimeKit clash from Sockeye does not arise here: AyaNova is already on `BouncyCastle.Cryptography` 2.x with MailKit/MimeKit 4.x.) - Publish `server\AyaNova\AyaNova.csproj` directly, never the solution, so no test project can reach the artifact. --- ## 0. Before changing anything -- [ ] Record any port work already done in earlier sessions (branch, commits) here first, and reconcile it with this list +- [x] Record any port work already done in earlier sessions (branch, commits) here first, and reconcile it with this list. Done: branch `net10-port` and the 8.3.0 version bump (`c2f2f922`); no code changes yet - [ ] `raven` and `raven-client` clean on `main` and pushed -- [ ] Annotated tag `net8-baseline` on the pre-port commit in `raven` and `raven-client`, pushed by name (`git push origin net8-baseline`), checked with `git ls-remote --tags origin` -- [ ] Branch `net10-port` created and pushed with upstream -- [ ] **First commit: version bump** in every place listed in `devdocs/build/BUILD.md` → "Version numbers" -- [ ] Dev machine has the .NET 10 SDK and keeps the .NET 8 runtime for baseline comparisons +- [x] Annotated tag `net8-baseline` on the pre-port commit in `raven` and `raven-client`. Present locally in both; confirm on the remote with `git ls-remote --tags origin` +- [x] Branch `net10-port` created and pushed with upstream (`origin/net10-port` exists in `raven`) +- [x] **First commit: version bump** in every place listed in `devdocs/build/BUILD.md` → "Version numbers" (`c2f2f922`, 8.3.0) +- [x] Dev machine has the .NET 10 SDK and keeps the .NET 8 runtime for baseline comparisons. SDK 10.0.103 (no 8 SDK; the 10 SDK builds `net8.0`); runtimes 8.0.24 and 10.0.3 ### Gates (prove on the unmodified code) - [ ] **Integration suite baseline.** Run `raven-test-integration` against the unmodified build and record the counts in the log below. Expected: everything passes except the three known server-side failures listed in that repo's CLAUDE.md -- [ ] **Licence verification gate.** Find out exactly how AyaNova 8 verifies a v8 licence: library, public key, and when it re-checks. If it uses BouncyCastle, write a test that runs every v8 row of Sockeye's `alicense` fixture through AyaNova's own verification routine. The fixture is customer data: it stays outside the repo and is found through an environment variable, as Sockeye's re-sign harness does with `SOCKEYE_RESIGN_FIXTURE`. Must be green on unmodified code. A regression here would reject licences customers already have installed +- [ ] **Licence verification gate.** Write a test that runs every v8 row of Sockeye's `alicense` fixture through AyaNova's own verification routine. The fixture is customer data: it stays outside the repo and is found through an environment variable, as Sockeye's re-sign harness does with `SOCKEYE_RESIGN_FIXTURE`. Must be green on unmodified code. A regression here would reject licences customers already have installed. How AyaNova verifies: + - **Library:** BouncyCastle (`util/License.cs:15-17`). `PemReader` reads a SubjectPublicKeyInfo PEM (`BEGIN PUBLIC KEY`, not PKCS#1) into an `AsymmetricKeyParameter`; `SignerUtilities.GetSigner("SHA256WITHRSA")` verifies (`License.cs:968-978`) + - **Public key:** hardcoded RSA-2048 PEM, "our real 2016 public key", `License.cs:957-965` + - **Signed payload:** the JSON between `[KEY` and `KEY]`, with whitespace outside strings stripped by regex (`License.cs:950`); base64 signature between `[SIGNATURE` and `SIGNATURE]` (`:951`) + - **Other checks in `Parse`** (`License.cs:926-1062`): `LicenseFormat == "8"`; `DBID` equals the server's schema id; perpetual key only on a perpetual build and subscription key only on a subscription build; `MExBB`, which rejects a perpetual non-trial key whose maintenance expiry is before **the build date, read from the PE linker timestamp** (`util/FileUtil.cs:861-882`). That timestamp is only a real date because `False` (`AyaNova.csproj:12`). Turning determinism on, or getting the CA2022 change at `FileUtil.cs:875` wrong, would reject perpetual licences with E1020. The gate must cover `MExBB`, not only the signature + - **When it runs:** at boot, `BootSafetyCheck` before any schema update (`Startup.cs:603`), then `InitializeAsync` (`Startup.cs:633`); after every fetch, `FetchKeyAsync` → `InstallAsync` → `InitializeAsync`, either manually (`LicenseController.cs:125`) or from `CoreJobLicense` (`generator/CoreJobLicense.cs:80`). Fetch interval (`CoreJobLicense.cs:21-34`): perpetual build 30 min while unlicensed, trial or expired, otherwise 24 h; subscription build 30 min; DEBUG 1 or 5 min. The installed key is not re-verified on a schedule; expiry is checked in CoreJobSweeper, according to the comment at `CoreJobLicense.cs:10` + - **Test access:** `Parse` is `private static` and depends on the static `ServerDbId` and on the running assembly's timestamp. The test needs `InternalsVisibleTo` or reflection - [ ] **Report baseline.** Render the advanced reports on the unmodified build and keep the PDFs for comparison. Sockeye shares only one report with AyaNova, so Sockeye's testing says little about AyaNova's reports -- [ ] **E2E baseline.** Cypress run against a devtest server on the unmodified build +- [ ] **E2E baseline.** Cypress run (`raven-test\e2e`) against a devtest server on the unmodified build ## 1. Framework and packages -- [ ] `net8.0` → `net10.0`, with every framework-matched `Microsoft.*` package to 10.x in the same step -- [ ] **Hosting model.** `WebHost`, `IWebHost` and `WebHostBuilder` are obsolete in .NET 10 (ASPDEPR008, ASPDEPR004). `Host.CreateDefaultBuilder(args).ConfigureWebHostDefaults(web => web.UseStartup())` keeps the `Startup` class unchanged -- [ ] **Asp.Versioning.** Older notes say AyaNova already uses `Asp.Versioning.Mvc` **[unverified]**. If not: rename from `Microsoft.AspNetCore.Mvc.Versioning`; `ApiVersion` moves to `Asp.Versioning`; wiring is `services.AddApiVersioning(...).AddMvc().AddApiExplorer(...)`. Leave AV0014 warnings: **do not add `[ApiController]`** +- [ ] `net8.0` → `net10.0`, with every framework-matched `Microsoft.*` package to 10.x in the same step (`AyaNova.csproj:3`; `raven-launcher.csproj:5`) +- [x] **Hosting model.** n/a: `Program.cs:330-351` already uses `Host.CreateDefaultBuilder(args).ConfigureWebHostDefaults(...)` with `UseStartup`. Only a commented-out `BuildWebHost` line remains (`:308`) +- [ ] **Asp.Versioning.** Already done: `Asp.Versioning.Mvc` and `.ApiExplorer` 8.1.0 (`AyaNova.csproj:36-37`), wired at `Startup.cs:200-217` as `AddApiVersioning(...).AddApiExplorer(...)`. Remaining: bump to 10.x and confirm `AddApiExplorer` still registers MVC versioning without an explicit `.AddMvc()`. Leave AV0014 warnings: **do not add `[ApiController]`** - [ ] **Npgsql / EF Core 10.** - - Check `Npgsql.EnableLegacyTimestampBehavior`. Sockeye's was commented out. If AyaNova sets it, `DateTime.Kind` is a real port item - - Check for `ConfigureWarnings(w => w.Throw())`. If present, every new EF warning becomes an exception naming its ID: fix the query, or move that ID to `.Log(...)` - - Exercise LINQ-based queries screen by screen. Data-list filters build raw SQL and are unaffected -- [ ] **BouncyCastle.** If AyaNova references `BouncyCastle`, it becomes `BouncyCastle.Cryptography` 2.x, moved together with MailKit/MimeKit 4.x. In Sockeye, `PemReader` still yields an `AsymmetricCipherKeyPair` for a PKCS#1 key -- [ ] **Swashbuckle 10 / OpenAPI.NET 2.** `Microsoft.OpenApi.Models` merged into `Microsoft.OpenApi`. Bearer requirement becomes `c.AddSecurityRequirement(document => new OpenApiSecurityRequirement { [new OpenApiSecuritySchemeReference("Bearer", document)] = [] })`. No explicit `Microsoft.OpenApi` reference -- [ ] **NLog 6.** `ArchiveNumbering` is gone; `NLogBuilder.ConfigureNLog` is replaced by `NLog.LogManager.Configuration = logConfig`. Archive numbers keep incrementing and the un-numbered file is always current. If the log viewer in the UI sorts by file name, the order breaks past 99 -- [ ] **Newtonsoft.Json.** `Mvc.NewtonsoftJson` 10.x sets a 13.0.3 floor (NU1605). Pin it as a direct reference -- [ ] **JwtBearer 10.x**, which also retires the vulnerable IdentityModel 6.10.0 dependency if AyaNova still resolves it -- [ ] **SMTP.** Any use of `System.Net.Mail.SmtpClient` (SYSLIB0014) moves to MailKit 4.x, as in Sockeye and contact -- [ ] **PuppeteerSharp → 25.x.** Reporting is AyaNova's main Puppeteer surface - - `BrowserFetcher.DefaultChromiumRevision` is gone: call `DownloadAsync()` with no argument - - `IJSHandle.ExecutionContext` is gone: evaluate on the handle. Evaluate results are `JsonElement?` + - [x] `Npgsql.EnableLegacyTimestampBehavior`: n/a, commented out (`Program.cs:26`) + - [ ] `ConfigureWarnings(w => w.Throw())` **is present** (`Startup.cs:179`). Every new EF warning becomes an exception naming its ID: fix the query, or move that ID to `.Log(...)` + - [ ] Exercise LINQ-based queries screen by screen. Data-list filters build raw SQL and are unaffected +- [ ] **BouncyCastle.** Already `BouncyCastle.Cryptography` 2.3.1 (`AyaNova.csproj:28`) with MailKit/MimeKit 4.5.0; no `Portable.BouncyCastle` in the graph. Version bump only. The Sockeye note about `PemReader` and PKCS#1 key pairs does not apply: AyaNova reads a public key (see the licence gate) +- [ ] **Swashbuckle 10 / OpenAPI.NET 2.** Applies. `Microsoft.OpenApi.Models` merged into `Microsoft.OpenApi`: `using` at `Startup.cs:15` and `ConfigureSwaggerOptions.cs:6`. Bearer requirement (`Startup.cs:235-244`) becomes `c.AddSecurityRequirement(document => new OpenApiSecurityRequirement { [new OpenApiSecuritySchemeReference("Bearer", document)] = [] })`. Remove the explicit `Microsoft.OpenApi` 1.6.14 reference (`AyaNova.csproj:47`) +- [ ] **NLog 6.** `NLogBuilder.ConfigureNLog` is already replaced (`Program.cs:206`, `LogManager.Setup().LoadConfiguration`). `ArchiveNumbering = Rolling` (`Program.cs:136`) is gone in 6. Archive numbers keep incrementing and the un-numbered file is always current. **The log viewer sorts by file name as a string** (`LogFilesController.cs:120`, `OrderByDescending(z => z)`), so `-10` sorts after `-2`. Fix the sort in the same commit +- [ ] **Newtonsoft.Json.** `Mvc.NewtonsoftJson` 10.x sets a 13.0.3 floor (NU1605). Already resolves 13.0.3 transitively; add it as a direct reference +- [ ] **JwtBearer 10.x.** The IdentityModel part is n/a: it resolves 7.1.2, not the vulnerable 6.10.0 +- [x] **SMTP.** n/a: `util/Mailer.cs` already uses MailKit's `SmtpClient` (`:1`, `:84`); no `System.Net.Mail` +- [ ] **PuppeteerSharp 17 → 25.x.** Reporting is AyaNova's main Puppeteer surface (`biz/ReportBiz.cs`) + - [x] `BrowserFetcher.DefaultChromiumRevision`: n/a, already `DownloadAsync()` with no argument (`ReportBiz.cs:800`, case 4586) + - [x] `IJSHandle.ExecutionContext`: n/a, the only use is commented out (`ReportBiz.cs:858-860`). Evaluate results are `JsonElement?`: check any remaining `Evaluate*` call sites - The default download is Chrome for Testing, into `Chrome/` and `ChromeHeadlessShell/` beside the app. AyaNova 8.2.0 already moved from Chromium to Chrome to follow Puppeteer's default, so the browser choice is settled - - Launch arguments must be separate array elements (`--no-sandbox` on its own). **This includes the report-render browser params setting** customers can edit + - [ ] Launch arguments must be separate array elements. **Both paths are wrong today:** the defaults are one element `"--headless --no-sandbox "` (`ReportBiz.cs:812`, `:829`), and `AYANOVA_REPORT_RENDER_BROWSER_PARAMS` is passed whole as one element (`:819-822`) - AppArmor restricts unprivileged user namespaces on Ubuntu 24.04+, which affects the Chrome sandbox -- [ ] Remaining packages to current releases -- [ ] Warnings: CA2022 → `ReadExactly`; ASP0019 → header indexer instead of `Headers.Append` -- [ ] **MiniProfiler.** Confirm it is absent: `git grep -i -e miniprofiler -e mini-profiler -e StackExchange.Profiling` in **both** `raven` and `raven-client` -- [ ] **BinaryFormatter.** Confirm it is unused, or that `EnableUnsafeBinaryFormatterSerialization` is already false -- [ ] **`System.GC.Server`.** Keep the Web SDK default (`true`). On one vCPU the runtime uses workstation GC anyway, and DATAS keeps server GC lean since .NET 9 -- [ ] **`InvariantGlobalization`.** Decide on EF Core string comparison and date-parsing grounds. Still open in Sockeye too +- [ ] Remaining packages to current releases. MailKit 4.5.0 carries a moderate advisory (NU1902). After the upgrade, check what still pulls in `System.Text.Json` 5.0.2 +- [ ] Warnings: CA2022 → `ReadExactly` at `FileUtil.cs:875` (**licence build-date path**: covered by the licence gate); ASP0019 → header indexer instead of `Headers.Append` at `Startup.cs:514` +- [ ] **MiniProfiler. Present, not absent.** `MiniProfiler.AspNetCore.Mvc` and `.EntityFrameworkCore` 4.3.8 (`AyaNova.csproj:48-49`); `AddMiniProfiler(...).AddEntityFramework()` (`Startup.cs:62-74`); auth workaround for `/profiler/results` (`Startup.cs:393-441`); `UseMiniProfiler()` (`Startup.cs:530`); `using StackExchange.Profiling.Internal` (`ReportBiz.cs:14`). Nothing in `raven-client`. Keep and upgrade, or remove: see Open questions +- [x] **BinaryFormatter.** n/a: not used anywhere in `server/AyaNova` +- [x] **`System.GC.Server`.** n/a: not set, so the Web SDK default (`true`) stays. On one vCPU the runtime uses workstation GC anyway, and DATAS keeps server GC lean since .NET 9 +- [ ] **`InvariantGlobalization`.** Not set. Decide on EF Core string comparison and date-parsing grounds. Note `DataListSqlFilterCriteriaBuilder.cs:835` reads `CurrentCulture.NumberFormat`. Still open in Sockeye too ## 2. Configuration behaviour to check -- [ ] **URL binding precedence.** An in-code `UseUrls` beats `ASPNETCORE_URLS`. Find where AyaNova takes its URLs from (Sockeye: `SOCKEYE_USE_URLS` in `config.json`) and make sure the documented setting is the one that wins -- [ ] **Logging providers.** `appsettings.json` very likely sets the Debug and Console providers to `Trace`, as Sockeye's does. Check whether `Program.cs` calls `ClearProviders()` before adding NLog. If it does not, `Trace` floods the journal under systemd. Fix the levels in the repo either way -- [ ] **Data Protection keys.** Only matters if AyaNova encrypts anything with Data Protection and runs under a service account with no home directory: then the key ring must be persisted explicitly. A JWT-only API is unaffected +- [x] **URL binding precedence.** n/a: `UseUrls(ServerBootConfig.AYANOVA_USE_URLS)` (`Program.cs:340`), read from `config.json`, environment and command line (`Program.cs:38`, `ServerBootConfig.cs:167-170`). The documented `AYANOVA_USE_URLS` is the one that wins over `ASPNETCORE_URLS`; a command-line `--urls` still overrides it through `UseConfiguration` (`Program.cs:341`) +- [ ] **Logging providers.** No journal flood: `Program.cs:348` calls `ClearProviders()` before `UseNLog()`. `appsettings.json:4-13` still sets the Debug and Console providers to `Trace`, with no effect today. Fix the levels in the repo +- [x] **Data Protection keys.** n/a: nothing uses Data Protection; authentication is JWT only (`Startup.cs:278-302`) ## 3. Fixes found through Sockeye (each its own commit) @@ -106,14 +112,14 @@ These are bugs, not port mechanics, so each is a separate commit that can be rev Sockeye included its equivalents in the port branch. John decides per item whether they go in this branch or a follow-up release. UPDATE: John has decided these all go in this branch and some more additional cases in the case database that are urgent, exact cases TBD. -- [ ] **Case 4676: data-list SQL injection — priority.** The numeric and role filters in `DataListSqlFilterCriteriaBuilder` built SQL from input. In AyaNova any authenticated user who can open a data list could reach them. Port the Sockeye fix -- [ ] **`DbUtil.AdminConnectionString`.** If it is `_dbConnectionString.Replace(_dbName, "postgres")`, any customer whose role name, password or host contains the database name as text gets a mangled admin connection, and the boot check authenticates as the wrong user. Fix: `new NpgsqlConnectionStringBuilder(cs) { Database = "postgres" }.ConnectionString` -- [ ] **Backup `pg_dump` call.** If it builds a `postgresql://user:password@host:port/db` URI with the raw password and runs it through a shell for a `> file` redirect, a password with spaces or `@ : / ? # %` breaks backups. Fix: password in `PGPASSWORD` in the process environment, connection parameters as separate arguments through `ProcessStartInfo.ArgumentList` (no shell), output with `-f` -- [ ] **Drop-and-recreate database.** Without `CREATEDB` it drops the database and then fails to create it. Test the permission before dropping, or change the order -- [ ] **Report browser download location.** Point it under the data directory in code. Beside the app it is read-only under a hardened systemd unit (`ProtectSystem=strict`), and a redeploy that replaces the app folder deletes it -- [ ] **Report failures must not kill the server.** On the Sockeye destination a failed browser download was an unhandled exception that ended the whole process (systemd `core-dump`, then restart); the client saw only `TypeError … reading 'length'`. Catch download and launch failures, log them through NLog, and return an error to the client -- [ ] **Data-list date filters, small fixes.** `>=` and `<` should use `ZeroSeconds`, not `MaxSeconds`; date `NotEqual` should exclude the whole minute (`< ZeroSeconds OR > MaxSeconds`). In DEBUG builds, `DataListField.UiFieldDataType` is read before `DataListField` is null-checked -- [ ] **Relative date tokens use the server's time zone** (`*today*`, `*yesterday*` …), which is wrong for any user in a different zone from the server. The fix builds the range from `clientTimeStamp.Offset`, with care for ranges that cross a DST change. **This changes behaviour** and affects the integration tests' `TIME_ZONE_ADJUSTMENT`. Probably better as a follow-up release +- [ ] **Case 4676: data-list SQL injection — priority.** Confirmed: `DataListSqlFilterCriteriaBuilder` appends the raw filter value for Roles (`:343-354`) and for every numeric type (Enum, MemorySize, Decimal, Currency, InternalId, Integer, `:866-888`). Text (`:233`) and tag (`:1024`) values escape `'`; TimeSpan goes through `int.Parse` (`:1099+`). Any authenticated user who can open a data list can reach this. Port the Sockeye fix +- [ ] **`DbUtil.AdminConnectionString`.** Confirmed: `DbUtil.cs:99` is `_dbConnectionString.Replace(_dbName, "postgres")`, used by the boot check (`:192`), create (`:286`) and drop/recreate (`:334`). Any customer whose role name, password or host contains the database name as text gets a mangled admin connection, and the boot check authenticates as the wrong user. Fix: `new NpgsqlConnectionStringBuilder(cs) { Database = "postgres" }.ConnectionString` +- [ ] **Backup `pg_dump` call.** Confirmed: `CoreJobBackup.cs:68` builds `postgresql://user:password@host:port/db` with the raw password, and `:94` adds `-Fc > file`; `RunProgram` runs it through `cmd.exe /C` (`RunProgram.cs:55`) or `/bin/sh -c` (`:82-89`). A password with spaces or `@ : / ? # %` breaks backups. Also, on Linux stderr is not redirected (`RunProgram.cs:86-92`), so a failed pg_dump returns empty output and is **reported as a successful backup** (`CoreJobBackup.cs:97`). Fix: password in `PGPASSWORD` in the process environment, connection parameters as separate arguments through `ProcessStartInfo.ArgumentList` (no shell), output with `-f`, stderr and exit code checked +- [ ] **Drop-and-recreate database.** Confirmed: `DbUtil.cs:342-345` drops, then creates; triggered by `AYANOVA_PERMANENTLY_ERASE_DATABASE` (`Startup.cs:567-570`). Without `CREATEDB` it drops the database and then fails to create it. Test the permission before dropping, or change the order +- [ ] **Report browser download location.** Confirmed: `new BrowserFetcher()` with the default path beside the app (`ReportBiz.cs:800`). Point it under the data directory in code. Beside the app it is read-only under a hardened systemd unit (`ProtectSystem=strict`), and a redeploy that replaces the app folder deletes it +- [ ] **Report failures: return a specific error.** The Sockeye crash is n/a: rendering runs as a job, and `JobsBiz.ProcessJobAsync` catches every exception, logs it through NLog and marks the job Failed (`JobsBiz.cs:390-439`), so the process survives. What remains: a browser download or launch failure (`ReportBiz.cs:800`, `:838`, outside the render `try`) reaches the client only as a generic job failure. Catch it in `DoRenderJob` and return a render error that says what failed +- [ ] **Data-list date filters, small fixes.** Confirmed: `>=` (`:790-793`) and `<` (`:795-798`) use `MaxSeconds` and should use `ZeroSeconds`; date `NotEqual` (`:806-814`) uses `MaxSeconds` on both sides and should exclude the whole minute (`< ZeroSeconds OR > MaxSeconds`). `DataListField` is dereferenced at `:24` and `:27` before the DEBUG-only null check at `:33`, so **release builds** throw a NullReferenceException on an unknown column, not only DEBUG builds +- [ ] **Relative date tokens use the server's time zone** (`*today*`, `*yesterday*` …). Confirmed: `ClientToday = clientTimeStamp.Date` (`:399-400`) has `Kind=Unspecified`, and `ToUniversalTime()` (`:1060`, `:1065`) converts it with the server's zone. Wrong for any user in a different zone from the server. The fix builds the range from `clientTimeStamp.Offset`, with care for ranges that cross a DST change. **This changes behaviour** and affects the integration tests' `TIME_ZONE_ADJUSTMENT`. Probably better as a follow-up release Not port items, listed so they are not forgotten: the three server bugs the integration suite is waiting on (case 4653, case 4654, the contract issue). @@ -125,7 +131,7 @@ suite is waiting on (case 4653, case 4654, the contract issue). - [ ] **Reports, all advanced reports**, compared with the baseline PDFs. On Windows and on Linux. On Linux also: `ldd` on the downloaded browser binary for missing libraries (install exactly what is reported; several Ubuntu 24.04+ names carry a `t64` suffix); the sandbox under the service account; no orphaned `chrome` processes after a cancelled render - [ ] Integration suite: same result as the baseline - [ ] Licence verification gate: green -- [ ] Cypress E2E against a devtest server running the .NET 10 build +- [ ] Cypress E2E (`raven-test\e2e`) against a devtest server running the .NET 10 build - [ ] `build-release.bat` produces all five packages; install and start each one: Windows standalone, Windows LAN, Linux server, Linux desktop, subscription Linux ## 5. Release @@ -160,20 +166,22 @@ suite is waiting on (case 4653, case 4654, the contract issue). ## Open questions - Release version number (Sockeye's port was released as 8.1.0) Answer: 8.3.0 will be the version -- `raven-launcher` (.NET 8, self-contained): move to .NET 10 in the same release? Answer: Yes +- `raven-launcher` (.NET 8, self-contained): move to .NET 10 in the same release? Answer: Yes. It has no package references, so this is the target framework only - `raven-test-integration`: move to `net10.0` (separate commit) or leave on `net8.0`? It only talks HTTP to the server Answer: Move to net10.0 for consistency - Does the client need a rebuild in this pass, or can its current output be carried across? Answer: yes it will have minor changes as well as version number to 8.3.0 - Section 3: which fixes go in this branch, which in a follow-up - How the .NET 10 runtime gets onto Ubuntu 24.04 hosts +- MiniProfiler (section 1): keep it and upgrade to 4.5.x (check that `MiniProfiler.EntityFrameworkCore` supports EF Core 10), or remove it in its own commit? ## Deferred Found during the port, out of scope. One line each. -- +- `License.DoFetchAsync` (`License.cs:629-661`) rethrows at `x == 3`, so the fifth server (Sockeye) is never tried, and it can return `null` to a caller that dereferences it +- Stray `raven-client\ayanova\cypress.json`: the Cypress suite lives in `raven-test\e2e` ## Log | Date | Step | Result | |---|---|---| -| | | | +| 2026-10-22 | Survey of sections 1–3 against the code (read-only) | Findings recorded per item: n/a items ticked with reasons, evidence added, MiniProfiler found present | diff --git a/devdocs/modernization/ROADMAP.md b/devdocs/modernization/ROADMAP.md index d322d71a..7b728ef4 100644 --- a/devdocs/modernization/ROADMAP.md +++ b/devdocs/modernization/ROADMAP.md @@ -8,12 +8,12 @@ the original plan (February–March 2026). Statuses are as of October 2026; any |---|---|---| | — | SVN → Git, self-hosted Gitea | ✅ Done | | — | Build process documented; Docker builds for front end and docs; Windows build VM exported and backed up | ✅ Done | -| A1 | E2E tests with Cypress 9.7.0, targeting remote devtest servers, with version validation and environment-based config | **[unverified]**: the March notes listed it as both done and in progress | +| A1 | E2E tests with Cypress 9.7.0, targeting remote devtest servers, with version validation and environment-based config | **[unverified]**: the suite exists in `raven-test\e2e` (last commit 2026-02-22), but the March notes listed it as both done and in progress | | A2 | Fix the outdated integration tests in `raven-test-integration` | ✅ Done, March 2026: compiles on `net8.0`, broken tests removed or rewritten | -| A3 | Data-list filter and sort integration tests | ✅ Done, March 2026, together with Tier 1–3 gap coverage. February notes still called them stubs; the March notes supersede that **[unverified]** | +| A3 | Data-list filter and sort integration tests | ✅ Done, March 2026, together with Tier 1–3 gap coverage. `raven-test-integration\DataList\DataListOperations.cs` holds 15 real tests, not stubs | | — | Fix the three server bugs the integration suite is waiting on: case 4653, case 4654, the contract issue | Open | | ▶ | **.NET 10 port of the back end** | **Current**: `NET10-PORT.md` | -| B | Git baseline: tag plus refactor branch | **[unverified]**. The port's `net8-baseline` tag covers the tag half | +| B | Git baseline: tag plus refactor branch | Tag half done: `net8-baseline` exists in `raven` and `raven-client`. Refactor branch **[unverified]** | | C | Architecture review | Not started | | D | Refactor: remove redundant code, simplify, improve performance. No behaviour changes | Not started | | E | Lower-level tests: unit and integration | Not started | diff --git a/devdocs/modernization/TESTING.md b/devdocs/modernization/TESTING.md index 2e65a14e..31b6d3d3 100644 --- a/devdocs/modernization/TESTING.md +++ b/devdocs/modernization/TESTING.md @@ -7,7 +7,7 @@ What test suites exist, what they protect, and the rules for adding to them. | Suite | Where | What it covers | State | |---|---|---|---| | API integration tests | `raven-test-integration` | Tier 1–3 API coverage against a running server | Green apart from three known server bugs (March 2026). Details in that repo's `CLAUDE.md` | -| E2E | Cypress 9.7.0 **[unverified: which folder]** | User workflows in a browser, against remote devtest servers, with version validation and environment-based config | **[unverified]** | +| E2E | `raven-test\e2e` (separate repo `raven-test`), Cypress 9.7.0 | User workflows in a browser, against remote devtest servers, with version validation and environment-based config | **[unverified]**: last commit 2026-02-22; current pass state unknown | | Licence verification gate | To be written (`NET10-PORT.md`, section 0) | Every stored v8 licence still verifies | Not started | | Unit tests | none | — | Roadmap step E |