188 lines
20 KiB
Markdown
188 lines
20 KiB
Markdown
# AyaNova 8 back end → .NET 10
|
||
|
||
The plan and running record for the port. Tick items and add a log line **in the same
|
||
commit** as the change they record. Anything found along the way that is out of scope goes
|
||
under "Deferred", not into the branch.
|
||
|
||
| | |
|
||
|---|---|
|
||
| Starting point | AyaNova 8.2.4: ASP.NET Core 8, EF Core 8 |
|
||
| Target | `net10.0`, every back-end package at its current release |
|
||
| Front end | Minimal or no change. Rebuild only if needed (open question) |
|
||
| Branch | `net10-port` in `raven` (and `raven-client` and `raven-test-integration` and `raven-launcher`) |
|
||
| Baseline tag | `net8-baseline`, on the last commit before the port |
|
||
| Release version | 8.3.0 |
|
||
|
||
## Where this plan comes from
|
||
|
||
Sockeye, the internal licensing server, is a near-copy of this codebase. It was ported from
|
||
.NET 6 to .NET 10 in September–October 2026 and released as `8.1.0`. This plan is that
|
||
port's carry-forward list (Appendix A of the Sockeye migration runbook), arranged as a
|
||
checklist.
|
||
|
||
**AyaNova starts from .NET 8, not .NET 6.** So every item is "check whether this applies",
|
||
not "this will apply". Several items will turn out to be already done or not present. Mark
|
||
those `n/a` with a one-line reason rather than deleting them.
|
||
|
||
When an item is unclear, look at how Sockeye did it:
|
||
|
||
```powershell
|
||
git -C ..\sockeye log --oneline net6-baseline..8.1.0
|
||
git -C ..\sockeye show <commit>
|
||
```
|
||
|
||
**Where AyaNova differs from Sockeye:** customers choose their own database role names and
|
||
passwords, every customer uses reports, and AyaNova *verifies* licences rather than issuing
|
||
them. Items touching those three areas carry more weight here than they did in Sockeye.
|
||
|
||
## Scope
|
||
|
||
- **In:** target framework; all NuGet packages in `server/AyaNova`; code changes those require; the Sockeye-found fixes in section 3, each as its own commit.
|
||
- **Out:** front-end dependency changes; refactoring (Roadmap step D); new features; PostgreSQL major upgrades.
|
||
- **Undecided:** see "Open questions".
|
||
|
||
## Working method
|
||
|
||
- Version bump first. One concern per commit.
|
||
- Prove each gate on the **unmodified** code before changing anything, so a later failure points at the port, not at the test.
|
||
- Packages that must move together: framework-matched `Microsoft.*` packages all go to 10.x in one step. (The BouncyCastle/MimeKit clash from Sockeye does not arise here: AyaNova is already on `BouncyCastle.Cryptography` 2.x with MailKit/MimeKit 4.x.)
|
||
- Publish `server\AyaNova\AyaNova.csproj` directly, never the solution, so no test project can reach the artifact.
|
||
|
||
---
|
||
|
||
## 0. Before changing anything
|
||
|
||
- [x] Record any port work already done in earlier sessions (branch, commits) here first, and reconcile it with this list. Done: branch `net10-port` and the 8.3.0 version bump (`c2f2f922`); no code changes yet
|
||
- [ ] `raven` and `raven-client` clean on `main` and pushed
|
||
- [x] Annotated tag `net8-baseline` on the pre-port commit in `raven` and `raven-client`. Present locally in both; confirm on the remote with `git ls-remote --tags origin`
|
||
- [x] Branch `net10-port` created and pushed with upstream (`origin/net10-port` exists in `raven`)
|
||
- [x] **First commit: version bump** in every place listed in `devdocs/build/BUILD.md` → "Version numbers" (`c2f2f922`, 8.3.0)
|
||
- [x] Dev machine has the .NET 10 SDK and keeps the .NET 8 runtime for baseline comparisons. SDK 10.0.103 (no 8 SDK; the 10 SDK builds `net8.0`); runtimes 8.0.24 and 10.0.3
|
||
|
||
### Gates (prove on the unmodified code)
|
||
|
||
- [ ] **Integration suite baseline.** Run `raven-test-integration` against the unmodified build and record the counts in the log below. Expected: everything passes except the three known server-side failures listed in that repo's CLAUDE.md
|
||
- [ ] **Licence verification gate.** Write a test that runs every v8 row of Sockeye's `alicense` fixture through AyaNova's own verification routine. The fixture is customer data: it stays outside the repo and is found through an environment variable, as Sockeye's re-sign harness does with `SOCKEYE_RESIGN_FIXTURE`. Must be green on unmodified code. A regression here would reject licences customers already have installed. How AyaNova verifies:
|
||
- **Library:** BouncyCastle (`util/License.cs:15-17`). `PemReader` reads a SubjectPublicKeyInfo PEM (`BEGIN PUBLIC KEY`, not PKCS#1) into an `AsymmetricKeyParameter`; `SignerUtilities.GetSigner("SHA256WITHRSA")` verifies (`License.cs:968-978`)
|
||
- **Public key:** hardcoded RSA-2048 PEM, "our real 2016 public key", `License.cs:957-965`
|
||
- **Signed payload:** the JSON between `[KEY` and `KEY]`, with whitespace outside strings stripped by regex (`License.cs:950`); base64 signature between `[SIGNATURE` and `SIGNATURE]` (`:951`)
|
||
- **Other checks in `Parse`** (`License.cs:926-1062`): `LicenseFormat == "8"`; `DBID` equals the server's schema id; perpetual key only on a perpetual build and subscription key only on a subscription build; `MExBB`, which rejects a perpetual non-trial key whose maintenance expiry is before **the build date, read from the PE linker timestamp** (`util/FileUtil.cs:861-882`). That timestamp is only a real date because `<Deterministic>False</Deterministic>` (`AyaNova.csproj:12`). Turning determinism on, or getting the CA2022 change at `FileUtil.cs:875` wrong, would reject perpetual licences with E1020. The gate must cover `MExBB`, not only the signature
|
||
- **When it runs:** at boot, `BootSafetyCheck` before any schema update (`Startup.cs:603`), then `InitializeAsync` (`Startup.cs:633`); after every fetch, `FetchKeyAsync` → `InstallAsync` → `InitializeAsync`, either manually (`LicenseController.cs:125`) or from `CoreJobLicense` (`generator/CoreJobLicense.cs:80`). Fetch interval (`CoreJobLicense.cs:21-34`): perpetual build 30 min while unlicensed, trial or expired, otherwise 24 h; subscription build 30 min; DEBUG 1 or 5 min. The installed key is not re-verified on a schedule; expiry is checked in CoreJobSweeper, according to the comment at `CoreJobLicense.cs:10`
|
||
- **Test access:** `Parse` is `private static` and depends on the static `ServerDbId` and on the running assembly's timestamp. The test needs `InternalsVisibleTo` or reflection
|
||
- [ ] **Report baseline.** Render the advanced reports on the unmodified build and keep the PDFs for comparison. Sockeye shares only one report with AyaNova, so Sockeye's testing says little about AyaNova's reports
|
||
- [ ] **E2E baseline.** Cypress run (`raven-test\e2e`) against a devtest server on the unmodified build
|
||
|
||
## 1. Framework and packages
|
||
|
||
- [ ] `net8.0` → `net10.0`, with every framework-matched `Microsoft.*` package to 10.x in the same step (`AyaNova.csproj:3`; `raven-launcher.csproj:5`)
|
||
- [x] **Hosting model.** n/a: `Program.cs:330-351` already uses `Host.CreateDefaultBuilder(args).ConfigureWebHostDefaults(...)` with `UseStartup<Startup>`. Only a commented-out `BuildWebHost` line remains (`:308`)
|
||
- [ ] **Asp.Versioning.** Already done: `Asp.Versioning.Mvc` and `.ApiExplorer` 8.1.0 (`AyaNova.csproj:36-37`), wired at `Startup.cs:200-217` as `AddApiVersioning(...).AddApiExplorer(...)`. Remaining: bump to 10.x and confirm `AddApiExplorer` still registers MVC versioning without an explicit `.AddMvc()`. Leave AV0014 warnings: **do not add `[ApiController]`**
|
||
- [ ] **Npgsql / EF Core 10.**
|
||
- [x] `Npgsql.EnableLegacyTimestampBehavior`: n/a, commented out (`Program.cs:26`)
|
||
- [ ] `ConfigureWarnings(w => w.Throw())` **is present** (`Startup.cs:179`). Every new EF warning becomes an exception naming its ID: fix the query, or move that ID to `.Log(...)`
|
||
- [ ] Exercise LINQ-based queries screen by screen. Data-list filters build raw SQL and are unaffected
|
||
- [ ] **BouncyCastle.** Already `BouncyCastle.Cryptography` 2.3.1 (`AyaNova.csproj:28`) with MailKit/MimeKit 4.5.0; no `Portable.BouncyCastle` in the graph. Version bump only. The Sockeye note about `PemReader` and PKCS#1 key pairs does not apply: AyaNova reads a public key (see the licence gate)
|
||
- [ ] **Swashbuckle 10 / OpenAPI.NET 2.** Applies. `Microsoft.OpenApi.Models` merged into `Microsoft.OpenApi`: `using` at `Startup.cs:15` and `ConfigureSwaggerOptions.cs:6`. Bearer requirement (`Startup.cs:235-244`) becomes `c.AddSecurityRequirement(document => new OpenApiSecurityRequirement { [new OpenApiSecuritySchemeReference("Bearer", document)] = [] })`. Remove the explicit `Microsoft.OpenApi` 1.6.14 reference (`AyaNova.csproj:47`)
|
||
- [ ] **NLog 6.** `NLogBuilder.ConfigureNLog` is already replaced (`Program.cs:206`, `LogManager.Setup().LoadConfiguration`). `ArchiveNumbering = Rolling` (`Program.cs:136`) is gone in 6. Archive numbers keep incrementing and the un-numbered file is always current. **The log viewer sorts by file name as a string** (`LogFilesController.cs:120`, `OrderByDescending(z => z)`), so `-10` sorts after `-2`. Fix the sort in the same commit
|
||
- [ ] **Newtonsoft.Json.** `Mvc.NewtonsoftJson` 10.x sets a 13.0.3 floor (NU1605). Already resolves 13.0.3 transitively; add it as a direct reference
|
||
- [ ] **JwtBearer 10.x.** The IdentityModel part is n/a: it resolves 7.1.2, not the vulnerable 6.10.0
|
||
- [x] **SMTP.** n/a: `util/Mailer.cs` already uses MailKit's `SmtpClient` (`:1`, `:84`); no `System.Net.Mail`
|
||
- [ ] **PuppeteerSharp 17 → 25.x.** Reporting is AyaNova's main Puppeteer surface (`biz/ReportBiz.cs`)
|
||
- [x] `BrowserFetcher.DefaultChromiumRevision`: n/a, already `DownloadAsync()` with no argument (`ReportBiz.cs:800`, case 4586)
|
||
- [x] `IJSHandle.ExecutionContext`: n/a, the only use is commented out (`ReportBiz.cs:858-860`). Evaluate results are `JsonElement?`: check any remaining `Evaluate*` call sites
|
||
- The default download is Chrome for Testing, into `Chrome/` and `ChromeHeadlessShell/` beside the app. AyaNova 8.2.0 already moved from Chromium to Chrome to follow Puppeteer's default, so the browser choice is settled
|
||
- [ ] Launch arguments must be separate array elements. **Both paths are wrong today:** the defaults are one element `"--headless --no-sandbox "` (`ReportBiz.cs:812`, `:829`), and `AYANOVA_REPORT_RENDER_BROWSER_PARAMS` is passed whole as one element (`:819-822`)
|
||
- AppArmor restricts unprivileged user namespaces on Ubuntu 24.04+, which affects the Chrome sandbox
|
||
- [ ] Remaining packages to current releases. MailKit 4.5.0 carries a moderate advisory (NU1902). After the upgrade, check what still pulls in `System.Text.Json` 5.0.2
|
||
- [ ] Warnings: CA2022 → `ReadExactly` at `FileUtil.cs:875` (**licence build-date path**: covered by the licence gate); ASP0019 → header indexer instead of `Headers.Append` at `Startup.cs:514`
|
||
- [ ] **MiniProfiler. Present, not absent.** `MiniProfiler.AspNetCore.Mvc` and `.EntityFrameworkCore` 4.3.8 (`AyaNova.csproj:48-49`); `AddMiniProfiler(...).AddEntityFramework()` (`Startup.cs:62-74`); auth workaround for `/profiler/results` (`Startup.cs:393-441`); `UseMiniProfiler()` (`Startup.cs:530`); `using StackExchange.Profiling.Internal` (`ReportBiz.cs:14`). Nothing in `raven-client`. Keep and upgrade, or remove: see Open questions
|
||
- [x] **BinaryFormatter.** n/a: not used anywhere in `server/AyaNova`
|
||
- [x] **`System.GC.Server`.** n/a: not set, so the Web SDK default (`true`) stays. On one vCPU the runtime uses workstation GC anyway, and DATAS keeps server GC lean since .NET 9
|
||
- [ ] **`InvariantGlobalization`.** Not set. Decide on EF Core string comparison and date-parsing grounds. Note `DataListSqlFilterCriteriaBuilder.cs:835` reads `CurrentCulture.NumberFormat`. Still open in Sockeye too
|
||
|
||
## 2. Configuration behaviour to check
|
||
|
||
- [x] **URL binding precedence.** n/a: `UseUrls(ServerBootConfig.AYANOVA_USE_URLS)` (`Program.cs:340`), read from `config.json`, environment and command line (`Program.cs:38`, `ServerBootConfig.cs:167-170`). The documented `AYANOVA_USE_URLS` is the one that wins over `ASPNETCORE_URLS`; a command-line `--urls` still overrides it through `UseConfiguration` (`Program.cs:341`)
|
||
- [ ] **Logging providers.** No journal flood: `Program.cs:348` calls `ClearProviders()` before `UseNLog()`. `appsettings.json:4-13` still sets the Debug and Console providers to `Trace`, with no effect today. Fix the levels in the repo
|
||
- [x] **Data Protection keys.** n/a: nothing uses Data Protection; authentication is JWT only (`Startup.cs:278-302`)
|
||
|
||
## 3. Fixes found through Sockeye (each its own commit)
|
||
|
||
These are bugs, not port mechanics, so each is a separate commit that can be reverted alone.
|
||
Sockeye included its equivalents in the port branch. John decides per item whether they go
|
||
in this branch or a follow-up release. UPDATE: John has decided these all go in this branch and some more additional cases in the case database that are urgent, exact cases TBD.
|
||
|
||
- [ ] **Case 4676: data-list SQL injection — priority.** Confirmed: `DataListSqlFilterCriteriaBuilder` appends the raw filter value for Roles (`:343-354`) and for every numeric type (Enum, MemorySize, Decimal, Currency, InternalId, Integer, `:866-888`). Text (`:233`) and tag (`:1024`) values escape `'`; TimeSpan goes through `int.Parse` (`:1099+`). Any authenticated user who can open a data list can reach this. Port the Sockeye fix
|
||
- [ ] **`DbUtil.AdminConnectionString`.** Confirmed: `DbUtil.cs:99` is `_dbConnectionString.Replace(_dbName, "postgres")`, used by the boot check (`:192`), create (`:286`) and drop/recreate (`:334`). Any customer whose role name, password or host contains the database name as text gets a mangled admin connection, and the boot check authenticates as the wrong user. Fix: `new NpgsqlConnectionStringBuilder(cs) { Database = "postgres" }.ConnectionString`
|
||
- [ ] **Backup `pg_dump` call.** Confirmed: `CoreJobBackup.cs:68` builds `postgresql://user:password@host:port/db` with the raw password, and `:94` adds `-Fc > file`; `RunProgram` runs it through `cmd.exe /C` (`RunProgram.cs:55`) or `/bin/sh -c` (`:82-89`). A password with spaces or `@ : / ? # %` breaks backups. Also, on Linux stderr is not redirected (`RunProgram.cs:86-92`), so a failed pg_dump returns empty output and is **reported as a successful backup** (`CoreJobBackup.cs:97`). Fix: password in `PGPASSWORD` in the process environment, connection parameters as separate arguments through `ProcessStartInfo.ArgumentList` (no shell), output with `-f`, stderr and exit code checked
|
||
- [ ] **Drop-and-recreate database.** Confirmed: `DbUtil.cs:342-345` drops, then creates; triggered by `AYANOVA_PERMANENTLY_ERASE_DATABASE` (`Startup.cs:567-570`). Without `CREATEDB` it drops the database and then fails to create it. Test the permission before dropping, or change the order
|
||
- [ ] **Report browser download location.** Confirmed: `new BrowserFetcher()` with the default path beside the app (`ReportBiz.cs:800`). Point it under the data directory in code. Beside the app it is read-only under a hardened systemd unit (`ProtectSystem=strict`), and a redeploy that replaces the app folder deletes it
|
||
- [ ] **Report failures: return a specific error.** The Sockeye crash is n/a: rendering runs as a job, and `JobsBiz.ProcessJobAsync` catches every exception, logs it through NLog and marks the job Failed (`JobsBiz.cs:390-439`), so the process survives. What remains: a browser download or launch failure (`ReportBiz.cs:800`, `:838`, outside the render `try`) reaches the client only as a generic job failure. Catch it in `DoRenderJob` and return a render error that says what failed
|
||
- [ ] **Data-list date filters, small fixes.** Confirmed: `>=` (`:790-793`) and `<` (`:795-798`) use `MaxSeconds` and should use `ZeroSeconds`; date `NotEqual` (`:806-814`) uses `MaxSeconds` on both sides and should exclude the whole minute (`< ZeroSeconds OR > MaxSeconds`). `DataListField` is dereferenced at `:24` and `:27` before the DEBUG-only null check at `:33`, so **release builds** throw a NullReferenceException on an unknown column, not only DEBUG builds
|
||
- [ ] **Relative date tokens use the server's time zone** (`*today*`, `*yesterday*` …). Confirmed: `ClientToday = clientTimeStamp.Date` (`:399-400`) has `Kind=Unspecified`, and `ToUniversalTime()` (`:1060`, `:1065`) converts it with the server's zone. Wrong for any user in a different zone from the server. The fix builds the range from `clientTimeStamp.Offset`, with care for ranges that cross a DST change. **This changes behaviour** and affects the integration tests' `TIME_ZONE_ADJUSTMENT`. Probably better as a follow-up release
|
||
|
||
Not port items, listed so they are not forgotten: the three server bugs the integration
|
||
suite is waiting on (case 4653, case 4654, the contract issue).
|
||
|
||
## 4. Test under realistic conditions
|
||
|
||
- [ ] **Database role.** Run under a non-superuser role **named like the database**, with a password containing spaces and `@ : / ? # %`. Boot, on-demand backup, scheduled backup, restore. Sockeye ran as `postgres` for years and never exercised either bug in section 3
|
||
- [ ] **Restore with a dedicated role.** `createdb -O <role>`, then `pg_restore --no-owner --no-privileges --role=<role>`. Without `--role`, a superuser restore leaves every table owned by `postgres`, and the integrity check, which counts through `information_schema`, fails. Compare with the restore instructions in the customer manual (`docs/`) and note any change needed
|
||
- [ ] **Reports, all advanced reports**, compared with the baseline PDFs. On Windows and on Linux. On Linux also: `ldd` on the downloaded browser binary for missing libraries (install exactly what is reported; several Ubuntu 24.04+ names carry a `t64` suffix); the sandbox under the service account; no orphaned `chrome` processes after a cancelled render
|
||
- [ ] Integration suite: same result as the baseline
|
||
- [ ] Licence verification gate: green
|
||
- [ ] Cypress E2E (`raven-test\e2e`) against a devtest server running the .NET 10 build
|
||
- [ ] `build-release.bat` produces all five packages; install and start each one: Windows standalone, Windows LAN, Linux server, Linux desktop, subscription Linux
|
||
|
||
## 5. Release
|
||
|
||
- [ ] **Every host that runs a framework-dependent build needs the .NET 10 runtime** before it gets the release:
|
||
- Hosted subscription droplets (Ubuntu 24.04). How `aspnetcore-runtime-10.0` is obtained on 24.04 needs checking **[unverified]**. On 26.04 it is in Ubuntu's own archive
|
||
- Self-hosted Linux customers (server and desktop zips are framework-dependent)
|
||
- Windows LAN: `lan.iss` prompts for the ASP.NET Core Hosting Bundle. Update its text and link to .NET 10
|
||
- Windows standalone and the launcher are self-contained: no runtime needed on the customer's machine
|
||
- [ ] Customer manual (`docs/`): system requirements and upgrade instructions for self-hosted customers
|
||
- [ ] Build VM: install the .NET 10 SDK; update `devdocs/build/DEV-VM-SETUP.md` and `devdocs/build/BUILD.md`
|
||
- [ ] Final gate runs on the finished code
|
||
- [ ] Merge, per repo:
|
||
```powershell
|
||
git switch net10-port; git status; git push; git fetch origin
|
||
git switch main; git pull
|
||
git log --oneline net10-port..main # must print nothing: main untouched
|
||
git merge --no-ff net10-port -m "Merge net10-port: .NET 10 port"
|
||
git diff net10-port main # must print nothing: main is exactly what was tested
|
||
git push origin main
|
||
git tag -a <version> -m "<release note>"
|
||
git push origin <version>
|
||
git ls-remote --tags origin
|
||
```
|
||
`--no-ff` keeps a merge commit, so the whole port can be backed out with `git revert -m 1 <merge>`
|
||
- [ ] Rebuild the client from `main`, publish, and test **the published artifact itself**
|
||
- [ ] Rollout: own test droplet first, then customer droplets by time zone, as in the weekly routine
|
||
- [ ] After release: delete `net10-port` locally and on the remote
|
||
|
||
---
|
||
|
||
## Open questions
|
||
|
||
- Release version number (Sockeye's port was released as 8.1.0) Answer: 8.3.0 will be the version
|
||
- `raven-launcher` (.NET 8, self-contained): move to .NET 10 in the same release? Answer: Yes. It has no package references, so this is the target framework only
|
||
- `raven-test-integration`: move to `net10.0` (separate commit) or leave on `net8.0`? It only talks HTTP to the server Answer: Move to net10.0 for consistency
|
||
- Does the client need a rebuild in this pass, or can its current output be carried across? Answer: yes it will have minor changes as well as version number to 8.3.0
|
||
- Section 3: which fixes go in this branch, which in a follow-up
|
||
- How the .NET 10 runtime gets onto Ubuntu 24.04 hosts
|
||
- MiniProfiler (section 1): keep it and upgrade to 4.5.x (check that `MiniProfiler.EntityFrameworkCore` supports EF Core 10), or remove it in its own commit?
|
||
|
||
## Deferred
|
||
|
||
Found during the port, out of scope. One line each.
|
||
|
||
- `License.DoFetchAsync` (`License.cs:629-661`) rethrows at `x == 3`, so the fifth server (Sockeye) is never tried, and it can return `null` to a caller that dereferences it
|
||
- Stray `raven-client\ayanova\cypress.json`: the Cypress suite lives in `raven-test\e2e`
|
||
|
||
## Log
|
||
|
||
| Date | Step | Result |
|
||
|---|---|---|
|
||
| 2026-10-22 | Survey of sections 1–3 against the code (read-only) | Findings recorded per item: n/a items ticked with reasons, evidence added, MiniProfiler found present |
|