mirror of
https://github.com/nestriness/nestri.git
synced 2026-09-19 17:25:19 +03:00
53d69ca289624ddb2b2f3d3a99984130867223fa
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Wanjohi
|
53d69ca289 |
fix(nesdoctor): parse df from the right; a device name can contain a space
Three more things the macOS CI log showed, none of which had ever been visible
from this laptop.
`df -P` fixes the column order but not that the filesystem name is one word.
macOS emits
map auto_home 0 0 0 100% /System/Volumes/Data/home
which shifts every field by one, so indexing from the left read the capacity
percentage as part of the mount point and a device name as the size. The row
appeared in the log as `100% /System/Volumes/Data/home`, which is what gave it
away. Columns are now counted from the right, where `df` actually guarantees
them: size, used, avail, capacity, mount. A test covers the plain row, the
two-word `map auto_home` row, and an SMB share whose device name contains a
space -- the case that makes left-indexing wrong in principle rather than just
on Macs.
The `/System` filter I claimed to have added in the previous commit was not in
the file. The assertion that was supposed to catch that passed against the
wrong block, so it went in silently and `/System/Volumes/xarts` kept appearing
in the very output I had just quoted as fixed. It is there now, along with
`/private/var/vm` and `/Volumes/Recovery`, and verified by grep rather than by
belief.
And a filesystem reporting no capacity is not storage: `map auto_home`, devfs
and macOS signed asset bundles all report zero and were padding the filesystem
count in the summary line.
Net effect on the runner, across this commit and the last: 11 filesystems and
"483 GiB free of 1600 GiB" on a 320 GiB machine, down to the one real volume.
|
||
|
Wanjohi
|
f32db5393b |
fix(nesdoctor): APFS volumes share one pool, and were counted eleven times
Found by reading what the macOS CI runner prints, which is the whole reason that step was added an hour ago. First time anyone had looked at what these probes return on a platform that is not this laptop: / 96 GiB free of 320 GiB /System/Volumes/VM 96 GiB free of 320 GiB /System/Volumes/Preboot 96 GiB free of 320 GiB /System/Volumes/Update 96 GiB free of 320 GiB /System/Volumes/Data 96 GiB free of 320 GiB 11 filesystems · 483 GiB free of 1600 GiB total On a machine with 320 GiB. An APFS container presents each volume as its own filesystem with its own `/dev/diskNsM`, so the device-name dedupe -- which correctly collapses btrfs subvolumes -- cannot see that the space is shared. Two changes. `/System/Volumes` and `/private/var/vm` are skipped: they are not user storage, and on a Mac they are most of the rows. And filesystems are deduped by *pool* as well as by device. Two filesystems reporting byte-identical capacity and byte-identical free space are one store, whatever their device names say -- which also covers bind mounts and thin-provisioned LVM, neither of which the device check catches either. Two genuinely separate disks agreeing to the byte on both figures would cost one row; a storage total inflated fivefold is a number a capacity plan gets built on. Simulated against the exact runner output: eight rows and 2560 GiB become one row and 320 GiB. The Windows runner, by contrast, was correct first time -- `C:` and `D:` are genuinely separate and totalled 179 GiB free of 299 GiB. Worth recording that the reason we know is that we looked, rather than that we reasoned about it. |
||
|
Wanjohi
|
786267f30a |
fix(nesdoctor): report storage properly, and stop being blind on Windows
A submission from a team machine with four drives and 22 TiB reported
`disk=8880`, and the field was not wrong so much as meaningless: it was the
free space on the single largest mount, with no capacity anywhere and no total.
A content store is sized against capacity.
Storage now reports four things, because they answer different questions and
one number could not:
diskfree total free across every real filesystem
disksize total capacity
diskmax the largest single filesystem, which is the real ceiling for any
one store -- a dataset cannot be spread across drives
disks how many there are
The ambiguous `disk` key is gone rather than silently redefined, so old rows
stay readable as what they were. `Get-PSDrive` reports Free *and* Used and we
were reading only Free, hence no capacity on Windows at all.
Pseudo-filesystems are now excluded by *type* rather than by mount path. Path
filtering missed `/tmp` on a tmpfs, whose free space is RAM -- so 7 GiB of
memory was being added to a storage total, which is exactly the sort of number
a capacity plan gets built on.
## The real finding, which was not about disks
"We are working blind on Windows" is correct, and both Windows bugs this tool
has had prove it: a virtual display adapter reported as the GPU, and a URL
truncated at its first `&`. Both were in code that only runs on Windows, both
were found by a person reading the results channel, and neither could have been
found here -- the development machine is Linux and `xdg-open` never sees a
shell.
Two things about that, and the first is the one that generalises.
`OPENERS` is now a const with a test asserting the property that actually
matters: **never hand a URL to anything that will re-parse it.** No `cmd`, no
`sh`, no `powershell`, no `start` builtin, and no argument that looks like it
wants the URL interpolated into it. Unlike the bug, that is checkable on every
platform in a millisecond. Verified by reintroducing `cmd /C start "" <url>`
and confirming the test fails with the right message, then reverting.
And CI already runs a real Windows machine and a real macOS one -- we simply
were not looking at them. Each smoke-tested target now prints its full report
and JSON into a collapsed log group. Deliberately not `set -e`: this step is
for looking, and a probe that misbehaves on a runner must not fail a release.
It turns "working blind" into "looking at it once per release", which would
have shown the Parsec adapter problem the first time a Windows binary was ever
built.
Version to 0.2.2.
|
||
|
Wanjohi
|
b055e40546 |
fix(nesdoctor): the first Windows submission recorded a virtual display adapter
Our first response, and the GPU field is wrong: gpu=Parsec%20Virtual%20Display%20Adapter&gpus=2 Parsec installs an indirect display driver, it enumerated first out of `Win32_VideoController`, and the primary was taken as the first entry -- so the real card on that machine is gone. `gpus=2` is the only reason we can tell anything was lost, and it cannot tell us what. This is not an edge case for this audience. Parsec, Sunshine, Moonlight, TeamViewer and Splashtop all install one, and a cloud-gaming community is precisely the population that has one already. A recorded gpu_model is a hard requirement for a host; a virtual display driver satisfies it in name only. Three changes. Adapters are now sorted so real hardware is first, by two keys: whether the name matches a known software adapter, then whether a vendor could be identified at all. Order is the only signal the rest of the program has for which GPU is "the" GPU. The vendor comes from `AdapterCompatibility` rather than from pattern-matching the marketing name. An "AMD Radeon" string is easy; an OEM-rebadged one is not. And every adapter name is now sent, not only the count. `gpus=2` told us something had been lost and not what, which is the kind of field that wastes a response we cannot ask again. The known-software-adapter list has unit tests, on all platforms -- it is a list of strings and it will need extending, so it should fail loudly rather than quietly stop matching. Version to 0.1.2. For the record, what that submission got right, because none of it needed asking: 50% of 165 Steam launch records fall in five hours of twenty-four (21:00-01:59) against five records across the whole of 07:00-13:59. That is 0017's evening peak, measured, from one person's own files. 140 of the 165 records are titles no longer installed -- restricting the histogram to installed titles, as review suggested, would have left 25 samples and lost the shape entirely. |
||
|
Wanjohi
|
79f1732a14 |
feat(nesdoctor): a host readiness checker that measures instead of asking (#310)
The first executable form of our host requirements. Until now a machine
was qualified by a human reading a table of hard requirements — and a
requirement that nothing can check is one that is silently optional.
It also replaces a form. Everything we wanted from a prospective host is
measurable, and most of it **cannot be answered honestly by a human
anyway**: almost nobody knows their real upstream, and essentially
nobody has ever seen their own bufferbloat figure. What's left for the
questions is only what a machine cannot know — intent, and what someone
already pays.
## What it does
```
nesdoctor
```
- **Checks every hard requirement**: `/dev/kvm`, an AMD or Intel GPU
with a DRM render node, `VK_KHR_video_encode_queue` plus a codec,
`virglrenderer`, the two stores, the `io` cgroup controller,
`virtiofsd`. Pass / fail / **unknown**, and unknown is never collapsed
into fail — a machine we could not ask is not a machine that failed, and
losing a capable host to a missing `lspci` is the failure mode that
matters.
- **Measures upstream and, the point of the whole thing, added latency
under load.** Grade bands come from the frame budget rather than
convention: the network allowance is ~40 ms because render, encode,
decode, display and jitter buffer have already spent ~58 ms.
- **Reads Steam, only with an explicit yes**, for library size and shape
plus an hour-of-day histogram of launches — one sample per title, which
is a real distribution obtained without asking anybody anything.
- **Asks at most five questions**, branched on what was found, all
skippable.
## No server
Nothing is uploaded and no telemetry endpoint exists. The network test
talks to Cloudflare's public speed-test sink and to `1.1.1.1`, neither
of which is ours. The output is a line on the terminal that the person
may choose to paste.
The shareable line carries **no hostname, IP, username, game title or
path** — a size band rather than a size, hours rather than dates. The
long version, which does include titles and paths, stays in a local JSON
file the person is told the path of.
That is a property of the design and not a promise about our intentions:
there is nothing to switch on later.
```
nesdoctor 0.1.0 | linux/x86_64 | gpu=AMD Barcelo | cpu=12t ram=13G |
kvm=y venc=y zfs=n boxfs=n io=y | up=28Mbps rtt=179ms bloat=+19ms grade=B |
disk=91G | edge=KE/JNB | steam=1 titles/<100G | plays=20-03h n=74 |
role=- share=- pays=- | HOST-READY-LOCAL
```
## Five bugs found by running it, every one of which would have produced
wrong data
- **`vulkaninfo --summary` lists ZERO `VK_KHR_video` entries** where
full `vulkaninfo` lists five on the same machine. Preferring the summary
reported "not advertised" on a card that advertises it — a false
negative on the check most likely to disqualify a host.
- **btrfs subvolumes counted as separate disks**: `/`, `/home` and
`/srv` each reporting 91 GiB of one 91 GiB device. Now deduped by
backing device, which the two-stores check needs anyway since it wants
*separate devices*.
- **Proton and the Steam Linux Runtimes are installed like games and are
not games.** Five of eight entries on the test machine, so the title
count was 5× too high and the library-shape question was corrupted.
- **`--quiet` printed the whole questionnaire** before its summary line,
breaking the one thing `--quiet` promises. Prompts are now skipped when
output is quiet or stdin is not a terminal — and a pipe is explicitly
*not* treated as consent to read a Steam library, unlike `--yes`.
- Boot history was reporting `13.2 h/day` off **two days** of history.
Under a three-day span it now reports the span and no rate.
## One finding, now encoded as a verdict
The development connection measures **179 ms idle RTT, served from
Johannesburg**. That machine passes every other check and cannot host
for a European player, because it is distance and no upgrade shortens
it.
`HOST-READY-LOCAL` exists for exactly that case, and the wording is
deliberate:
> Every requirement passes and your uplink queues cleanly. But the idle
round trip to the nearest major network is already most of the latency
budget, and that is distance rather than a fault: no upgrade shortens
it. So this machine is a good host for people on your side of the world
and cannot be one for anybody else. **If you are somewhere without a
cloud gaming edge, that is not a consolation prize — it is the only way
anyone there gets a playable stream.**
## CI
- **`ci.yml` gains a `nesdoctor` job** — fmt, `clippy -D warnings`,
test, one real run. Scoped to this member deliberately: the rest of the
Rust half has never been under CI, so `--workspace` would turn every PR
red for unrelated reasons. Widen it one member at a time.
- **`release-nesdoctor.yml`** builds four targets on tag `nesdoctor-v*`
— x86_64 linux-musl, x86_64 windows-msvc, aarch64 and x86_64 macOS —
with `SHA256SUMS`. musl rather than glibc so one Linux binary runs on
every distro.
The step that justifies the workflow **runs the binary it just built,
network included**. `ring` under rustls resolves root certificates
through the host trust store, so a static musl build can compile cleanly
and then fail TLS on the machine it ships to — breaking the network
test, silently, and only for other people. The step fails the build if
the summary line comes back `net=unmeasured`.
## Dependencies
Four: `anyhow`, `clap`, `serde`, `ureq`. The VDF parser, every platform
probe and the text wrapping are in-tree. A binary handed to strangers
has a dependency tree that is part of its interface, so anything that
could be done with `std` is.
4 MB release binary.
## What it deliberately does not claim
- **A pass is not a promise.** Every check is a *necessary* condition,
and nothing here runs under load — a machine that passes can still fail
on block I/O.
- **The encode extension being advertised is not proof the path works.**
We have had a correct extension list over a broken path before, so that
row says so.
- **Whether `libvirglrenderer` carries the native-context patches cannot
be determined from outside**, so that row reports presence only and
stays `unknown` rather than `pass`.
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
The PR adds the nesdoctor host-readiness executable, local Steam
analysis, network measurement, installers, CI validation, and
multi-platform release packaging. Two attempted correctness fixes remain
incomplete:
- physical disk deduplication does not resolve common device-mapper
source names before comparing backing devices
- unknown historical Steam appids can still be counted as game launches
without passing runtime filtering
<h3>Confidence Score: 3/5</h3>
The PR is not yet safe to merge because shared LVM-backed stores can be
reported as physically independent and unknown Steam tools can still be
reported as game launches.
The new disk resolver fails open for common device-mapper names,
preserving a false host-readiness verdict, while Steam history still
counts absent appids without determining whether they are games or
runtime tools.
**Files Needing Attention:** apps/nesdoctor/src/sys.rs,
apps/nesdoctor/src/hostreq.rs, apps/nesdoctor/src/steam.rs
<h3>Important Files Changed</h3>
| Filename | Overview |
|----------|----------|
| apps/nesdoctor/src/sys.rs | Adds system and disk discovery, but
unresolved device-mapper names undermine physical-backing comparisons. |
| apps/nesdoctor/src/hostreq.rs | Implements host requirement verdicts
and uses physical-device sets that can falsely classify shared LVM
backing as independent. |
| apps/nesdoctor/src/steam.rs | Adds manifest and launch-history
analysis, but unknown appids bypass runtime classification and
contaminate launch metrics. |
| apps/nesdoctor/src/net.rs | Adds bounded upload-based upstream and
bufferbloat measurement; the previously reported unbounded request path
is addressed. |
| .github/workflows/release-nesdoctor.yml | Builds, smoke-tests,
packages, checksums, and publishes the four release targets. |
| .github/workflows/ci.yml | Adds focused formatting, linting, testing,
and offline execution checks for nesdoctor. |
<h3>Flowchart</h3>
```mermaid
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Run[nesdoctor] --> Host[Host requirement probes]
Run --> Net[Upload and latency measurement]
Run --> Consent{Steam consent}
Consent -->|yes| Steam[Installed manifests and LastPlayed records]
Host --> Physical[Resolve filesystem sources to physical devices]
Physical --> Verdict[Host readiness verdict]
Net --> Report[Detailed JSON and shareable summary]
Steam --> Report
Verdict --> Report
```
<details><summary>Prompt To Fix All With AI</summary>
`````markdown
### Issue 1
apps/nesdoctor/src/sys.rs:369-374
**Mapper devices remain unresolved**
When root and box-store filesystems are separate LVM or dm-crypt mappings on the same physical disk, `df` supplies `/dev/mapper/...` names that do not exist under `/sys/class/block`. This branch returns those unrelated logical names unchanged, so the overlap check passes stores that still share one physical I/O queue.
### Issue 2
apps/nesdoctor/src/steam.rs:247-250
**Unknown appids bypass runtime filtering**
If `localconfig.vdf` retains `LastPlayed` data for an uninstalled Proton build, Steam runtime, or other non-game tool, its appid is absent from the installed-manifest map and this branch treats it as an uninstalled game. The tool activity then changes the launch histogram, peak window, and shareable `n` value.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
`````
</details>
<sub>Reviews (5): Last reviewed commit: ["fix(nesdoctor): three valid P1
findings
..."](
|