CI test-MySQL tuning: redo capacity + buffer pool + io_capacity + table caches (bookshelf-7w1mn) #1335
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "bd-bookshelf-7w1mn"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Tunes the
pergamum-test-mysqlCI image to survive the concurrent-DDL burst from ~8 parallel ginkgo procs each cloning the schema on startup. The migration thundering-herd is already solved (single template-DB migration ininternal/dbtest/dbtest.go); this PR raises the server's ceiling for the residualcloneSchemaFromTemplateCREATE TABLEstorm.Changes to
docker/mysql-ci/ci-tuning.cnfinnodb_redo_log_capacity2Ginnodb_buffer_pool_size512M1Ginnodb_io_capacity/_max200/2000(default)20000/40000table_open_cache/table_definition_cache400/2000(default)4000/4000Image tag bump
:1→:2Dockerfile.mysql-ciheader comments updated..forgejo/workflows/ci-image-mysql.ymljob name +--destinationflags updated.ci.yml(×3) andloadtest.yml(×1) digest pins intentionally left on:1— they need the new:2digest which doesn't exist until the image is built (see HANDOFF below).BLOCKED ON IMAGE BUILD
To finish:
.forgejo/workflows/ci-image-mysql.yml(workflow_dispatch) against branchbd-bookshelf-7w1mn— builds & pushespergamum-test-mysql:2from this branch's ci-tuning.cnf.:1@sha256:ae7608...to:2@sha256:<new>.Closes bead bookshelf-7w1mn on merge.
Code Review — bookshelf-7w1mn
Reviewed diff:
.forgejo/workflows/ci-image-mysql.yml,.forgejo/workflows/ci.yml,.forgejo/workflows/loadtest.yml,Dockerfile.mysql-ci,docker/mysql-ci/ci-tuning.cnf[MINOR] Dockerfile.mysql-ci:12 — comment claims buffer pool was raised to 1G; actual value is 512M
The Dockerfile header comment reads "Raise innodb_buffer_pool_size to 1G (8 schemas resident during clone storm)" but
ci-tuning.cnfsets it to 512M with an explicit note explaining WHY it was kept at 512M (OOM risk on 2G tmpfs). The comment in the Dockerfile was not updated to match the final decision. No correctness impact; a future reader will be confused. Fix: update the bullet to "Buffer pool kept at 512M; redo-log is the lever (see ci-tuning.cnf)".[MINOR] docker/mysql-ci/ci-tuning.cnf:14 — stale comment about max_connections startup default
The comment says "512 here is the startup default; the effective limit during tests is 1000" but the actual value is now 1000 (the PR raised it). The first sentence is now false — 512 is neither the value in the file nor the effective runtime value. Fix: remove or rewrite the sentence to "1000 matches the value raised at runtime by dbtest.go".
Tuning value analysis (correctness of 768M redo on 2G tmpfs):
Sizing is sound. MySQL pre-allocates the full
innodb_redo_log_capacityon startup as#ib_redo*files. At 768M, the math given in the comment holds: 768M redo + ~90M system tablespaces ≈ 858M at rest, leaving ~1.2G free on the 2G tmpfs for data pages during the DDL storm. Choosing 768M (not 2G) is correct — 2G would nearly exhaust the tmpfs at startup before any data page is written. The agent notes local verification passed withmake integrationunder--tmpfs size=2g.innodb_io_capacity=20000/40000for a RAM-backed tmpfs is appropriate and consistent with MySQL documentation guidance (tmpfs is effectively memory I/O; the 200 default is conservative for spinning disks). No OOM vector from these.table_open_cache=4000/table_definition_cache=4000for 8 schemas × ~60 tables each (~480 tables live simultaneously) is adequate — the previous default of 2000/400 would thrash badly. No heap explosion concern; these are descriptor counts, not memory-backed structures at the scale of 4000.innodb_buffer_pool_size=512Mkept unchanged — correct. The 2G tmpfs holds the datadir; a 1G buffer pool on top in the same container would push RSS toward ~1.5G+ and risk OOM kill. The redo-log fix is the right lever.Digest pin audit:
All 4 image refs (
ci.yml×3,loadtest.yml×1) point to:2@sha256:e29429e0fdd68af622f4b7b725621596c8f9297d86720f4a70b797f660deb6e9. No leftover:1ref orae7608digest survives anywhere in the diff or in the branch files. Clean.--skip-tls-verify-registryaddition:Consistent with
ci-image.yml, which already carries bothforgejo.lanandgit.zombor.netentries. Justified (self-signed cert on forgejo.lan; Cloudflare TLS preflight on git.zombor.net). No new security surface beyond the established pattern.CI status (HEAD
4cbe6427):Combined status: failure. Failing jobs: JS Unit Tests, E2E Browser, Integration. Passing: Lint, Test Race, Coverage, E2E API.
The branch is based off
fbe9bca0(PR #1331) and main has since advanced 2 commits (PR #1336 — an e2e axe timeout fix, PR #1335's own merge base is stale by those 2 commits). The failing jobs (JS Unit Tests, E2E Browser, Integration) are not touched by this diff at all — this PR changes only CI image configuration. These failures are characteristic of main drift rather than a regression introduced here. However CI is currently red and the PR needs a rebase onto current main before the combined status can be trusted as clean.PR is marked
mergeable: Trueby the API, but a rebase is required to confirm CI green after incorporating the 2 new main commits.REVIEW VERDICT: 0 blocker, 0 major, 2 minor
Security review — PR #1335 (bookshelf-7w1mn) — CI MySQL image :1→:2 + InnoDB tuning
Scope
CI infra only. The
pergamum-test-mysqlimage is consumed exclusively by three service containers in.forgejo/workflows/ci.yml(Integration, E2E API, E2E Browser jobs) and one inloadtest.yml. It is not referenced indocker-compose.yml,Dockerfile, or any production/runtime path. Zero production risk.Findings
[MINOR] .forgejo/workflows/ci-image-mysql.yml:67 —
--skip-tls-verify-registry=git.zombor.netadded but not in original :1 workflowThe existing
ci-image.yml(forpergamum-ci) does carry both--skip-tls-verify-registry=forgejo.lanand--skip-tls-verify-registry=git.zombor.net, so this mirrors the established pattern. The new flag forgit.zombor.netis scoped exclusively to the image-BUILD job (ci-image-mysql.yml, manual dispatch only) — it is not present in ci.yml's runtime image pulls. Runtime pulls remain digest-pinned (tag:2@sha256:e29429e0…), which provides cryptographic integrity regardless of TLS. The added flag is consistent with the existing ci-image.yml convention and carries no new attack surface. Noting as MINOR only because TLS skip on a push path is worth documenting; the PR comment already explains the rationale.Explicit confirmations (per review brief)
Digest pins: All four service-container
image:references in ci.yml (x3) and loadtest.yml (x1) are updated togit.zombor.net/zombor/pergamum-test-mysql:2@sha256:e29429e0fdd68af622f4b7b725621596c8f9297d86720f4a70b797f660deb6e9. All four are identical — no tag-only ref, no mismatched digest, no downgrade from digest-pinned to tag-only. Supply-chain integrity is maintained.TLS skip scope:
--skip-tls-verify-registryflags appear only inci-image-mysql.yml(the image-BUILD workflow). The ci.yml runtime image pulls contain no TLS skip flags; they rely solely on the digest pin for integrity. This is correctly scoped and mirrors the existingci-image.ymlprecedent.ci-tuning.cnf production safety: The file opens with
# CI throwaway-database tuning — DO NOT use on production databases.No production docker-compose or Dockerfile referencespergamum-test-mysql,Dockerfile.mysql-ci, orci-tuning.cnf. The durability-off settings (innodb_flush_log_at_trx_commit=0,innodb_doublewrite=0,skip-log-bin) are confined to the ephemeral CI image and carry no production risk.Secret exposure: No tokens, passwords, or credentials are hardcoded in any changed file. The workflow uses
${{ secrets.REGISTRY_TOKEN }}viaenv:injection; the base credentials block follows the same pattern as the existing ci-image.yml. Dockerfile.mysql-ci contains noENV/ARGsecrets.Base image pin:
Dockerfile.mysql-cipins the base asmysql:8.0@sha256:7dcddc01…— digest-locked, not floating.Production path:
pergamum-test-mysqlis referenced in zero production files.docker-compose.ymlusesmysql:8.4(the upstream image). No runtime surface consumes this CI image.REVIEW VERDICT: 0 blocker, 0 major, 1 minor
4cbe642707f5f3475675f5f347567564aefe626564aefe626572c6f26cf0