Repository navigation
fix(s3compatsigv4): ストレージ側エラーの異常系ハンドリングを全面改善(クォータ超過の明示化とコミット成否不明時の案内) - #98
Open
tishin-endou wants to merge 6 commits into
Open
tishin-endou wants to merge 6 commits into
tishin-endou wants to merge 6 commits into
Conversation
tishin-endou
force-pushed
the
fix/s3compatsigv4-quota-handling
branch
from
September 12, 2026 11:08
dcb370b to
ee267a6
Compare
tishin-endou
changed the title
tishin-endou
force-pushed
the
fix/s3compatsigv4-quota-handling
branch
from
September 16, 2026 12:46
59a6a13 to
96b3af6
Compare
tishin-endou
added a commit
to tishin-endou/RDM-waterbutler
that referenced
this pull request
Sep 20, 2026
…notes (repo convention). Comment/docstring-only change, AST-verified identical. Full rationale preserved in PR RCOSDP#98 description and per-commit messages
tishin-endou
changed the title
tishin-endou
changed the title
tishin-endou
marked this pull request as ready for review
September 24, 2026 06:24
tishin-endou
added a commit
to tishin-endou/RDM-waterbutler
that referenced
this pull request
Sep 24, 2026
CompleteMultipartUpload is not idempotent. A re-send after the first attempt
succeeded meets a consumed UploadId and comes back `NoSuchUpload`, so whatever
error code is observed belongs to the last attempt and says nothing about the
upload. Two separate mechanisms can re-send it and each needs its own stop.
Measured on the current code:
- core's retry loop sends the commit 3 times on 408 / 502 / 503 / 504
(`retry` defaults to 2, `retry_on` is {408, 502, 503, 504}).
- aiohttp follows a 307/308 itself, below core's retry budget. The new
`commit_server` test records ['/first', '/second'] -- the commit body is
re-POSTed to the redirect target, which answers `SignatureDoesNotMatch`.
This is the first measurement of the redirect path for s3; `aiohttpretty`
injects above `ClientSession._request` and cannot reach it, so the
fixture from s3compatsigv4 (PR RCOSDP#98) is ported here.
`test_commit_request_states_both_preconditions` watches the two keywords
directly rather than only through their effect, and
`test_the_other_upload_requests_keep_the_defaults` pins that they stay scoped
to the commit.
7 failing, no production change yet.
tishin-endou
added a commit
to tishin-endou/RDM-waterbutler
that referenced
this pull request
Sep 24, 2026
Adds the tests for 決定-13 ahead of the implementation. 117 new tests, all failing: `S3Provider` has no `UPLOAD_MAY_HAVE_COMPLETED_MESSAGE`, `_commit_outcome_note` or `_observed_error_code`, and the provider module has no `DEFINITIVE_REJECTION_CODES`. What the tests say the commit's outcome is. Three values -- it succeeded, it definitely did not happen, or nobody knows -- and the difference that matters is the last two. "The file was not saved" tells the user to upload again, and saying it when the object is in fact on the storage costs a second copy that only an administrator can remove. So UNKNOWN is the fallback: an unknown code and a missing code both land there. The verdict comes from the S3 error code alone. The HTTP status class cannot carry it -- a failed CompleteMultipartUpload arrives as 200 with an <Error> body, so the status says nothing about how definite the storage was. `test_the_note_ignores_the_status_class` states that directly and the 96-cell product states it through five transports. The product is taken in one place on purpose. Per-transport parameter sets cannot expose a contradiction *between* transports, and that is how all three of PR RCOSDP#98's review rounds missed one. The 48 observable cells (three transports x eight codes x two abort outcomes) assert that the same observed code gives the same verdict everywhere; the 32 latent cells (disconnect, truncated body) assert that no code is observable at all, with the premise spied on so that an implementation emitting the notice unconditionally cannot pass them. The code string is planted in the disconnect's message and inside the truncated body, so reading a code from anywhere but a parsed response body -- or by substring -- fails. `_complete_multipart_upload` is never replaced by a mock. NOTE_SEMANTICS_DESIGN v2.2 §4-2d: a test that judges the notice must not mock the code that decides it. Injection goes to the boundary below (`make_request`, `resp.read`) or above (`_upload_parts`). RCOSDP#98 measured two mutations surviving 360 tests when that rule was broken. Table rows get one parameter each rather than being generated from the implementation, so that deleting a row cannot delete its own test -- 4 of 6 row-deleting mutations survived in RCOSDP#98 for that reason. Two deviations from RCOSDP#98, to be recorded in PHASE_TK2_REPORT: * the quota-suppression branch is not ported. K-11 established that `s3` has no quota mechanism at all -- absent from `ADDON_METHOD_PROVIDER` and from `website/util/quota.py`'s `PROVIDERS` -- so there is no quota response to suppress. * K-1's abort-outcome wording stays, and the notice is combined with it rather than replacing it. They answer different questions: "is the object there?" and "is there rubbish left behind?".
tishin-endou
added a commit
to tishin-endou/RDM-waterbutler
that referenced
this pull request
Sep 24, 2026
…4, green) Ports PR RCOSDP#98's three-valued commit outcome. A multi-part upload whose commit fails now says one of two things: the file was not saved, or it may have been saved and the file list should be checked first. Until now it said neither -- the message named an unexpected error and left the user to guess, and guessing wrong in the "not saved" direction means uploading again on top of an object that is already there. The verdict comes from the S3 error code alone. `DEFINITIVE_REJECTION_CODES` holds the nine codes that prove nothing was assembled; every other code, and the case where no code could be read, is UNKNOWN and gets the notice. The asymmetry is deliberate: an over-reported notice costs a look at the file list, an under-reported one costs a duplicate object that only an administrator can remove. The HTTP status class is not consulted, and cannot be. S3 sends the status line before it starts assembling the parts, so a commit that fails part way through arrives as 200 with an <Error> body -- which K-3 already turns into a 502. Classifying on that would report the storage's most definite answers as server errors. The model is sound only because the commit is sent exactly once (K-5, the previous commit). Under a re-send the observed code belongs to the last attempt, and a first attempt that succeeded comes back `NoSuchUpload` -- which is why `NoSuchUpload` is not in the table. Marking happens at the commit boundary, not at the decision. `retry=0` closes the commit into a single `await`, so "sent" is just "entered that await": everything from the request through the body read is marked, whatever the exception type. Narrowing that `except` would drop the notice for exactly the failures nobody anticipated. A connection failure after entering the await may in fact never have reached the wire; that is not distinguishable here, so it goes to UNKNOWN. Two deviations from RCOSDP#98: * RCOSDP#98 suppresses the notice for quota exhaustion ahead of the table, because "you are out of space" and "it may have completed" contradict each other. Not ported: K-11 established that `s3` has no quota mechanism at all -- absent from `ADDON_METHOD_PROVIDER` and from `website/util/quota.py`'s `PROVIDERS` -- so there is no quota response here to suppress. * K-1's abort-outcome wording stays. RCOSDP#98 has no equivalent, so there was nothing to combine it with there. Here the notice goes between the failure sentence and the abort outcome: "is the object there?" and "is there rubbish left behind?" are different questions and the user needs both answers. The table is transcribed from MinIO measurements (NOTE_SEMANTICS_DESIGN v2.2 §2-2). Only `EntityTooSmall` was actually observed on a CompleteMultipartUpload; the other eight rest on the S3 specification and MinIO's error definitions. AWS S3 is unverified -- TEST_SPEC E-1 reconciles it. This is a GRDM difference (G-8), marked `# GRDM:` and excluded from the upstream drafts. Mutations: M-10 (drop `EntityTooLarge` from the table) -> 2 failed, killed by the per-row parameter and the table pin. M-11 (return `''` instead of the notice) -> 79 failed. M-7 (invert the abort branch) still kills, now 6 failed. s3: 290 passed. Full suite: 2169 passed / 7 skipped, flake8 clean.
yacchin1205
requested changes
Sep 28, 2026
yacchin1205
left a comment
yacchin1205
left a comment
There was a problem hiding this comment.
コメントを記載させていただきました。全体的に、実施したいことに対して、過剰なコメント、不要と思われる処理などが多いと感じました。見直していただけますと幸いです。
| return None, None | ||
| try: | ||
| parsed = xmltodict.parse(body) | ||
| except ExpatError: |
There was a problem hiding this comment.
XMLの解析失敗を (None, None) に置き換えると、「応答が壊れていて読めなかった」と「正常に読めたが対象フィールドがなかった」を呼び出し元で区別できません。ストレージ側の異常応答なのか、こちらの文字コード変換・解析処理の問題なのかを調べる際に、必要な原因情報が失われます。解析失敗は原因を保持して伝播するか、明示的な異常応答エラーに変換してください。
S3互換ストレージは、製品によって対応範囲や挙動に差異があります。そのため、予期しない形式は早期にエラー報告をした方が、互換性検証もしやすく、全体として良いように感じます。現状のコードですと、この処理が正常に動作しない、と利用者から問い合わせがあった場合、調査が非常に困難になりそうです。
なお、以下のようなコードも同様です。Noneに落とすにしても、Logging等の配慮がないと、後で「なぜ動作しないのか？」の調査が困難になることが予想されます。
if not isinstance(parsed, dict): return None, None
Author
There was a problem hiding this comment.
ご指摘のとおり、解析失敗の原因を失っていました。失敗の種別(XMLとして解析不能 / Error が要素として存在しない / Error はあるが Code がない)を区別し、例外の型と応答本文(上限つき)をWARNINGで記録するようにしました(703f007)。戻り値を別のエラーに変換していない理由は、この関数がストレージのエラーステータス(403等)を受け取った後の分類に使われるためで、ここで変換するとストレージ本来のステータスが失われます。成功ステータスで異常本文が返る経路(_check_for_200_error)は既に明示的な502への変換になっています。
There was a problem hiding this comment.
ありがとうございます。 703f007 では、
if not isinstance(parsed, dict):
return None, None
が、そのままになっています。理由はありますでしょうか？
これが修正漏れなのだとすると、
思ったのは、isinstanceでチェックしながら落とすのではなく、 error['Code'] のような、シンプルなPython呼び出しの形で実装して、それをtry ~ exceptで囲って、errorはloggingする形で落とした方が、見通しが良いし、このような漏れが少ないのではと感じました。ご検討ください。
Author
There was a problem hiding this comment.
ご指摘のとおり修正漏れです。最初のコメントで引用いただいていた箇所にもかかわらず、当該分岐にログを付けておらず、申し訳ありません。ご提案の形——素直にアクセスし、失敗を例外として1箇所で受けてログする——に書き直しました(4a428d6)。isinstance による分岐と個別のログは廃止し、例外の型とメッセージで原因が分かるようにしています(補足: xmltodict は解析できた入力に対して常にマッピングを返すため当該分岐は実際には到達しないものでしたが、ご指摘の「漏れやすい構造」の問題はそのまま当てはまり、構造ごと改めました)。見通しの改善につながるご提案、ありがとうございます。同じ構造だった _check_for_200_error の取得部分も同じ形に揃えています(cd586d8)。
There was a problem hiding this comment.
ありがとうございます。エラー出力について... type(err).__name__ のようにしていますが、このコードですと、Tracebackは失われるかと思います。これは意図したものでしょうか？(手間をかけて、情報を削っているように見えます。) logger.exception()等、Tracebackを出力する形でのログ出力をすべきではないと判断した理由があれば、教えてください。また、修正内容に対して、commitが非常に多数になり、マージ後に問題調査などの妨げになる恐れがあると考えます。適切なsquash等ご検討いただけますと幸いです。
Author
There was a problem hiding this comment.
ご指摘ありがとうございます。①は意図したものでしたが、過剰でした。aiohttpの接続エラーはメッセージにpresigned URL(署名を含む)を埋め込み、またストレージ応答由来のエラーはメッセージに応答本文を持つため、logger.exception() ではそれらがトレースバックの最終行としてログに出てしまう、というのが型名のみにした理由です。ただ、スタックまで捨てる必要はありませんでした。スタックフレームは全て出力し、例外メッセージはURLのクエリ文字列を除去・長さ上限つきで出力する形に改めました(4589eaa)。②コミットは意味の単位6本に整理しました(force-push済み。整理前後でツリーが同一であることを確認しています)。
Author
|
ご指摘ありがとうございます。コメントについては、レビュー過程の設計判断をコード内に残しすぎており、保守性の観点でご指摘のとおりでした。「定義箇所に理由を1回・呼び出し箇所では繰り返さない・依存ライブラリやテストへの言及は本体に書かない・コードで自明な動作は書かない」を基準に全体を見直しました(0fbfe05, d9156ce, 7cfb7ee: いずれもコメント/docstringのみでASTは同一。provider.py のコメント行 359→78)。処理については、XML解析失敗の原因を残すログの追加と正規表現の事前チェックの削除を行いました(703f007)。CI は全件通過しています。 |
Author
|
レビュー中の追加になり恐縮ですが、1コミット(ccc5fd3)を追加しました。フォルダ一括削除で DeleteObjects が HTTP 200 の応答内に鍵ごとの失敗を返した場合、現行では読まずに破棄しており運用ログに痕跡が残らない問題への対応です「エラーの握りつぶし」として指摘を受けていた類型のため、凍結前に含めました)。ログ出力のみで利用者への応答は変更していません。あわせて boto3 の例外を型名とコードを持つエラーにラップし、素の500にならないようにしています。利用者にも失敗として見せる対応は別PRで行います。本PRから分離した方がレビューしやすければ別PRに切り出します。 |
tishin-endou
added a commit
to tishin-endou/RDM-waterbutler
that referenced
this pull request
Oct 1, 2026
Error reporting. One helper, _raise_from_client_error, now stands at every site
that catches a botocore or make_request failure, and reports the exception's type
name and S3's error code and nothing else. The messages it replaces both carry
material that must not reach a client or a log: botocore quotes S3's prose with
its request and host ids, and core's exception_from_response quotes the request
URL, which under SigV4 is a presigned URL carrying X-Amz-Credential -- the access
key id -- and X-Amz-Signature. waterbutler.server.api.v1.core.write_error hands
exc.message to the client, so check_key_existence was answering 404 with the
signature in the body, and _chunked_upload was logging the same through repr().
The two conversion points also raise `from None`: leaving the original on
__context__ meant traceback.format_exception -- reached through log_exception's
exc_info -- printed the very message they refused to copy. The original is still
logged at both sites, by type and status.
The helper re-raises asyncio.CancelledError unchanged. Python 3.6 derives it
from Exception, so each of these broad excepts was reporting a cancelled request
as a provider failure and stopping the cancellation from propagating.
_chunked_upload gets its own guard because it aborts the session before it
reports. get_s3_bucket_object_location had no handler at all: a botocore error
left the provider as itself and the API layer could only answer 500 with no code,
and it is the first request of every operation, so that is the least informative
place to lose the error.
Response parsing. xmltodict keys elements by the name as written, so a listing
whose root is spelled <s3:ListBucketResult>, or a body that is not XML, left
doc.get() giving back {} -- indistinguishable from an empty bucket. A folder
would list as empty and a delete would report success having found no version to
remove. _parse_listing fails closed with a 502 instead.
The commit. S3 sends the status line before it starts assembling a multi-part
upload, so a commit that fails part way through arrives as 200 with an <Error>
body; expects=(200, 201) only looks at the status and read that as a completed
upload. The body is now read, and a commit counts as successful only when the
body says so -- a CompleteMultipartUploadResult carrying an ETag, which is
computed from the assembled object and therefore exists only once the assembly
finished. An empty or scalar <Error>, a body that is not XML, a truncated or
empty body, a result with no ETag and an unknown root element are all evidence
for neither outcome, so they raise rather than reporting a file that may not be
there.
The commit is now sent exactly once: retry=0 stops core's retry loop and
allow_redirects=False stops aiohttp following a 307/308 below it, and neither
substitutes for the other. The scope is the commit alone -- the session
creation, the part uploads and the abort keep core's defaults, where a re-send
changes nothing the caller reads. The accepted cost is that a 503 or 504 on the
commit reaches the user on the first attempt instead of the third; Complete is
not idempotent, and a second attempt after a first that actually succeeded
answers NoSuchUpload, which would be read as a definitive rejection.
A failed commit now tells the user which of two things happened: the file was not
saved, or it may have been saved and the file list should be checked first. The
verdict comes from the S3 error code alone -- a table of the codes that prove
nothing was assembled; every other code, and the case where no code could be
read, is unknown and carries the notice. The asymmetry is deliberate: an
over-reported notice costs a look at the file list, an under-reported one costs a
duplicate object that only an administrator can remove. The HTTP status class is
not consulted and cannot be, since a failed commit arrives as a 200. The table
is transcribed from MinIO measurements; only EntityTooSmall was actually observed
on a CompleteMultipartUpload, and reconciliation against AWS S3 is a separate
piece of work.
This ports RCOSDP#98's three-valued outcome, with two deviations.
RCOSDP#98 suppresses the notice for quota exhaustion, because "you are out of space"
and "it may have completed" contradict each other; s3 has no quota mechanism at
all -- it is absent from ADDON_METHOD_PROVIDER and from website/util/quota.py's
PROVIDERS -- so there is no quota response here to suppress. And the abort
outcome keeps its own wording, with the notice placed between the failure
sentence and it: "is the object there?" and "is there rubbish left behind?" are
different questions and the user needs both answers.
Finally, an abort that itself raised used to replace the verdict entirely.
_abort_chunked_upload returns False only when it read answers and parts were
still there; a DELETE answering 404, 403 or 500 comes back as an exception,
raised after the notice is computed and before it is used, so both the failure
sentence and the notice were discarded and the user heard only about the cleanup.
404 NoSuchUpload on the abort is the worst case and a real answer -- it is what
S3 says when the UploadId is already consumed, which is exactly when the user
most needs to go and look. The abort is wrapped, treated as not aborted, and all
three sentences are reported. CancelledError is re-raised first.
Also fixes a test-isolation defect this work uncovered: a failing
ClientSession._request installed through monkeypatch was undone at teardown,
after the conftest hook had already deactivated aiohttpretty, which put
aiohttpretty's fake back permanently for every later test that needs a real
socket. It is restored inside the test body instead.
_abort_chunked_upload logged "failed to remove temporary parts" on success and stayed silent on failure, reporting part-residue status backwards.
… 507 Add _parse_s3_error_body / _raw_error_body to extract S3 XML error Code and Message. Map known quota codes (QuotaExceeded, XMinioAdminBucketQuotaExceeded, XMinioStorageFull) and HTTP 507 to a user-facing 507 with actionable message. Raw XML and presigned URLs are never exposed to the caller.
… add commit-outcome model - Connection errors and unreadable responses become HTTP 502. - resp.release() in raise paths prevents connection pool leaks. - QUOTA_EXCEEDED_ERROR_CODES from provider config; malformed JSON falls back to defaults with a warning instead of breaking the provider. - CompleteMultipartUpload can return 200+<Error>. _commit_outcome_note classifies commit results as NOT_COMMITTED / UNKNOWN using an error-code table. retry=0 and allow_redirects=False guarantee exactly one POST on the socket. - _parse_s3_error_body: remove regex pre-filter, rely on XML parse. - Comments and docstrings trimmed to intent-only.
…tocore errors When DeleteObjects returns HTTP 200 with per-key errors inside the body, the failures were silently discarded. Now logged as WARNING with the failed key count and first five keys. botocore exceptions are wrapped with type and code instead of surfacing as a bare 500.
…vering set Mechanically integrate and deduplicate PR-added tests using a 74-variant mutation matrix. Test functions reduced from 104 to 101 (PR-added: 73 to 31); test cases from 375 to 152. All 70 develop-branch test functions unchanged. Mutation detection: 71 killed / 3 documented equivalent / 0 mismatches — identical before and after.
tishin-endou
force-pushed
the
fix/s3compatsigv4-quota-handling
branch
2 times, most recently
from
October 2, 2026 03:26
6e82db4 to
03fd831
Compare
- _parse_s3_error_body and _check_for_200_error rewritten from isinstance chains to try/except (EAFP). _local_name_lookup raises KeyError when key absent and no default given. - Add _log_exception: outputs stack frames via traceback.format_tb (no exc_info — the formatter would append unredacted str(exc) containing presigned URLs). Exception message is redacted separately. Replaces all type(err).__name__-only log sites.
tishin-endou
force-pushed
the
fix/s3compatsigv4-quota-handling
branch
from
October 2, 2026 05:23
03fd831 to
4589eaa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
S3CompatSigV4 の機関ストレージと外部ストレージ(アドオン)の双方で、ストレージ側の
クォータを超過したとき、アップロードが失敗したことも、その原因も利用者に伝わらない
問題を修正します。
原因は独立した4系統でした。
_chunked_upload()がストレージの実エラーを捨て、常に原因不明の HTTP 500 に置き換えていた
失敗時には警告を出していなかった(残存パーツの有無を正反対に伝えていた)
接続断(
aiohttp.ClientError)が未捕捉。セッション作成が try 外CompleteMultipartUploadは失敗をHTTP 200 +
<Error>で返す。何もコミットされていないのに「成功」を返していた4 を塞ぐと、commit が失敗したとき「オブジェクトが存在するのかしないのか」を
どう伝えるかという設計判断が必要になります。これが本 PR の後半の主題です。
設計: コミット成否の3値モデル
NOT_COMMITTEDUNKNOWN行い、HTTP ステータス階級や輸送経路には依存させません。評価順は
まずクォータ判定、次に確定拒否の分類表です。分類表に載らないコードと
ベンダー固有コードは、すべて
UNKNOWN(安全側)に倒します(過剰注記は許容、誤断定は不可)
EntityTooSmallの1件のみで、残る8件は S3 仕様と MinIO のエラー定義に基づきます。表外を
UNKNOWNに倒す設計は、この未検証性を前提にしたものです(表が誤っていても、誤りは「注記が過剰に出る」側にしか倒れません)
担保は2つで、
retry=0が core の再送を、allow_redirects=Falseが aiohttp 3.6.2既定の 307/308 自動再 POST を止めます。片方でも外すと分類表が無効になるため
(実ソケット実測で
posts=2 / stored=True / 注記なし= 誤断言が再現)、どちらも外すとテストが落ちるようにしてあります
Changes
変更範囲は 3 ファイルのみです(
waterbutler/providers/s3compatsigv4/provider.py/同
settings.py/tests/providers/s3compatsigv4/test_provider.py)。QuotaExceeded/XMinioAdminBucketQuotaExceeded/XMinioStorageFull)を 507 + 対処方法に変換。応答自体が 507 ならコードによらず容量超過として扱う。判別できない場合も生の本文は返しません(Resource パスや
署名付き URL が利用者に露出するため)。容量超過は
is_user_error=TrueとしてSentry の 5xx アラート経路から外します
<Error>があれば分類できなくても 502 で raise(fail-closed)。raise 経路で漏れていた
resp.release()も塞ぎ、接続リークを解消_commit_outcome_noteという単一の純粋関数に集約し、クォータ分岐を先に評価。abort の
NoSuchUploadは鵜呑みにせずListPartsで確認しますQUOTA_EXCEEDED_ERROR_CODESは provider config で拡張可能。不正な JSON でもimport 時に落とさず(落とすと s3compatsigv4 だけが全リクエストに 404 を返す)、
警告を出して既定値に倒します
6 コミットに整理してあり、どれを単独でチェックアウトしても
tests/providers/s3compatsigv4/が全件 PASS し flake8 も通ります(git bisect可能)。f77db367f55609fe064df08b424dcd4a096f2ac74589eaa2テストは変異テストの被覆集合を用いて機械的に削減し、
コミット 5 時点で 74 種 71 killed / 3 documented equivalent / 0 mismatches で
削減前と完全一致。コミット 6 の EAFP 書き直し + 到達不能ガード削除（S10 対象コード消滅）
により 73 種 69 killed / 4 documented equivalent / 0 mismatches。
develop 由来の 70 テスト関数はすべて不変です（詳細は
FREEZE_REPORT.md§13, §14）。Side effects
500 → 502、200 +
<Error>/UploadId欠落は成功扱い → 502あります。設計上、実際にはコミットされていないケースでも付きえます(誤断定を避けるため)
「UNKNOWN + 注記」か「分類表どおりの
NOT_COMMITTED」で、誤断言ではありませんs3compatsigv4配下に限定しており、他プロバイダには影響しませんQA Notes
CI / ユニットテスト
6 コミットそれぞれを単独でチェックアウトして pytest と flake8 を実行し、6/6 で
失敗 0 件を確認しています。pinned 環境(py3.6 / aiohttp 3.6.2)での全件実行:
2003 passed / 10 skipped / 0 failed。
実機 E2E（staging2 / play.min.io）
チケット番号
https://redmine.devops.rcos.nii.ac.jp/issues/63206?issue_count=3&issue_position=1&next_issue_id=59994
run-20260920-105903。バケットはe2e-quota-59774f91（機関ストレージ）とe2e-addon-59774f91（アドオン）、いずれも hard quota 10MiB。fangorn.jsの 507 専用分岐はこの文言をxhr.status === 507のときだけ出すため、文言一致が 507 到達の独立した裏付けになるsize=129,000,000/takes_chunked_path=true/is_user_error=true)。mc ls --incompleteの出力なし = 未完了マルチパートの残留なし。なお UI 経路の chunked 超過は 900 秒無応答となったため、本 run ではプロバイダ直叩きハーネスで取得しています(無応答そのものは本 PR と独立した WaterButler 側の欠陥。スコープ外 4)200 PUT .../osfstorage/...とバケット上のオブジェクトで確認XMinioAdminBucketQuotaExceeded。settings.pyの既定値に既存のため更新不要E2E 実施 SHA と以後の変更
上表の E2E は
96b3af61に対して実施しています（デプロイ実施者が Jenkins ビルドログでcheckout Revision を目視確認）。
96b3af61は squash 前の 6 コミット目（3値モデル）にあたり、現在のコミット 3（
064df08b）の機能と同一です。以後のコミット 4〜6 の変更と E2E への影響:
424dcd4a096f2ac74589eaa2_log_exceptionヘルパ追加（traceback 保全 + 署名除去）、到達不能ガード削除 + テスト 2 件追加、what コメント削除strip_whitespace=True既定により.strip()が先に AttributeError になるため本番で到達しない。73 変異テストで 69 killed / 4 equivalent / 0 mismatches を確認）スコープ外(別チケット推奨)
s3/provider.py:276/s3compat/provider.py:377(s3compatinstitutionsにも波及)。本 PR と同じ1行s3compatinstitutionsでは 406、本 PR では 507 になる二重体系 —RDM フロントエンドの分岐確認が必要
(同一レスポンス内の言語混在を避けるため4定数まとめて扱う必要があり、
表示層の設計判断はプロバイダ層の責務ではない)
stream_request_bodyとソケットペアのデッドロック)— 本 PR の修正対象外。E2E で踏んだため記録として残す
delete_objectsの例外未捕捉 /downloadの生 XML メッセージ /_chunked_uploadのCancelledError未捕捉 /_parse_s3_error_bodyのサイズ上限 /docs ページ / 冪等でない S3 操作のリトライ禁止)
後続で提出予定の関連 PR
同一リリース判定への同乗を希望します。
Deployment Notes
特別な設定は不要です。ベンダー独自のクォータ超過コードがある場合のみ provider config で
S3COMPAT_PROVIDER_CONFIG_QUOTA_EXCEEDED_ERROR_CODESを拡張してください。