Make every retry refer to the same intended bytes
Imagine splitting 200 GB into 100 MB parts, about 2,000 parts using decimal units. Each part upload includes an upload ID, part number, and checksum. If part 417 is interrupted, the client asks which parts have been accepted and retries only what is missing. It must not create a brand-new logical file on every retry.
A subtle problem occurs if two requests use the same part number with different data. In services such as S3 multipart upload, uploading the same part number again can replace the prior part for that upload ID. Therefore a client-side retry must mean 'send the same bytes for this part', not merely 'send any bytes to the same address'. Record the expected checksum or require the upload session to be immutable at the part specification level, and validate it again during completion.
There are at least two checksum layers worth discussing. Per-part checksums detect damaged or mismatched chunks before completion. A full-object checksum checks that the assembled file matches the intended content. Don't universally treat the ETag of a multipart object as the MD5 hash of the whole file. Its interpretation depends on the provider and upload method. Choose an explicit checksum API and store the algorithm with its value.
Publishing is the moment that changes the contract
With S3-style multipart semantics, uploaded parts are not a completed object until the multipart completion call succeeds. That gives a useful single-object publication boundary. But the application's logical file might involve another record in a database: name, owner, access permissions, content version, preview status, and share links. The object completion and a database insert are not one atomic transaction.
A safe design can upload to a unique, immutable version key such as objects/account-7/upload-123/version-1, verify that object, and then conditionally change an authoritative catalog pointer from old version to new version. Readers resolve the pointer before fetching the blob. If catalog publication fails, the new immutable object is an unreferenced candidate that can be reconciled and deleted later. If the catalog succeeds but the response is lost, the client can query its session ID and receive the already-published result. Keep the session's final state and pointer update idempotent.
Consider two collaborators uploading design-final.zip. If they both finish, which one wins must be a declared policy, not an accident of request timing. We can use a compare-and-swap on the catalog's expected version number, or a provider conditional write where the object name itself is authoritative. For a shared document workspace, rejecting the second publication with a version conflict is often preferable to silent overwrite. The losing upload can remain a private version for the owner to rename.
The completion response itself can be ambiguous
Suppose the final request times out. It might have succeeded but lost its response. Starting an entirely new upload is wasteful, while blindly completing with a stale part list can be unsafe. The service must ask the provider for the object's state and verify the expected version, size, and checksum. A completed session should return the same durable result for repeated application-level completion calls.
There is a separate race with cleanup. A sweeper sees a session whose lease expired, but the last part is currently uploading. An abort request can race that part's completion. Use explicit session-state transitions, a lease or generation fence, and a cleanup grace period. At the storage-provider level, incomplete multipart uploads may incur cost until aborted, and aborting while parts are still in flight may require checking that no residual parts remain. Lifecycle cleanup is a backstop, not a substitute for the product's own session state machine.
Security belongs in the design rather than bolted on at the end. An upload token should authorize only the intended tenant, object prefix, part operation, and bounded size or expiry. Validate MIME claims and content after assembly as appropriate. A user's upload progress endpoint must not reveal another tenant's object names or checksums. Content becomes available to readers only after both integrity validation and the required authorization checks have succeeded.
I would test this with a fault matrix: connection dies during part upload, duplicate part upload with different bytes, finalization is retried, provider completes but database update fails, two publishers race for one name, and a cleanup worker wakes during finalization. Each test should verify a simple invariant: a visible file reference always resolves to one complete, verified version, and an interrupted session can be recovered or safely abandoned. That is what makes a 200 GB upload reliable. Making HTTP chunks arrive is only the beginning.
