# Echo file-upload production design: receiving safely with multipart, S3 streaming, presigned URLs, and validation

> A guide to implementing file upload at production quality with Go Echo (v5). With real code, it explains: how to use c.FormFile / c.MultipartForm, S3 streaming upload that doesn't load into memory (AWS SDK Go v2), the presigned-URL approach that doesn't route huge files through the API, size limits, MIME validation, path-traversal countermeasures for filenames, and content deduplication.

- Published: 2026-06-28
- Author: 友田 陽大
- Tags: Go, Echo, セキュリティ, AWS, アーキテクチャ設計, コスト最適化
- URL: https://tomodahinata.com/en/blog/go-echo-file-upload-multipart-s3-streaming-presigned-url-guide
- Category: Go & Echo in production
- Pillar guide: https://tomodahinata.com/en/blog/go-echo-framework-production-guide

## Key points

- Don't load the whole upload into memory; stream with io.Copy. With c.FormFile → file.Open() → io.Copy, even huge files don't eat memory.
- Don't route huge files through the API; do a client→S3 direct upload with a presigned URL. It dramatically reduces server bandwidth, memory, and cost.
- Don't trust file.Filename. Using it directly as a path is path traversal. Name with a UUID etc., and validate the extension and MIME with an allowlist.
- Size limit on both BodyLimit and the S3 side. For MIME, look at the substance with http.DetectContentType on the first 512 bytes, not the extension.
- Saving to S3 is multipart-parallel with AWS SDK Go v2's manager.Uploader. For idempotency, deduplicate with a content hash as the key.

---

File upload is easy to "get working," but **receiving it safely in production** is a deep area. Memory overflows with huge files, server files get overwritten with a malicious filename, you're handed an executable with a faked extension, upload bandwidth clogs the server — none of these are prevented by "just receive it" code.

This article is the file-upload chapter of the [Go Echo production-operations guide](/blog/go-echo-framework-production-guide). Starting from Echo v5's `multipart` API, it implements, in a production-usable form: **streaming that doesn't load into memory**, the **presigned-URL approach that doesn't route huge files through the API**, and **security validation.**

> **Rules for this article**: Echo's API is based on the **official documentation (v5, as of June 2026).** AWS SDK for Go v2 is updated, so confirm the latest in the official docs. **Credentials are premised on an IAM role / environment variables**; don't write access keys in code.

---

## 1. Basics: receive one file with `c.FormFile` (streaming)

Echo v5's file reception is `c.FormFile("field")` → `*multipart.FileHeader`. The point is to **stream with `io.Copy` without reading the whole file into memory.**

```go
func (h *UploadHandler) Upload(c *echo.Context) error {
	file, err := c.FormFile("file") // *multipart.FileHeader
	if err != nil {
		return echo.NewHTTPError(http.StatusBadRequest, "file is required")
	}

	src, err := file.Open()
	if err != nil {
		return err
	}
	defer src.Close()

	dst, err := os.Create(safeName(file.Filename)) // ← ファイル名は必ずサニタイズ（第4章）
	if err != nil {
		return err
	}
	defer dst.Close()

	if _, err = io.Copy(dst, src); err != nil { // ストリーミング：メモリに全部載せない
		return err
	}
	return c.JSON(http.StatusCreated, map[string]string{"status": "uploaded"})
}
```

Multiple files are taken from `c.MultipartForm()`.

```go
form, err := c.MultipartForm()
if err != nil {
	return echo.NewHTTPError(http.StatusBadRequest, "invalid multipart form")
}
for _, file := range form.File["files"] { // <input name="files" multiple>
	src, _ := file.Open()
	// ... 同様に io.Copy でストリーミング保存
	src.Close()
}
```

> **Why streaming matters**: loading all bytes into memory with `io.ReadAll` consumes 5GB for a 100MB file × 50 concurrent requests, and the process dies on OOM. `io.Copy` transfers little by little with a fixed-size buffer, so **memory usage is constant regardless of file size.**

---

## 2. Size limit: reject "before reading" with BodyLimit

Reject huge requests by size **before starting to read the body.** Make the [middleware guide's](/blog/go-echo-middleware-cors-csrf-jwt-rate-limit-security-guide) `BodyLimit` effective on the upload path.

```go
// アップロード用グループにだけ大きめの上限を設定（v5 は整数バイト）
uploads := e.Group("/uploads")
uploads.Use(middleware.BodyLimit(25_000_000)) // 25MB を超えたら 413
```

`BodyLimit` is **the outermost layer of defense in depth.** In addition, validate on the storage side (S3 object size) and with business rules (per-plan limits). Not "protect in one place" but "protect at each layer" is the production practice.

---

## 3. Streaming save to S3: AWS SDK Go v2

In production, save uploads not to the server's local disk but to **object storage (S3, etc.).** AWS SDK for Go v2's `manager.Uploader` **uploads multipart in parallel** while accepting an `io.Reader`, so you can **stream the result of `file.Open()` directly.**

```go
import (
	"github.com/aws/aws-sdk-go-v2/feature/s3/manager"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

type S3Storage struct {
	uploader *manager.Uploader
	bucket   string
}

func NewS3Storage(client *s3.Client, bucket string) *S3Storage {
	return &S3Storage{uploader: manager.NewUploader(client), bucket: bucket}
}

func (s *S3Storage) Save(ctx context.Context, key, contentType string, body io.Reader) error {
	_, err := s.uploader.Upload(ctx, &s3.PutObjectInput{
		Bucket:      &s.bucket,
		Key:         &key,
		Body:        body,         // io.Reader をそのまま渡す＝メモリに載せない
		ContentType: &contentType,
	})
	return err
}
```

From the handler, pass `c.Request().Context()` to [propagate cancellation/timeout](/blog/go-echo-database-postgresql-pgx-sqlc-gorm-transaction-guide#1-最重要contextをハンドラからdbまで貫通させる).

```go
func (h *UploadHandler) UploadToS3(c *echo.Context) error {
	file, err := c.FormFile("file")
	if err != nil {
		return echo.NewHTTPError(http.StatusBadRequest, "file is required")
	}
	src, err := file.Open()
	if err != nil {
		return err
	}
	defer src.Close()

	contentType, err := sniffContentType(src) // 第4章：実体から MIME を判定
	if err != nil || !allowedMIME[contentType] {
		return echo.NewHTTPError(http.StatusUnsupportedMediaType, "unsupported file type")
	}

	key := "uploads/" + uuid.NewString() + extByMIME(contentType) // ファイル名は自前で決める
	if err := h.storage.Save(c.Request().Context(), key, contentType, src); err != nil {
		return err
	}
	return c.JSON(http.StatusCreated, map[string]string{"key": key})
}
```

> **Don't write credentials**: configure `s3.Client` from the SDK's default credential chain (ECS task role / EC2 instance role / environment variables). Don't hardcode access keys in code or environment variables; the correct answer is to grant least privilege (just `PutObject` on the target bucket) with an **IAM role** ([deployment secret management](/blog/go-echo-deployment-docker-distroless-ecs-cloud-run-graceful-shutdown-guide#7-シークレット管理とイメージ衛生)).

---

## 4. Security: three mandatory validations

An upload is **a mass of external input.** Observe at minimum these three.

### 4.1 Don't trust the filename (path-traversal countermeasure)

`file.Filename` is **a string the client freely decided.** Using a value like `../../etc/passwd` or `../app.go` **directly as a path lets the server's arbitrary file be overwritten.** **The server decides the filename.**

```go
// ❌ 危険：クライアント由来の名前をそのままパスに
dst, _ := os.Create("/data/" + file.Filename) // ../ でディレクトリを抜けられる

// ✅ 安全：自前で命名（UUID）。元の名前はメタデータとして「値」で保持
key := uuid.NewString() + filepath.Ext(file.Filename) // 拡張子だけは検証後に流用
// 表示用の元ファイル名は DB のカラムに保存し、パスには使わない
```

### 4.2 Look at MIME by "substance," not extension

A `.png` extension can lie. Pass **the file's first 512 bytes** to `http.DetectContentType`, judge the substance's Content-Type, and match it against an allowlist.

```go
var allowedMIME = map[string]bool{
	"image/jpeg": true, "image/png": true, "application/pdf": true,
}

func sniffContentType(rs io.ReadSeeker) (string, error) {
	buf := make([]byte, 512)
	n, err := rs.Read(buf)
	if err != nil && err != io.EOF {
		return "", err
	}
	if _, err := rs.Seek(0, io.SeekStart); err != nil { // 読んだぶんを巻き戻す
		return "", err
	}
	return http.DetectContentType(buf[:n]), nil
}
```

Since `multipart.File` satisfies `io.ReadSeeker`, you can **read the head to judge → `Seek(0)` to rewind and save.**

### 4.3 Other defenses

| Risk | Countermeasure |
| --- | --- |
| Mixing in an executable | MIME allowlist + re-attaching the extension (don't use the client extension) |
| Malware | Asynchronously scan after upload ([S3 malware scanning](/blog/aws-guardduty-malware-protection-s3-standalone-scanning-guide), etc.) and keep private until quarantine is done |
| Image bomb (decompression bomb) | Validate size/dimensions for images, then re-encode |
| Excessive upload | [RateLimiter](/blog/go-echo-middleware-cors-csrf-jwt-rate-limit-security-guide#9-ratelimiter重い処理の前で過剰アクセスを遮断) on the upload path |

---

## 5. Presigned URL: don't route huge files through the API

This is **the most important production pattern.** Uploading a video or a large image **via the API server** consumes the server's bandwidth, memory, and execution time (and billing). Instead, make the **server only issue an "upload permit (presigned URL),"** and have the **client upload directly to S3.**

```go
import "github.com/aws/aws-sdk-go-v2/service/s3"

// サーバー：アップロード先と短命の署名付き URL を発行する（ファイル本体は通さない）
func (h *UploadHandler) CreateUploadURL(c *echo.Context) error {
	claims, _ := CurrentUser(c) // 認証必須（誰のアップロードか）
	key := "uploads/" + claims.UserID + "/" + uuid.NewString()

	ps := s3.NewPresignClient(h.s3)
	req, err := ps.PresignPutObject(c.Request().Context(), &s3.PutObjectInput{
		Bucket: &h.bucket,
		Key:    &key,
	}, s3.WithPresignExpires(5*time.Minute)) // 短命：5分で失効
	if err != nil {
		return err
	}
	return c.JSON(http.StatusOK, map[string]string{"url": req.URL, "key": key})
}
```

The client **uploads directly to S3 with `PUT`** to the returned `url`. After upload completes, it notifies the server of the `key` to associate it in the DB.

| Approach | Via server | presigned-URL direct upload |
| --- | --- | --- |
| Server bandwidth | All data passes through | **Doesn't pass (just the permit)** |
| Memory/execution time | Proportional to file size | **Constant (minimal)** |
| Large files | Concern of API timeout | **S3 receives directly** |
| Cost | Server transfer/execution billing | **Greatly reduced** |
| Suited situation | Small files, immediate processing | **Video, large images, bulk upload** |

> **Rule of thumb for the judgment**: if it's a few MB or less and needs immediate validation/processing, **via server**; beyond that or bulk, **presigned URL.** Even with a presigned URL, receive upload completion with an S3 event ([S3 → Lambda async processing](/blog/aws-sqs-lambda-eventbridge-idempotent-async-processing-guide)) to connect to validation, scanning, and thumbnail generation. How to offload heavy post-processing in serverless is the "API thin, heavy processing to event-driven" standard I also adopted on the [payment/document platform](/case-studies/restaurant-matching).

---

## 6. Idempotency: don't save the same file twice

To prevent **the same file being saved multiple times** on retries or double submissions, use **the content's hash as the key.** Make it "the same content = the same key," and a re-upload just overwrites the same object, producing no duplicates.

```go
// 内容ベースのキー（content-addressed storage）
func contentKey(prefix string, sum [32]byte, ext string) string {
	return prefix + "/" + hex.EncodeToString(sum[:]) + ext // 同一内容 → 同一キー
}
```

The hash can be computed in parallel with the save stream (`io.TeeReader` for "hash while saving"). This idea of "guaranteeing idempotency structurally" is the same principle as [payment double-charge prevention](/blog/payment-double-charge-prevention-idempotency-procurement-guide).

---

## Conclusion: the 7 principles to make file upload production-quality

1. **Stream with `io.Copy`.** Don't load into memory with `io.ReadAll`.
2. **Reject by size before reading with `BodyLimit`** (the outermost layer of defense in depth).
3. **To S3, connect an `io.Reader` directly with `manager.Uploader`.** Credentials via an IAM role.
4. **Don't trust `file.Filename`.** The server names with a UUID (path-traversal countermeasure).
5. **Judge MIME by substance with `http.DetectContentType`** and match with an allowlist.
6. **Huge files via a presigned URL, client→S3 direct upload** (bandwidth, memory, cost reduction).
7. **Idempotency with a content hash** as the key, and offload post-processing to event-driven.

File upload becomes production-quality only when Echo's thin `multipart` API meshes with the design of storage, security, and cost. If you need real-time progress notifications, [WebSocket/SSE](/blog/go-echo-websocket-sse-realtime-streaming-guide); for the big picture, the [production-operations guide](/blog/go-echo-framework-production-guide).