deepmap
Problems with File Upload / Multipart Request (strict-server)
Hi,
I am new to this project and tried to create a Go server from a simple OpenAPI specification. But I didn't get what I wanted for a file upload. Would someone be so kind and put me on the right track?
The spec is my own, so I am free to change it, if this helps.
I used Version 2.4.0, but 2.2.0 gives the same result
openapi: 3.0.3
info:
title: Company Management API
paths:
/companies/{id}/upload:
post:
summary: Upload an Excel file for a company
parameters:
- name: id
in: path
required: true
schema:
type: integer
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
properties:
file:
type: string
format: binary
description: Excel file to be uploaded
companyName:
type: string
description: Company name related to the file
example: "Acme Corporation"
period:
type: string
description: Period for the data in the format YYYY-MM
example: "2023-09"
force:
type: boolean
description: Force overwrite existing data
example: false
required:
- file
- companyName
- period
responses:
'200':
description: File uploaded successfully
'400':
description: Invalid request or missing parameters
'500':
description: Server error
using this config.yml:
package: api
output: server.gen.go
generate:
gin-server: true
strict-server: true
models: true
What I get is this:
type PostCompaniesIdUploadRequestObject struct {
Id int `json:"id"`
Body *multipart.Reader
}
...
// PostCompaniesIdUploadMultipartBody defines parameters for PostCompaniesIdUpload.
type PostCompaniesIdUploadMultipartBody struct {
// CompanyName Company name related to the file
CompanyName string `json:"companyName"`
// File Excel file to be uploaded
File openapi_types.File `json:"file"`
// Force Force overwrite existing data
Force *bool `json:"force,omitempty"`
// Period Period for the data in the format YYYY-MM
Period string `json:"period"`
}
// StrictServerInterface represents all server handlers.
type StrictServerInterface interface {
// Upload an Excel file for a company
// (POST /companies/{id}/upload)
PostCompaniesIdUpload(ctx context.Context, request PostCompaniesIdUploadRequestObject) (PostCompaniesIdUploadResponseObject, error)
}
The question is:
Do I have to parse the *multipart.Reader myself when implementing the StrictServerInterface? This somehow defeats the use of the generator for this case.
The other issue I am facing is, that force is defined with type boolean. If I add a validation middleware like this:
r.Use(middleware.OapiRequestValidatorWithOptions(swagger, validatorOptions))
I get this error:
curl -X POST http://localhost:8081/companies/1/upload `
>> -H "Authorization: Bearer <your_access_token>" `
>> -F "[email protected]" `
>> -F "companyName=Acme Corporation" `
>> -F "period=2023-09" `
>> -F "force=true"
{"error":"error in openapi3filter.RequestError: request body has an error: doesn't match schema: Error at \"/force\": value must be a boolean"}
The middleware reads the request body, but interprets every property as of type string.
Can someone with more knowledge please help me how to solve the problem?
Unfortunately I have no real solution, I decode the request manually:
params, err := ParseMultipartForm(request.Body, (100 << 20)) // 100 MB
if err != nil {
return nil, err
}
fmt.Printf("Company: %v\n", params.CompanyName)
fmt.Printf("Period: %v\n", params.Period)
fmt.Printf("File: %v\n", params.File.Filename)
fmt.Printf("Force: %v\n", params.Force)
if err != nil {
return nil, err
}
and switched off the request validation.
In parallel, I am experimenting with ogen, which performs the decoding correctly.
Same here, ended up with manual parsing.
There are more troubles when you don't want to store any temporary files or load them into memory, for instance, to pass a reader directly to a Minio S3.
OpenAPI 3.x spec (simplified and redacted):
/upload/image:
post:
operationId: uploadImage
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
properties:
type:
type: string
id:
description: ID of the entity to which the image is attached
$ref: '#/components/schemas/UUID'
file:
type: string
format: binary
responses:
'200':
description: Image uploaded successfully
#...whatever you need
REST API handler
// PostUploadImage upload image for articles or profiles.
func (h *FileHand) UploadImage(ctx context.Context, req rest.UploadImageRequestObject) (rest.UploadImageResponseObject, error) {
mr := req.Body
var fileType, entityID string
for {
part, err := mr.NextPart()
if err == io.EOF {
break
}
if err != nil {
return nil, fmt.Errorf("failed to read multipart: %w", err)
}
switch part.FormName() {
case "type":
b, _ := io.ReadAll(part)
fileType = string(b)
part.Close()
case "id":
b, _ := io.ReadAll(part)
entityID = string(b)
part.Close()
case "file":
// At this point we must have type and id already read
if fileType == "" || entityID == "" {
part.Close()
return nil, fmt.Errorf("type and id must be provided before file")
}
fileName := part.FileName()
//...init metadata fields
// Prepare metadata
metadata := &models.Metadata{
OriginalName: fileName,
Size: -1, // MinIO will determine size
//...extend to your needs
}
// stream upload
uploaded, err := h.prov.Upload(ctx, part, metadata)
part.Close()
if err != nil {
return nil, err
}
return rest.UploadImage200JSONResponse{
ID: uploaded.ID,
FileName: uploaded.OriginalName,
UploadURL: uploaded.StoragePath,
ContentType: uploaded.ContentType,
}, nil
default:
part.Close()
}
}
return nil, fmt.Errorf("no file provided in request")
}
S3 wrapper/provider
func (s *Prov) Upload(ctx context.Context, file io.Reader, metadata *models.Metadata) (*models.Metadata, error) {
var (
err error
)
metadata.ID, err = uuid.NewV4()
if err != nil {
return nil, fmt.Errorf("failed to generate file ID: %w", err)
}
// Build storage path with restrictions: company_id/[user_id/]file_id_original_name
storagePath := s.buildStoragePath(metadata)
metadata.StoragePath = storagePath
opts := minio.PutObjectOptions{
ContentType: metadata.ContentType,
UserMetadata: map[string]string{
"user-id": metadata.UserID.String(),
"original-name": metadata.OriginalName,
"file-id": metadata.ID.String(),
},
}
info, err := s.client.PutObject(ctx, s.bucketName, storagePath, file, metadata.Size, opts)
if err != nil {
return nil, fmt.Errorf("failed to upload file: %w", err)
}
metadata.Size = info.Size
//... store in the database, if needed
return metadata, nil
}
Order of the fields sent by the client matters.
File should be the last, otherwise, you won't be able to assign metadata during the upload to S3. Not a big deal, can be updated afterwards, just a convenience.
