How to create http request with file in body?

1

Typical POST request I get from request builder in golang and dump request to plain. This is request example

POST /upload HTTP/1.1
Host: localhost:4500
Accept: */*
Content-Length: 191
Content-Type: multipart/form-data; boundary=------------------------c9ef9ede1067aaa2
User-Agent: curl/8.2.1

--------------------------c9ef9ede1067aaa2
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain

test

--------------------------c9ef9ede1067aaa2--

The boundary can be almost anyone html - What is the boundary in multipart/form-data? - Stack Overflow i made a random string

    let boundary = rand_str(60);
    let mut stream = std::net::TcpStream::connect("127.0.0.1:4500").unwrap();
    stream
        .write("POST /upload HTTP/1.1\r\nHost:127.0.0.1:4500\r\n".as_bytes())
        .unwrap();
    stream
        .write(
            format!(
                "Content-Type: multipart/form-data;bondary={}\r\n",
                &boundary
            )
            .as_bytes(),
        )
        .unwrap();
    stream
        .write(format!("--{}\r\n", &boundary).as_bytes())
        .unwrap();
    stream
        .write(
            "Content-Disposition: form-data; name=\"file\"; filename=\"content.dat\"\r\n"
                .as_bytes(),
        )
        .unwrap();
    stream
        .write("Content-Type: application/octet-stream\r\n".as_bytes())
        .unwrap();
    stream.write(&data).unwrap();
    let _ = match stream.write(format!("\r\n--{}--\r\n\r\n", boundary).as_bytes()) {
        Ok(_) => {}
        Err(err) => panic!("{}", err), //always panic: `connection reset by peer`
    };
    stream.flush().unwrap();

Help me, how I can make POST request with mulipart over TcpStream? Thanks for answers!

2

You should probably use at least an HTTP library for that, e.g. hyper. Better yet, use a higher-level client library, e.g. reqwest.

3

Thanks for answer, but I do not use any http clients. I want to anderstand how it works at low level.

4

What happens when you use the code you posted?

5

Always panic with connection reset by peer at

let _ = match stream.write(format!("\r\n--{}--\r\n\r\n", boundary).as_bytes()) {
        Ok(_) => {}
        Err(err) => panic!("{}", err), //always panic
    };
6

Can you debug the server to see why it's rejecting it?

7

With this difficulty, here is my server code

package main

import (
	"errors"
	"fmt"
	"io"
	"log"
	"net/http"
	"net/http/httputil"
	"os"
	"path/filepath"
	"time"
)

const MAX_UPLOAD_SIZE = 1024 * 1024 * 1024 * 1024

func IndexHandler(w http.ResponseWriter, r *http.Request) {
	w.Header().Add("Content-Type", "text/html")
	http.ServeFile(w, r, "index.html")
}

func uploadHandler(w http.ResponseWriter, r *http.Request) {
	log.Println("[handler call]", r.RemoteAddr)
	reqdump, _ := httputil.DumpRequest(r, true)
	os.WriteFile("req.txt", reqdump, 0755)
	if r.Method != "POST" {
		log.Println(errors.New("Method not POST"))
		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
		return
	}

	if err := r.ParseMultipartForm(32 << 20); err != nil {
		log.Println(err)
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}

	files := r.MultipartForm.File["file"]

	for _, fileHeader := range files {
		if fileHeader.Size > MAX_UPLOAD_SIZE {
			log.Println(fmt.Sprintf("The uploaded image is too big: %s.", fileHeader.Filename), http.StatusBadRequest)
			http.Error(w, fmt.Sprintf("The uploaded image is too big: %s.", fileHeader.Filename), http.StatusBadRequest)
			return
		}

		file, err := fileHeader.Open()
		if err != nil {
			log.Println(err)
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		defer file.Close()

		buff := make([]byte, 512)
		_, err = file.Read(buff)
		if err != nil {
			log.Println(err)
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		_, err = file.Seek(0, io.SeekStart)
		if err != nil {
			log.Println(err)
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		err = os.MkdirAll("./uploads", os.ModePerm)
		if err != nil {
			log.Println(err)
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		f, err := os.Create(fmt.Sprintf("./uploads/%d%s", time.Now().UnixNano(), filepath.Ext(fileHeader.Filename)))
		if err != nil {
			log.Println(err)
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		defer f.Close()

		_, err = io.Copy(f, file)
		if err != nil {
			log.Println(err)
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}
	}

	fmt.Fprintf(w, "ok")
}

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/", IndexHandler)
	mux.HandleFunc("/upload", uploadHandler)

	if err := http.ListenAndServe(":4500", mux); err != nil {
		log.Fatal(err)
	}
}

And debugs will works with curl, but not works with rust code. I mean log.Println not say anything when I run rust code.

8

You misspelled boundary here

9

Thanks, but this not solve problem :slight_smile:

thread 'main' panicked at 'Connection reset by peer (os error 104)', src/main.rs:600:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[8]  + 69141 IOT instruction  cargo r --release
10

this is the POST request dump from curl -F file=@/tmp/test.txt http://localhost:4500/upload, file stored to filesystem.

POST /upload HTTP/1.1
Host: localhost:4500
Accept: */*
Content-Length: 198
Content-Type: multipart/form-data; boundary=------------------------46b4fe28f353713c
User-Agent: curl/8.2.1

--------------------------46b4fe28f353713c
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain

test qwerty

--------------------------46b4fe28f353713c--

This is request implementation in rust

let mut stream = std::net::TcpStream::connect("127.0.0.1:4500").unwrap();
    stream
        .write("POST /upload HTTP/1.1\r\n".as_bytes())
        .unwrap();
    stream.write("Host: localhost:4500\r\n".as_bytes()).unwrap();
    stream.write("Accept: */*\r\n".as_bytes()).unwrap();
    stream.write("Content-Length: 198\r\n".as_bytes()).unwrap();
    stream
        .write("Content-Type: multipart/form-data; \r\n".as_bytes())
        .unwrap();
    stream
        .write("boundary=------------------------46b4fe28f353713c\r\n".as_bytes())
        .unwrap();
    stream
        .write("User-Agent: curl/8.2.1\r\n\r\n".as_bytes())
        .unwrap();
    stream
        .write("--------------------------46b4fe28f353713c\r\n".as_bytes())
        .unwrap();
    stream
        .write(
            "Content-Disposition: form-data; name=\"file\"; filename=\"test.txt\"\r\n".as_bytes(),
        )
        .unwrap();
    stream
        .write("Content-Type: text/plain\r\n\r\n".as_bytes())
        .unwrap();
    stream.write("test qwerty\r\n\r\n".as_bytes()).unwrap();
    stream
        .write("--------------------------46b4fe28f353713c--\r\n\r\n".as_bytes())
        .unwrap();
    stream.flush().unwrap();

Have no errors, but file not save to filesystem and any debug prints...

11 
    stream.flush().unwrap();
    let mut buffer = String::new();
    let _ = stream.read_to_string(&mut buffer).unwrap();
    println!("{}", buffer);

Hmmmm

HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close

400 Bad Request
12

stream.write() isn't guaranteed to write all of the buffer. You are not checking its return value (the number of bytes actually written), so you probably meant stream.write_all() instead.

2 Likes
13

Just in case you change your mind:

use reqwest::{multipart::Form, multipart::Part, Client};

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let bytes1: Vec<u8> = vec![b't', b'e', b's', b't'];
    let bytes2: Vec<u8> = vec![b'm', b'o', b'r', b'e', b' ', b'd', b'a', b't', b'a'];
    let client = Client::new();
    let form = Form::new()
        .part("data1", Part::bytes(bytes1))
        .part("data2", Part::bytes(bytes2));
    client
        .post("http://127.0.0.1:4500/upload")
        .multipart(form)
        .send()
        .await?;
    Ok(())
}

Server:

use axum::{extract::Multipart, routing::post, Router};

async fn upload(mut multipart: Multipart) {
    while let Some(field) = multipart.next_field().await.unwrap() {
        let name = field.name().unwrap().to_string();
        let data = field.bytes().await.unwrap();

        println!("Length of `{}` is {} bytes", name, data.len());
    }
}

#[tokio::main]
async fn main() {
    let app = Router::new().route("/upload", post(upload));
    axum::Server::bind(&"0.0.0.0:4500".parse().unwrap())
        .serve(app.into_make_service())
        .await
        .unwrap();
}
14

While tracing through that server using your client, I noticed that Axum was dropping the stream content which contains the multipart payload. Something about how you're formatting it confuses Axum, and likely the server framework you're using. I stopped trying to track it down, but my current guess is it might be because you don't use content-length in the main header.

15

Content-length does not matter


fn read_file(filename: &str) -> Vec<u8> {
    let mut f = std::fs::File::open(&filename).unwrap();
    let metadata = std::fs::metadata(&filename).unwrap();
    let mut buffer = vec![0; metadata.len() as usize];
    f.read(&mut buffer).unwrap();

    buffer
}

fn send(_data: Vec<u8>) {
    let mut stream = std::net::TcpStream::connect("127.0.0.1:4500").unwrap();
    stream
        .write_all("POST /upload HTTP/1.1\r\n".as_bytes())
        .unwrap();
    stream
        .write_all("Host: localhost:4500\r\n".as_bytes())
        .unwrap();
    stream.write_all("Accept: */*\r\n".as_bytes()).unwrap();
    stream
        .write_all(format!("Content-Length: {}\r\n", &_data.len()).as_bytes())
        .unwrap();
    stream
        .write_all("Content-Type: multipart/form-data; boundary=------------------------46b4fe28f353713c\r\n".as_bytes())
        .unwrap();
    stream
        .write_all("User-Agent: stream_client\r\n\r\n".as_bytes())
        .unwrap();
    stream
        .write_all("--------------------------46b4fe28f353713c\r\n".as_bytes())
        .unwrap();
    stream
        .write_all(
            "Content-Disposition: form-data; name=\"file\"; filename=\"test.txt\"\r\n".as_bytes(),
        )
        .unwrap();
    stream
        .write_all("Content-Type: text/plain\r\n\r\n".as_bytes())
        .unwrap();
    stream.write_all(&_data).unwrap();
    stream.write_all("\r\n\r\n".as_bytes()).unwrap();
    stream
        .write_all("--------------------------46b4fe28f353713c--\r\n\r\n".as_bytes())
        .unwrap();
    stream.flush().unwrap();
    let mut buffer = String::new();
    let _ = stream.read_to_string(&mut buffer).unwrap();
    println!("{}", buffer);
}

result is the same

HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close

400 Bad Request
16

Look at the requests curl is making in your OP and later reply carefully, there's one big disrepancy with what you're doing in your code: the Content-Length is much larger. The Content-Length header is not for the total size of the files you are sending or anything like that, instead it's the length of the content of your HTTP message, meaning everything after the headers.

https://www.rfc-editor.org/rfc/rfc9112#name-content-length:

the Content-Length field value provides the framing information necessary for determining where the data (and message) ends

For example, in the OP, the Content-Length: 191 indicates the length of

--------------------------c9ef9ede1067aaa2
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain

test

--------------------------c9ef9ede1067aaa2--

not the length of test.

4 Likes
17

Thanks, that works!