Using Git2 to clone, create a branch and push a branch to Github

Hi!

I'm a total Rust noob and I'm trying to build a software that will interact with repos that are hosted on Github and Gitlab.

I've followed the code from How to use git2::Remote::push correctly? but I'm still encountering an error.

Here's what I'm trying to accomplish

git clone https://github.com/Scoubi/test.git
git checkout -b test 
echo Hello World > test.txt
git add test.txt
git commit -m "message"
git push origin test

Here's the code

use std::{thread, time};
use std::io::{self, BufReader};
use git2::{IndexAddOption, Repository, Signature};
use std::path::{Path, PathBuf};

fn main(){

    let repo_url = "https://github.com/Scoubi/test.git";  
    let repo_path = "/tmp/test/";

    let repo = match Repository::clone(repo_url, repo_path) {
        Ok(repo) => repo,
        Err(e) => panic!("failed to clone: {}", e),
    };

    let my_branch = "test"; 

    let head = repo.head().unwrap();
    let oid = head.target().unwrap();
    let commit = repo.find_commit(oid).unwrap();
    let branch = repo.branch(
        my_branch,
        &commit,
        false,
    );
    let obj = repo.revparse_single(&("refs/heads/".to_owned() + my_branch)).unwrap();

    repo.checkout_tree(
        &obj,
        None
    );

    repo.set_head(&("refs/heads/".to_owned() + my_branch));

    // based on : https://github.com/rust-lang/git2-rs/issues/561
    let file_name = "test.txt";
    create_file(&repo_path, file_name);
    git_add_all(&repo);
    git_commit_push(&repo, repo_url, my_branch);

}

fn create_file(repo_path:&str, file_name: &str) {
//    let file_path = repo_path.join(file_name);
    let file_path= format!("{}{}", repo_path, file_name);
    std::fs::File::create(file_path).unwrap();
//    println!("{}", file_path)
}

fn git_add_all(repo: &git2::Repository) {
    let mut index = repo.index().unwrap();
    index
        .add_all(&["."], git2::IndexAddOption::DEFAULT, None)
        .unwrap();
    index.write().unwrap();
}

fn git_commit_push(repo: &git2::Repository, repo_url: &str, my_branch: &str) {
    let mut index = repo.index().unwrap();
    let tree = repo
		.find_tree(index.write_tree().unwrap())
		.unwrap();
    let author = Signature::now("x", "x@x.xxx").unwrap();

    //let mut parents = vec![];
    let mut update_ref = Some("HEAD");

    if let Ok(head) = repo.head() {
        update_ref = Some("HEAD");
    } else {
        update_ref = None; // no HEAD = first commit
    }
    let parent_commit = repo.head().unwrap().peel_to_commit().unwrap();

    dbg!(&parent_commit);
    dbg!(update_ref);
    
	let commit_oid = repo
		.commit(update_ref, &author, &author, "commit message", &tree, &[&parent_commit])
		.unwrap();

    //update branch pointer
    let branch = repo
    .branch(my_branch, &repo.find_commit(commit_oid).unwrap(), true)
    .unwrap();
    let branch_ref = branch.into_reference();
    let branch_ref_name = branch_ref.name().unwrap();
    repo.set_head(branch_ref_name).unwrap();

	// add remote as "origin" and push the branch
	let mut origin = repo.remote("origin", my_branch).unwrap();
    origin.push(&[branch_ref_name], None).unwrap();

}

First time that I run it I get the error
404: Not Found
thread 'main' panicked at 'failed to clone: remote authentication required but no callback set; class=Http (34); code=Auth (-16)'

If I clone it manually once, delete the folder and run the code again I get the following error
update_ref = Some(
"HEAD",
)
thread 'main' panicked at 'called Result::unwrap() on an Err value: Error { code: -1, klass: 4, message: "cannot force update branch 'test' as it is the current HEAD of the repository." }', src/bin/main.rs:84:6

Any help would be appreciated, I've been running in circle for 2 days now :frowning:

I'd like to fix the code and also know how I'm supposed to use the token to authenticate to github. I generated my token and it works when I use it in the shell, but haven't figured how to use it successfully in Rust.

Thanks

It sounds like you need to you call connect_auth() on your origin remote and make sure you pass a RemoteCallbacks which provides a credentials callback that returns your token.

That part seems to work if I manually clone first.

But I'm still unable to PUSH my new branch to the origin (Github.com) I still get the error

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error 
{ code: -1, klass: 4, message: "cannot force update branch 'test' as it is the 
current HEAD of the repository." }', src/main.rs:85:22

But if I go in the directory and manually issue git push origin test it works.

So the branch is getting created, the file is being created, the file is correctly added, the commit is there as well. Only the push remains broken.

The error seems to be within this bit

    //update branch pointer
    let branch = repo
                    .branch(my_branch, &repo.find_commit(commit_oid).unwrap(), true)
                    .unwrap();

I've tried changing my_branch to "main" but that's not it either. I feel like I'm very close but there's still a small error somewhere in my git logic/parameters.

Here's some git output from the console

git branch
  main
* test
git log
commit 7df348c094060c49707d6b56ffdbf31bb4f464e7 (HEAD -> test)
Author: x <x@x.xxx>
Date:   Wed Oct 4 09:55:15 2023 -0400

    commit message

commit 1ea1024528f3647cdcb10c3e05759c9533e32024 (origin/main, origin/HEAD, main)
Author: Scoubi <Scoubi@users.noreply.github.com>
Date:   Fri Sep 22 15:27:57 2023 -0400

    Update README.md

commit fdbfc64b17e2ec4ec015fe61a3f2ddbabbebd962
Author: Scoubi <Scoubi@users.noreply.github.com>
Date:   Fri Sep 22 15:27:26 2023 -0400

    Initial commit
git show-ref
1ea1024528f3647cdcb10c3e05759c9533e32024 refs/heads/main
7df348c094060c49707d6b56ffdbf31bb4f464e7 refs/heads/test
1ea1024528f3647cdcb10c3e05759c9533e32024 refs/remotes/origin/HEAD
1ea1024528f3647cdcb10c3e05759c9533e32024 refs/remotes/origin/main
git show HEAD
commit 7df348c094060c49707d6b56ffdbf31bb4f464e7 (HEAD -> test)
Author: x <x@x.xxx>
Date:   Wed Oct 4 09:55:15 2023 -0400

    commit message

diff --git a/test.txt b/test.txt
new file mode 100644
index 0000000..e69de29

that's not how you update an branch pointer, and why the unwrap() panics. Repository::branch() is for creating new branches.

if you commit to an active branch, repo.commit(Some("HEAD"),...) will do. if you commit to an inactive branch, you can provide the actual branch ref to the commit call, something like repo.commit(Some(branch.get().name().unwrap()),...)

Hi @nerditation!

Very happy you're here/found this thread!
That code comes from an example you've given here : How to use git2::Remote::push correctly? - #6 by nerditation

It is very likely I didn't fully understood the workflow and made the wrong modifications.

So the commit seems to work in another part of the code. I think the only thing missing is actually push the new branch back to origin. Kind of git push origin test But so far, I've failed at find how to do it. The branch I want to push should be the active one and the one with the latest modifications.
Any tips?

Thanks!

I don't know, your previous comment indicates the panic happens at this line:

as I said, you don't need to "update" the current branch pointer, it's automatically updated when you commit.

on the other hand, your statement saiys you have problem push-ing, but I don't see any obvious error in the code to push, i.e. these lines looks fine:

can you clarify what's the real problem you try to solve?

Hi @nerditation,

I've commented the update branch pointer section of the code and now I get : remote authentication required but no callback set

I'm trying to authenticate like this


	let commit_oid = repo
		.commit(update_ref, &author, &author, "commit message", &tree, &[&parent_commit])
		.unwrap();

    let mut callbacks = RemoteCallbacks::new();
    callbacks.credentials(|_url, username_from_url, _allowed_types| {
        Cred::userpass_plaintext("Scoubi", "MY_GITHUB_PASSWORD")
    });
    let mut origin = repo.find_remote("origin").unwrap();
    origin.connect_auth(git2::Direction::Push, Some(callbacks), None);
    origin.push(&["refs/heads/test:refs/remotes/test"], None).unwrap();

Ideally I would prefer to use a Github token that I have created instead of User/Pass, but haven't found out how to use that yet.. I'm guessing that's not how callbacks work, but I haven't found good sample code yet.

Thanks

You have to use the github token in the place of the password. In any case github has disabled password authentication a while ago for security reasons.

see

I ever only used ssh and don't know how the github authenticates https protocol, but I guess PushOption::custom_headers() might be relevant?

Ok, so I've figured out how to use the Access Token for authentication. It's actually pretty easy.
Found the answer in this post : git - Clone A Private Repository (Github) - Stack Overflow

git clone https://<TOKEN>@github.com/user/repo.git

But now I get :

called `Result::unwrap()` on an `Err` value: Error { code: -4, klass: 7, message: "remote 'origin' already exists" }

This is the code I have and the first line seems to be the problem.

    let mut origin = repo.remote("origin", &repo_url).unwrap();
    origin.push(&[my_branch], None).unwrap();

When I look in my repo and try to do a manual git push I get :

fatal: The current branch test has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin test

To have this happen automatically for branches without a tracking
upstream, see 'push.autoSetupRemote' in 'git help config'.

I feel like I'm really close to the solution.. but not quite there yet.

did you read the panic message? it tells you the reason! it says:

message: "remote 'origin' already exists"

you are adding a new remote with the same name! read the document carefully:

pub fn find_remote(&self, name: &str) -> Result<Remote<'_>, Error>

Get the information for a particular remote

pub fn remote(&self, name: &str, url: &str) -> Result<Remote<'_>, Error>

Add a remote with the default fetch refspec to the repository’s configuration.

you either find and use existing remote, or add the new remote with different name. if you are making a tool, you must consider the different configurations of a repo, you can't just assume the repository is always in the same state.

that's a totally different error. it's specific to the git command and irrelevant to your example code with libgit2. on the command line, you typically do git push without explicitly specify the local branch remote branch, so the current HEAD (if it points to a branch, that is) and its "tracking remote branch" is used.

for libgit2, you do the push with explicit self: &mut Remote and refspecs: &[Str] arguments, so you don't need a "tracking" remote, or even a "current" branch to be configured for the repo.

btw, you can overwrite the local branch and remote branch on the command line too. see the manual for details.


some words:

it seems to me you are not familiar with basic concepts on how git (and libgit2) works. your problem is not rust specific. if you don't know what you are doing, even if you managed to get the code to compile, it might even just "happens to work", it probably is still incorrect.

for basic git concepts, I recommend you read the git book, especially Chapter 10: Git Internals.

for libgit2 usage, I recommend the 101 libgit2 samples from the official website, but you should really learn the git basics first.

Hi,

Yes you are quite right. My git knowledge is also pretty limited, I know how to do basic things in the cli and that's about it. Thanks for the link I will review.

Unlike other people (perhaps you?) I don't really learn vwell by reading things, I don't know why. I need to do it to learn it. That's why I'm working on this personal project, to learn about git & rust.

Thanks for pointing the find_remote function, when you point it out, it's pretty obvious that's what was needed.

For anyone who ever come across this post in search of some code, I finally manage to get a working code, so I'll add it here.

use git2::{Repository, Signature};

fn main(){

    let repo_url = "https://YOUR_GITHUB_TOKEN_GOES_HERE@github.com/User/repo.git";  
    let repo_path = "/tmp/test/";

    let repo = match Repository::clone(repo_url, repo_path) {
        Ok(repo) => repo,
        Err(e) => panic!("failed to clone: {}", e),
    };

    let my_branch = "test"; 

    let head = repo.head().unwrap();
    let oid = head.target().unwrap();
    let commit = repo.find_commit(oid).unwrap();
    let branch = repo.branch(
        my_branch,
        &commit,
        false,
    );
    let obj = repo.revparse_single(&("refs/heads/".to_owned() + my_branch)).unwrap();

    repo.checkout_tree(
        &obj,
        None
    );

    repo.set_head(&("refs/heads/".to_owned() + my_branch));

    // based on : https://github.com/rust-lang/git2-rs/issues/561
    let file_name = "test.txt";
    create_file(&repo_path, file_name);
    git_add_all(&repo);
    git_commit_push(&repo, repo_url, my_branch);

}

fn create_file(repo_path:&str, file_name: &str) {
    let file_path= format!("{}{}", repo_path, file_name);
    std::fs::File::create(file_path).unwrap();
}

fn git_add_all(repo: &git2::Repository) {
    let mut index = repo.index().unwrap();
    index
        .add_all(&["."], git2::IndexAddOption::DEFAULT, None)
        .unwrap();
    index.write().unwrap();
}

fn git_commit_push(repo: &git2::Repository, repo_url: &str, my_branch: &str) {
    let mut index = repo.index().unwrap();
    let tree = repo
        .find_tree(index.write_tree().unwrap())
        .unwrap();
    let author = Signature::now("x", "x@x.xxx").unwrap();

    let mut update_ref = Some("HEAD");

    if let Ok(head) = repo.head() {
        update_ref = Some("HEAD");
    } else {
        update_ref = None; // no HEAD = first commit
    }

    let parent_commit = repo.head().unwrap().peel_to_commit().unwrap();
    
    let commit_oid = repo
        .commit(update_ref, &author, &author, "commit message", &tree, &[&parent_commit])
        .unwrap();

    let mut origin = repo.find_remote("origin").unwrap();
    origin.push(&["refs/heads/".to_owned() + my_branch], None).unwrap();
}

There's probably ways to improve the code (check for errors instead of just .unwrap() but it's a good starting point.

Thanks to everyone who help and particularly @nerditation !