I launched Kellnr proxy cache instance in the local network, but have no stable Internet connection to update it. Is there any option for the Tor / I2P / Yggdrasil / Mycelium that I can setup my outproxy?
And the question: is the crates.io officially blocking Tor IP's? Because I can't connect there by using this method. I mean index.crates.io and static.crates.io resources.
Okay, here's the process I followed to try to reproduce this problem. To be clear, I'm running on macOS:
The guide here gives some info on how to do this, but I didn't set up the whole OS to route through Tor (the Tor project themselves recommend against doing that), just Cargo:
% brew install tor
% brew services start tor
I then checked the Tor log to make sure it actually bootstrapped successfully.
% tail -n50 /opt/homebrew/var/log/tor.log
Feb 27 12:10:20.568 [notice] Tor 0.4.9.5 running on Darwin with Libevent 2.1.12-stable, OpenSSL 3.6.1, Zlib 1.2.12, Liblzma N/A, Libzstd N/A and Unknown N/A as libc.
Feb 27 12:10:20.568 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://support.torproject.org/faq/staying-anonymous/
Feb 27 12:10:20.568 [notice] Configuration file "/opt/homebrew/etc/tor/torrc" not present, using reasonable defaults.
Feb 27 12:10:20.572 [notice] Opening Socks listener on 127.0.0.1:9050
Feb 27 12:10:20.572 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050
Feb 27 12:10:20.000 [notice] Parsing GEOIP IPv4 file /opt/homebrew/Cellar/tor/0.4.9.5/share/tor/geoip.
Feb 27 12:10:20.000 [notice] Parsing GEOIP IPv6 file /opt/homebrew/Cellar/tor/0.4.9.5/share/tor/geoip6.
Feb 27 12:10:20.000 [notice] Set list of supported TLS groups to: ?*X25519MLKEM768 / ?SecP256r1MLKEM768 / *P-256:?X25519:P-224
Feb 27 12:10:20.000 [notice] Bootstrapped 0% (starting): Starting
Feb 27 12:10:20.000 [notice] Starting with guard context "default"
Feb 27 12:10:21.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
Feb 27 12:10:21.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
Feb 27 12:10:21.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
Feb 27 12:10:22.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
Feb 27 12:10:22.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
Feb 27 12:10:22.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
Feb 27 12:10:22.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus
Feb 27 12:10:22.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
Feb 27 12:10:22.000 [notice] Bootstrapped 40% (loading_keys): Loading authority key certs
Feb 27 12:10:23.000 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services.
Feb 27 12:10:23.000 [notice] Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors
Feb 27 12:10:23.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/9687, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.)
Feb 27 12:10:24.000 [notice] Bootstrapped 50% (loading_descriptors): Loading relay descriptors
Feb 27 12:10:29.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
Feb 27 12:10:32.000 [notice] Bootstrapped 56% (loading_descriptors): Loading relay descriptors
Feb 27 12:10:33.000 [notice] Bootstrapped 63% (loading_descriptors): Loading relay descriptors
Feb 27 12:10:33.000 [notice] Bootstrapped 69% (loading_descriptors): Loading relay descriptors
Feb 27 12:10:33.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
Feb 27 12:10:34.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
Feb 27 12:10:34.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
Feb 27 12:10:34.000 [notice] Bootstrapped 100% (done): Done
I now know that the proxy is running on port 9050.
To do this, I edit my account's cargo config:
% touch ~/.cargo/config.toml
% open -a TextEdit.app ~/.cargo/config.toml
The docs for this format can be found here: https://doc.rust-lang.org/stable/cargo/reference/config.html#httpproxy. But, to make sure it's actually working, I deliberately put wrong config in here!
[http]
# the port is supposed to be 9050, but I want to make sure it's really using my config!
proxy = "socks5://localhost:9999"
I then deliberately wiped my Cargo download cache, to make sure it actually got used:
% rm -rf ~/.cargo/registry ~/.cargo/git
Finally, I tried building a crate (pulldown-cmark, in this case):
% rm -rf target
% cargo test
Updating crates.io index
warning: spurious network error (3 tries remaining): [7] Couldn't connect to server (Failed to connect to localhost port 9999 after 0 ms: Couldn't connect to server)
warning: spurious network error (2 tries remaining): [7] Couldn't connect to server (Failed to connect to localhost port 9999 after 0 ms: Couldn't connect to server)
Cool, my configuration is actually being used. Let's fix it:
[http]
proxy = "socks5://localhost:9050"
And run the build:
% rm -rf target
% cargo test
Updating crates.io index
Downloaded cast v0.3.0
Downloaded clap_lex v0.7.0
Downloaded ciborium-io v0.2.2
Downloaded cfg-if v1.0.0
Downloaded itoa v1.0.11
Downloaded is-terminal v0.4.12
Downloaded lazy_static v1.4.0
Downloaded plotters-backend v0.3.5
Downloaded anes v0.1.6
Downloaded autocfg v1.2.0
Downloaded anstyle v1.0.6
Downloaded oorandom v11.1.3
Downloaded version_check v0.9.4
Downloaded crossbeam-deque v0.8.5
Downloaded ciborium-ll v0.2.2
Downloaded bincode v1.3.3
Downloaded same-file v1.0.6
Downloaded quote v1.0.36
Downloaded plotters-svg v0.3.5
Downloaded unicase v2.7.0
Downloaded getopts v0.2.21
Downloaded ciborium v0.2.2
Downloaded either v1.11.0
Downloaded criterion-plot v0.5.0
Downloaded walkdir v2.5.0
Downloaded bitflags v2.5.0
Downloaded tinytemplate v1.2.1
Downloaded once_cell v1.19.0
Downloaded clap v4.5.4
Downloaded crossbeam-epoch v0.9.18
Downloaded unicode-width v0.1.11
Downloaded unicode-ident v1.0.12
Downloaded ryu v1.0.17
Downloaded crossbeam-utils v0.8.19
Downloaded serde_derive v1.0.198
Downloaded proc-macro2 v1.0.81
Downloaded num-traits v0.2.18
Downloaded rayon-core v1.12.1
Downloaded half v2.4.1
Downloaded serde v1.0.198
Downloaded memchr v2.7.2
Downloaded itertools v0.10.5
Downloaded criterion v0.5.1
Downloaded clap_builder v4.5.2
Downloaded plotters v0.3.5
Downloaded serde_json v1.0.116
Downloaded aho-corasick v1.1.3
Downloaded rayon v1.10.0
Downloaded regex v1.10.4
Downloaded syn v2.0.60
Downloaded regex-syntax v0.8.3
Downloaded regex-automata v0.4.6
Downloaded libc v0.2.153
Downloaded 53 crates (4.3MiB) in 3.29s
Compiling proc-macro2 v1.0.81
Compiling unicode-ident v1.0.12
Compiling crossbeam-utils v0.8.19
Compiling serde v1.0.198
Compiling memchr v2.7.2
Compiling version_check v0.9.4
Compiling autocfg v1.2.0
Compiling serde_json v1.0.116
Compiling cfg-if v1.0.0
Compiling either v1.11.0
Compiling rayon-core v1.12.1
Compiling num-traits v0.2.18
Compiling unicase v2.7.0
Compiling libc v0.2.153
Compiling aho-corasick v1.1.3
Compiling half v2.4.1
Compiling clap_lex v0.7.0
Compiling pulldown-cmark v0.10.3 (/Users/michaelhowell/pulldown-cmark/pulldown-cmark)
Compiling pulldown-cmark-escape v0.10.1 (/Users/michaelhowell/pulldown-cmark/pulldown-cmark-escape)
Compiling crossbeam-epoch v0.9.18
Compiling itoa v1.0.11
Compiling plotters-backend v0.3.5
Compiling ryu v1.0.17
Compiling quote v1.0.36
Compiling unicode-width v0.1.11
Compiling anstyle v1.0.6
Compiling crossbeam-deque v0.8.5
Compiling regex-syntax v0.8.3
Compiling syn v2.0.60
Compiling ciborium-io v0.2.2
Compiling clap_builder v4.5.2
Compiling plotters-svg v0.3.5
Compiling ciborium-ll v0.2.2
Compiling getopts v0.2.21
Compiling itertools v0.10.5
Compiling cast v0.3.0
Compiling bitflags v2.5.0
Compiling same-file v1.0.6
Compiling plotters v0.3.5
Compiling walkdir v2.5.0
Compiling is-terminal v0.4.12
Compiling rayon v1.10.0
Compiling once_cell v1.19.0
Compiling anes v0.1.6
Compiling clap v4.5.4
Compiling oorandom v11.1.3
Compiling lazy_static v1.4.0
Compiling regex-automata v0.4.6
warning: unexpected `cfg` condition name: `rustbuild`
--> pulldown-cmark/src/lib.rs:74:13
|
74 | #![cfg_attr(rustbuild, feature(staged_api, rustc_private))]
| ^^^^^^^^^
|
= help: expected names are: `docsrs`, `feature`, and `test` and 31 more
= help: consider using a Cargo feature instead
= help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(rustbuild)'] }
= help: or consider adding `println!("cargo::rustc-check-cfg=cfg(rustbuild)");` to the top of the `build.rs`
= note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
= note: `#[warn(unexpected_cfgs)]` on by default
warning: unexpected `cfg` condition name: `rustbuild`
--> pulldown-cmark/src/lib.rs:75:13
|
75 | #![cfg_attr(rustbuild, unstable(feature = "rustc_private", issue = "2781...
| ^^^^^^^^^
|
= help: consider using a Cargo feature instead
= help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(rustbuild)'] }
= help: or consider adding `println!("cargo::rustc-check-cfg=cfg(rustbuild)");` to the top of the `build.rs`
= note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
Compiling criterion-plot v0.5.0
Compiling serde_derive v1.0.198
warning: `pulldown-cmark` (lib) generated 2 warnings
Compiling regex v1.10.4
Compiling ciborium v0.2.2
Compiling bincode v1.3.3
Compiling tinytemplate v1.2.1
Compiling criterion v0.5.1
warning: `pulldown-cmark` (lib test) generated 2 warnings (2 duplicates)
Finished `test` profile [unoptimized + debuginfo] target(s) in 28.32s
Running unittests src/lib.rs (target/debug/deps/pulldown_cmark-777a23d75a61df3e)
running 52 tests
[truncated for space]
So, no, sorry. I can't reproduce your problem. It doesn't seem like crates.io is blocking Tor.
Can you please post your specific error messages?
Yes, afaik we are not explicitly blocking Tor.
We do block API requests that don't set a user agent or have a very generic user agent (like curl's default, for example) because they're the ones that don't respect the crates.io data access policy (note that setting a user agent with a means of contacting you is part of the policy).
So if you're testing connectivity by running curl to a crates.io API endpoint and getting an error, that might be why, you might not actually be blocked.
The exact error message will give more information.