Issue in unescaping the escape-sequences in Rust

[kumar2000s]

Hi I am trying to create a Splunk CLI tool , that CLI reads the TOML config and get the search string. the search string contains double quotes (") and when rust reads it, it adds escape characters to it like (\") as expected.

now the real problem is that when i used this string with escapes characters it give me error as Splunk Search does not support escape characters in eval commands.
is there a way i can remove escape sequence from string before passing it to Splunk API ?

here is code snippet:

Config File:

searches = [ 
    'index="my_index" source=apache* earliest=-1h@h latest=@h | eval alert_name = host+"_"+source'
    ]

Here is the Struct

#[derive(Debug,Serialize,Deserialize)]

pub struct SplunkConfig{
    pub searches: Vec<String>,
}

here is the program output

PS C:\kumar\Rust\rust_projects\splunkcli> cargo run -- -f C:\kumar\Rust\rust_projects\splunkcli\search.toml
   Compiling splunkcli v0.2.0 (C:\kumar\Rust\rust_projects\splunkcli)
    Finished dev [unoptimized + debuginfo] target(s) in 7.32s
     Running `target\debug\splunkcli.exe -f C:\kumar\Rust\rust_projects\splunkcli\search.toml`
[2021-08-21T19:51:14Z INFO  splunkcli] search file name: Some("C:\\kumar\\Rust\\rust_projects\\splunkcli\\search.toml")
[2021-08-21T19:51:14Z INFO  splunkcli] Search query is: "index=\"my_index\" source=apache* earliest=-1h@h latest=@h | eval alert_name = host+\"_\"+source"
[2021-08-21T19:51:16Z INFO  splunkcli::splunk] Search request status code is : 201 Created
[2021-08-21T19:51:16Z INFO  splunkcli] Search SID is: 1629575475.684825_ED51B6CA-996E-4498-AF35-DBCF822DB2BE
[2021-08-21T19:51:16Z INFO  splunkcli::splunk] Search request status is: false
[2021-08-21T19:51:27Z INFO  splunkcli] Search sacn count is: 0
[2021-08-21T19:51:27Z INFO  splunkcli] Search event count is: 0
[2021-08-21T19:51:27Z INFO  splunkcli] Search result count is: 0
[2021-08-21T19:51:27Z INFO  splunkcli] Search run duration is: 0.004
[2021-08-21T19:51:28Z INFO  splunkcli::splunk] Status code for search result is : 400 Bad Request
Err(
    CustomStatusCode {
        status_code: "400",
        msg: "{\"messages\":[{\"type\":\"FATAL\",\"text\":\"Error in 'eval' command: The expression is malformed.\"}]}",        
    },
)

[Heliozoa]

It looks like you're printing using the debug representation, which will include escape characters, but the strings shouldn't actually contain any. For example

let s = "\"string\" with escapes";
println!("{}", s);
println!("{:?}", s);

will print

"string" with escapes
"\"string\" with escapes"

If you print out the search string using something like

println!("{}", config.searches[0]);

does it still contain the slashes? If not, there's probably some other issue with the string or how you're passing it forwards. If you're not familiar with the difference between {}, {:?} and such, you can find more info on formatting in the std::fmt docs.

[kumar2000s]

Thanks for pointing out, let me check this and see if the issue is somewhere else .will update you with my findings

[kumar2000s]

Ok, I got the issue. basically the search query is fine. but when it send the post request to Splunk servers, the splunk servers receive the query without (+) sign in eval expression as shown below:

index="my_index" source=apache* earliest=-1h@h latest=@h | eval alert_name = host "_" source

here is the request which i am sending, this is having the perfect query, not sure where the (+) sign missing in between.

Request {
    method: POST,
    uri: https://mysplunk.com/services/search/jobs/,
    version: HTTP/1.1,
    headers: {
        "content-type": "application/x-www-form-urlencoded",
        "authorization": "Basic xyzzzzzz",
    },
    body: Body(
        Full(
            b"search=search index=\"my_index\" source=apache* earliest=-1h@h latest=@h | eval alert_name = host+\"_\"+source",
        ),
    ),
}

here is the actual code for request builder code

    let req = Request::builder()
    .method(Method::POST)
    .uri(&splunk_api_url)
    .header("content-type", "application/x-www-form-urlencoded")
    .header("Authorization",&credentials)
    .body(Body::from(search_body))?;

    let https = HttpsConnector::new();
    let client = Client::builder().build::<_, hyper::Body>(https);
  
    println!("{:#?}",&req);
    let search_request = client.request(req).await?;

[Heliozoa]

I'm guessing you need to url encode the body to make sure special characters get through. Try running the string through urlencoding::encode from https://crates.io/crates/urlencoding before sending it. If that doesn't work you may need to look at splunk's docs to see what could be going wrong.

[kumar2000s]

Thanks Heliozoa for all your suggestion and BIG HELP.
the issue got resolved by replacing (+) sign by Unicode

here is the updated toml config, just changed (+) with (%2B) and it works

searches = [ 
    'index="my_index" source=apache* earliest=-1h@h latest=@h | eval alert_name = host%2B"_"%2Bsource'
    ]

[kumar2000s]

Hi Everyone, just want to thank everyone for their support, now my CLI is live in prod and working perfectly fine. Rust is Awesome