GitHub warning "Low Rust analysis quality"

help
1

Does anybody understand what this warning that comes up in GitHub means?

I can't make head nor tails of it :confused:

Apparently "Percentage of macro calls with call target" is too low, but what does it mean ?!?!

Is this even a "problem" in my code that I am supposed to fix?

...or does it simply mean that GitHub's code analysis has some sort of "internal" issue?

(should this issue be reported to GitHub?)

grafik
grafik1282×724 39.2 KB

Any insight would be appreciated.

Best regards.

2

:sweat_smile: Seems that it's some jargon invented by CodeQL. Apparently I have no idea what does it mean either.

Guess so. Seems it has some difficulty around macro expansion?

Based on my search, not at all.

1 Like
3

I interpret the message as saying:

  • We failed to figure out what macros are being called,
  • therefore we could not analyze the code containing macro calls,
  • therefore the analysis will not have good coverage,
  • therefore you can’t rely on CodeQL accurately indicating lack of problems.

I’ve never used CodeQL so I can’t say whether it is actually worth using.

4 Likes
4

Supposing that the analysis tool works correctly from a technical perspective, I keep wondering where these thresholds actually come from. What scientific foundations justify them?

Even if you analyze a large collection of codebases and compute a mean, median, or some other derived value, why should code that fails to meet such an arbitrary number automatically be classified as poor quality?

In my subjective opinion, the idea that an automated analysis can simply output an OK or NACK verdict is… well, I cannot write that word here.

1 Like
5

Thank you all for your input!

6

I think they're just randomly chosen as "we need to indicate at some point that our analysis is failing", and 50% sounded good enough. Perhaps more investigation would reveal better thresholds, but that would cost money and time for dubious benefit.

Also, to my reading, the warning is that the analysis itself is low quality, not that the code is. The only reason it suggests remediation is because otherwise there may be issues that fly under the radar, due to the incomplete analysis.

3 Likes