XML -> SQL -> analysis in Rust? Any feedback appreciated

I am beginning to feel secure enough to begin a first “real” project. Still small, but if it works, I will be using it in the workplace.

The situation: my company has to fill out a questionnaire for each project it delivers in a certain field, to the government. This questionnaire is filled out in excel and then exported as XML. We collect those XML-files for analysis with Power BI.

I would like to try if I can do some analysis in Rust. My basic idea for this is:

  1. Make a table in MySQL with the same fields as the XML-files
  2. A module to read all XML-files in a certain directory, (with a xml crate) and update the MySQL table (with a sql crate)
  3. A module to make sql selections and use the results for analysis in Rust.

If this works, I want to add more tables, with data from other sources, all available in XML, to make more complex analysis possible. But that is if I succeed in the first step.

So my question really is: does this sound like the right path, or should I approach this in a different way? I am, at his moment, not asking for technical help, but for feedback on the idea itself.

(And no doubt, I will ask for technical help later on, but I want to try myself first)

If you don't have a specific reason to use SQL, it sounds like a dataframe library might be more straightforward. If you want to use Rust, you can take a look at Polars.

1 Like

Hi Abmvk,

I have done something similar before, with Delphi though. I don't see anything wrong with that. We import data from external sources, keep them permanently in a database, so that we can access the data when we need it.

To manage database migration, a fancy term for database (tables) creations and modifications etc... I have found SQLx CLI an excellent tool.

-- Personally, I like it very much. It is part of the sqlx crate, but we can use it as an independent tool to manage database development for any database application, written in any language.

Best regards,

...behai.

1 Like

One reason is I want to use my lessons in SQL from 30 years ago, finally :slight_smile:

If analyzing XML documents is the goal, then what you want is XQuery. It's a W3C standard, and allows, as the name implies, directly querying XML documents, as well as performing other computations with the results. It's pretty much made for this use case.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.