How can I string-split a string field into a list field in Arrow Datafusion?

help
1

I have a CSV-file people.csv which looks like this:

name;age;hobbies
Alice;25;dance,music
Bob;30;music
Charlie;28;football,tv

I can read the file like this:

let people = ctx.read_csv("people.csv", CsvReadOptions::new().has_header(true).delimiter(';' as u8)).await?;

This results in the field 'hobbies' being a single string (Utf8) field. However, its values are comma-separated strings. I would like to transform that field into a "list-of-string" field by splitting the strings on comma.
Is this possible in DataFusion? I know that Arrow has list-type, but I'm not sure whether DataFusion supports or allows this.

2

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.