Having problems deserializing a CSV produced with calamine

[egonik]

Hello!
I need to convert a XLSX files to csv. For that i made a converter using Rust and the calamine library. When I try to parse it with the csv crate I always get an UnexpectedEndOfRow error.
I tried looking into this error and I found that when lines are above a certain length they get automatically split and so the parser stops. I tried wrapping long strings in quotation marks, using a line delimiter among many other things, but none of them worked.
Here is a sample of the data I tried to parse

0,34582,IPBC,-,3-Iodo-2-propynyl butylcarbamate,55406-53-6,259-627-5,-,-,-,1,-,-,-,3-Iodo-2-propynyl butylcarbamate;iodopropynyl butylcarbamate,0,;34582;IPBC;3-Iodo-2-propynyl butylcarbamate;55406-53-6;259-627-5;3-Iodo-2-propynyl butylcarbamate;iodopropynyl butylcarbamate;,-,3-Iodo-2-propynyl butylcarbamate,V/56,PRESERVATIVE,Harmonised classification for acute toxicity/ Harmonised classification for aquatic toxicity/ Harmonised classification for eye damage/ Harmonised classification for skin sensitisation/ Harmonised classification for specific target organ toxicity/ Suspected carcinogen/ Suspected hazardous to the aquatic environment/ Suspected persistent in the environment/ Suspected skin sensitiser,"Ingrediente potencialmente cancerígeno y con capacidad para alterar la información genética de un organismo biológico, sensibilizador de la piel y con efectos duraderos en el ambiente.",40467,"Harmonised classification for acute toxicity/ Harmonised classification for aquatic toxicity/ Harmonised classification for eye damage/ Harmonised classification for skin sensitisation/ Harmonised classification for specific target organ toxicity/ Suspected carcinogen/ Suspected hazardous to the aquatic environment/ Suspected persistent in the environment/ Suspected skin sensitiser. According to the harmonised classification and labelling (ATP06) approved by the European Union, this substance is toxic if inhaled, causes damage to organs through prolonged or repeated exposure, is very toxic to aquatic life, is very toxic to aquatic life with long lasting effects, is harmful if swallowed, causes serious eye damage and may cause an allergic skin reaction. This substance is approved for use as a biocide in the EEA and/or Switzerland, for: product preservation, wood preservation, preservation for working / cutting fluids. It is being reviewed for use as a biocide in the EEA and/or Switzerland, for: preservation films, preservation of fibres, leather, rubber, or polymers, preservation for construction materials. Notwithstanding the absence of firm evidence for IPBC being dehalogenated to produce free iodine in animals or in humans, the patch test reactions to IPBC in iodine-allergic subjects were possibly caused by free iodine released from this preservative agent, thus underlining the EU restrictions regarding the use of this preservative in cosmetics.",";European Commission Legislation (Regulation (EC) N° 1223/2009 y 2019/831, Annexes I to VI and all the amendments to the cosmetics regulation).; Regulation (EU) No 528/2012 concerning the making available on the market and use of biocidal products Evaluation of active substancesAssessment Report (January 2015, Denmark); European Chemical Agency (ECHA);",NaN,"European Commission Legislation (Regulation (EC) N° 1223/2009 y 2019/831, Annexes I to VI and all the amendments to the cosmetics regulation).",-,-,Suspected carcinogen,IPBC is not toxic to reproduction or a developmental toxicant. In experimental animal studies IPBC did not affect fertility and did not cause developmental toxicity. The evidence suggests that this substance does not possess significant potential with respect to toxicity to reproduction.,"Regulation (EU) No 528/2012 concerning the making available on the market and use of biocidal products Evaluation of active substancesAssessment Report (January 2015, Denmark); European Chemical Agency (ECHA)","PBC and PBC are not included in the EU list of potential endocrine disruptors (COM DG ENV, 2000). IPBC and PBC have not been found on the Endocrine disruptor website of the European Commission: Annex 13 (List of 146 substances with endocrine disruption categorizations prepared in the Expert meeting) and 15 (List of 66 Category 1 substances with categorisation high, medium or low exposure concern). IPBC and PBC are not covered by the interim criteria described in Article 5.3 in BPR. Conclusion is that neither IPBC nor PBC has endocrine disruptor properties.","European Commission Legislation (Regulation (EC) N° 1223/2009 y 2019/831, Annexes I to VI and all the amendments to the cosmetics regulation).",Suspected skin sensitiser. IPBC is of moderate acute toxicity by the oral route and of low toxicity by the dermal route. IPBC is classified toxic by inhalation. The substance is not irritating to skin but is a severe eye irritant and a skin sensitizer.,"We suspect that, notwithstanding the absence of firm evidence for IPBC being dehalogenated to produce free iodine in animals or in humans, the patch test reactions to IPBC in iodine-allergic subjects were possibly caused by free iodine released from this preservative agent, thus underlining the EU restrictions regarding the use of this preservative in cosmetics."
"This substance is being reviewed for use as a biocide in the EEA and/or Switzerland, for: preservation films, preservation of fibers, leather, rubber, or polymers, preservation for construction materials.","Regulation (EU) No 528/2012 concerning the making available on the market and use of biocidal products Evaluation of active substancesAssessment Report (January 2015, Denmark)",-,"European Commission Legislation (Regulation (EC) N° 1223/2009 y 2019/831, Annexes I to VI and all the amendments to the cosmetics regulation).",Suspected hazardous to the aquatic environment/ Suspected persistent in the environment.,1,0,2,0,1,2,5.1
52,-,Ziram,-,-,-,-,-,-,-,-,-,-,-,-,5,;Ziram;,-,Ziram,-,-,-,Ingrediente clasificado por IARC como Grupo 3. Los agentes químicos pertenecientes a este grupo no son clasificables en cuanto a su carcinogenicidad para el ser humano. Los agentes se colocan en esta categoría cuando no se los puede incluir en ningún otro grupo.,-,Ingrediente clasificado por IARC como Grupo 3. Los agentes químicos pertenecientes a este grupo no son clasificables en cuanto a su carcinogenicidad para el ser humano. Los agentes se colocan en esta categoría cuando no se los puedo incluir en ningún otro grupo.,";International Agency for Research on Cancer. Overall Evaluations of Carcinogenicity: An Updating of IARC Monographs Volumes 1–42. IARC Monographs Supplement 7. https://monographs.iarc.fr/;European Commission Legislation (Regulation (EC) N° 1223/2009 y 2019/831, Annexes I to VI and all the amendments to the cosmetics regulation).;",3,https://monographs.iarc.fr/,"Sup 7, 53",1991,The IARC Working Group concluded that the following agents are not classifiable as to their carcinogenicity to humans.
Agents are placed in this category when they do not fall into any other group.,-,-,-,-,NaN,NaN,NaN,EU_Res 1223_2009 y 2019/831 An II,NaN,-,1,NaN,NaN,NaN,4,NaN,4.1
54,-,Mirex,-,-,-,-,-,-,-,-,-,-,-,-,5,;Mirex;,-,Mirex,-,-,-,"Ingrediente clasificado por IARC como Grupo 2B, como posiblemente cancerígenos para el ser humano. Los agentes químicos pertenecientes a este grupo cuentan con pruebas limitadas de carcinogenicidad en animales de experimentación.",-,-,;International Agency for Research on Cancer. Overall Evaluations of Carcinogenicity: An Updating of IARC Monographs Volumes 1–42. IARC Monographs Supplement 7. https://monographs.iarc.fr/;,2B,https://monographs.iarc.fr/,"20, Sup 7",1987,Group 2B - The agent is possibly carcinogenic ta humans.
"This category is generally used for agents for which there is limited evidence in humans in the absence of suficient evidence in experimental animaIs. It may also be used when there is inadequate evidence of carcinogenicity in humans or when human data are nonexistent but there is sufficíent evidence of carcinogenicity in experimental animaIs. ln some instances, an agent for which there is inadequate evidence or no data in humans but limited evidence of carcinogenicity in experimental animaIs together with supporting evidence from other relevant data may be placed in this group.",-,-,-,-,NaN,NaN,NaN,NaN,NaN,-,4,NaN,NaN,NaN,NaN,NaN,0.4

Thanks in advance!

[quinedot]

Can you share your CSV reader construction and the type being deserialized into?

[paramagnetic]

@quinedot

CSV reader construction and the type being deserialized

I don't think that matters; the CSV posted above is clearly corrupted.

@egonik Calamine doesn't seem to contain a CSV emitter. It therefore appears to me that you are generating the CSV manually and naïvely, therefore you are omitting proper escaping (among others).

Use csv::Writer instead.