I'm trying to convert a csv file to objects in an array and write it to a new file.
I've converted the data with csv-parser, but I'm having trouble writing the array to a new file and accessing the data in the array.
This is my code converting the csv file:
const csv = require('csv-parser');
const fs = require('fs');
const readStream = fs.createReadStream('agents.csv');
const writeStream = fs.createWriteStream('agent_data.txt');
let realtorArray = [];
readStream.pipe(csv())
.on('data', (data) => {
realtorArray.push(data);
})
.on('end', () => {
//I've tried writing the array to a file in three ways
//The first way was JSON.stringify() but that failed due to the file size I think.
//Second I tried toString-ing the array as below...
writeStream.write(realtorArray.toString());
//Third way was creating a buffer...
let buf = Buffer.from(String(realtorArray));
writeStream.write(buf);
console.log('CSV file successfully processed');
writeStream.end();
});
The CSV data looks like this when converted:
[
{
'License Type': 'BRK',
'License Number': '12345',
'Full Name': 'John Doe',
'License Status': '20',
'Original License Date': '1234567',
'License Expiration Date': '12345678',
'Education Status': '0',
'Agency Identifier': '5431424'
}, {
'License Type': 'BRK',
'License Number': '4312241',
'Full Name': 'Jane Doe',
'License Status': '20',
'Original License Date': '5433234',
'License Expiration Date': '12345435',
'Education Status': '0',
'Agency Identifier': '11222234444'
},
......
]
Then when I go to write to the file in the various ways I've tried...
When I use .toString()
the data looks like this:
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]...............
When I create a buffer the data looks like this:
<Buffer 5b 6f 62 6a 65 63 74 20 4f 62 6a 65 63 74 5d 2c 5b 6f 62 6a 65 63 74 20 4f 62 6a 65 63 74 5d 2c 5b 6f 62 6a 65 63 74 20 4f 62 6a 65 63 74 5d 2c 5b 6f ... >
And when I try to read the data...
If I buffer or toString()
the data on write and use JSON.stringify(data)
, to read it:
{
"type": "Buffer",
"data": [91,111,98,106,101,99,116,32,79,98,106,101,99,........]
}
If I .toString()
it on write and try to read it, it looks the same as above or like
[object Object],[object Object],[object Object],[object Object]......
Anyone know what I'm doing wrong? And thanks for any advice in advance.