CSV¶

This module provides functions for working with CSV (Comma Separated Values) data in TypeScript. It includes functions for parsing CSV data into an array of rows (each of which is an array of cell values), and for converting an array of rows into CSV data as a string.

Introduction to CSV¶

CSV is a simple file format used to store tabular data, such as a spreadsheet or database. It stores data in plain text, with each line representing a row and each field (column) within that row separated by a comma or other delimiter. The first line usually contains the names of the fields, also known as the header row.

For example, a CSV file might look like this:

name,email,age
John Smith,john.smith@example.com,30
Jane Doe,jane.doe@example.com,25

This CSV file has three fields (name, email, and age) and two rows of data. The first row contains the field names, and the second and third rows contain the data for each person.

To escape special characters within cell values in a CSV file, you can use double quotes to surround the cell value. For example, if a cell value contains a comma, you can escape it by surrounding the value with double quotes:

"John,Smith","john.smith@example.com","30"

If a cell value contains a double quote character, you can escape it by using two double quotes in a row:

"John Smith","john.smith@example.com","He said ""Hello"" to me."

CSV files are commonly used to transfer data between programs, such as from a database to a spreadsheet application or vice versa. They are also used to import and export data for websites and other applications.

The MIME type for CSV files is text/csv. This is the correct MIME type to use when serving CSV files over the internet, or when specifying the type of CSV file in an HTML <input type="file"> element.

Importing the core module¶

Once the fi.hg.core module is installed, you can import it in your TypeScript code using the following syntax:

import { Csv, parseCsvRow, stringifyCsvRow } from './fi/hg/core/csv';

Parsing CSV Data¶

To parse a CSV string into an array of rows, you can use the parseCsv function. This function takes a string as input and returns an array of rows, where each row is an array of cells.

Here is an example of how to use the parseCsv function:

import { Csv, parseCsv } from './fi/hg/core/csv';

const csvString = 'id,name,age\n1,Alice,30\n2,Bob,40\n';
const csv: Csv = parseCsv(csvString);

console.log(csv); // [['id', 'name', 'age'], ['1', 'Alice', '30'], ['2', 'Bob', '40']]

You can also use the parseCsvRow function to parse a single row of CSV data. This function takes a string as input and returns an array of cells.

Here is an example of how to use the parseCsvRow function:

import { parseCsvRow } from './fi/hg/core/csv';

const csvRowString = '1,Alice,30';
const csvRow: string[] = parseCsvRow(csvRowString);

console.log(csvRow); // ['1', 'Alice', '30']

Generating CSV data¶

The HG CSV module provides several functions for generating CSV data as a string from an array of rows or from a list of objects.

Converting cell values to strings¶

Before we can generate CSV data, we need to be able to convert individual cell values into strings suitable for inclusion in a CSV file. The stringifyCsvCellValue function can be used for this purpose. It takes a single cell value as a parameter, and returns a string representation of the value.

If the input value is an array, the elements of the array are joined with commas. If the input value is undefined, an empty string is returned. Otherwise, the value is simply converted to a string using the toString method.

Here's an example of using the stringifyCsvCellValue function:

import { stringifyCsvCellValue } from './fi/hg/core/csv';

console.log(stringifyCsvCellValue([1, 2, 3]));  // Outputs "1,2,3"
console.log(stringifyCsvCellValue(undefined));  // Outputs ""
console.log(stringifyCsvCellValue(123));        // Outputs "123"
console.log(stringifyCsvCellValue("abc"));      // Outputs "abc"

Generating rows from objects¶

The getCsvRowFromJsonObject function can be used to generate a single row of CSV data from an object.

It takes an object, an array of property names to include in the row, and an optional map of property transformers as parameters.

The property transformers are functions that take an object and a property name, and return the value to be included in the row for that property.

If no property transformer is provided for a given property, the value of the property is retrieved using the get function from the Lodash library and passed to the stringifyCsvCellValue function.

Here's an example of using the getCsvRowFromJsonObject function:

import { getCsvRowFromJsonObject } from './fi/hg/core/csv';

const obj = {
    id: 123,
    name: "John Smith",
    email: "john.smith@example.com"
};

const row = getCsvRowFromJsonObject(obj, ['id', 'name', 'email']);
console.log(row);  // Outputs ["123", "John Smith", "john.smith@example.com"]

You can also provide a property transformer map to customize the values included in the row for certain properties. For example:

import { getCsvRowFromJsonObject } from './fi/hg/core/csv';

const obj = {
    id: 123,
    name: "John Smith",
    email: "john.smith@example.com"
};

const propertyTransformer: CsvPropertyTransformerMap<typeof obj> = {
    name: (item: typeof obj) => `"${item.name}"`,
    email: (item: typeof obj) => item.email.toLowerCase()
};

const row = getCsvRowFromJsonObject(obj, ['id', 'name', 'email'], propertyTransformer);
console.log(row);  // Outputs ["123", "\"John Smith\"", "john.smith@example.com"]

In this example, the name property is wrapped in double quotes, and the email property is converted to lower case before being included in the row.

Generating an entire CSV file from a list of objects¶

The getCsvFromJsonObjectList function can be used to generate an entire CSV file from a list of objects.

It takes a list of objects, an optional array of property names to include in the rows, an optional flag to include a header row with the property names, and an optional map of property
transformers as parameters.

If no property names are provided, the keys of the first object in the list are used.

If the includeHeader flag is true, the first row of the CSV file will be a header row with the property names.

The property transformers work the same way as in the
getCsvRowFromJsonObject function, allowing you to customize the values
included in the rows for certain properties.

Here's an example of using the getCsvFromJsonObjectList function:

import { getCsvFromJsonObjectList } from './fi/hg/core/csv';

const list = [
    {
        id: 123,
        name: "John Smith",
        email: "john.smith@example.com"
    },
    {
        id: 456,
        name: "Jane Doe",
        email: "jane.doe@example.com"
    }
];

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email'], true);
console.log(csv);
// Outputs:
// [
//   ["id", "name", "email"],
//   ["123", "John Smith", "john.smith@example.com"],
//   ["456", "Jane Doe", "jane.doe@example.com"]
// ]

You can also omit the header row or use different property names by setting the includeHeader flag to false or providing a different array of property names, respectively.

You can also use a property transformer map to customize the values included in the rows for certain properties. For example:

import { getCsvFromJsonObjectList } from './fi/hg/core/csv';

const list = [
    {
        id: 123,
        name: "John Smith",
        email: "john.smith@example.com"
    },
    {
        id: 456,
        name: "Jane Doe",
        email: "jane.doe@example.com"
    }
];

const propertyTransformer: CsvPropertyTransformerMap<typeof list[0]> = {
    name: (item: typeof list[0]) => `"${item.name}"`,
    email: (item: typeof list[0]) => item.email.toLowerCase()
};

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email'], true, propertyTransformer);
console.log(csv);
// Outputs:
// [
//   ["id", "name", "email"],
//   ["123", "\"John Smith\"", "john.smith@example.com"],
//   ["456", "\"Jane Doe\"", "jane.doe@example.com"]
// ]

Customizing the output¶

All the CSV generation functions in the HG CSV module accept optional parameters to customize the output of the generated CSV data.

In this section, we'll discuss how to use the separator, quote, and propertyTransformer parameters to customize the output of the CSV generation functions.

Changing the separator and quote characters¶

By default, the CSV generation functions use a comma (,) as the separator between cells and a double quote (") as the quote character around cell values that contain the separator or newline characters. You can change these characters by passing different values for the separator and quote parameters.

For example, here's how you could use the pipe (|) as the separator and single quotes (') as the quote character:

import { parseCsv } from './fi/hg/core/csv';

const csvString = "'name'|'email'|'age'\n'John Smith'|'john.smith@example.com'|'30'\n'Jane Doe'|'jane.doe@example.com'|'25'";

// Use the pipe character (|) as the separator and single quotes (') as the quote character
const csv = parseCsv(csvString, '|', "'");

console.log(csv);

This code parses the following CSV data:

'name'|'email'|'age'
'John Smith'|'john.smith@example.com'|'30'
'Jane Doe'|'jane.doe@example.com'|'25'

And produces the following output:

[
  ['name', 'email', 'age'],
  ['John Smith', 'john.smith@example.com', '30'],
  ['Jane Doe', 'jane.doe@example.com', '25']
]

Alternatively, you can use the stringifyCsv function to generate CSV data using different separator and quote characters.

Here is an example of how you can use the separator and quote parameters with the stringifyCsv function:

import { stringifyCsv } from './csv';

const csv = [
  ['name', 'email', 'age'],
  ['John Smith', 'john.smith@example.com', '30'],
  ['Jane Doe', 'jane.doe@example.com', '25'],
];

// Use the pipe character (|) as the separator and single quotes (') as the quote character
const csvString = stringifyCsv(csv, '|', "'");

console.log(csvString);

This code generates the following CSV data:

'name'|'email'|'age'
'John Smith'|'john.smith@example.com'|'30'
'Jane Doe'|'jane.doe@example.com'|'25'

The stringifyCsv function generates a string representation of the CSV data, using the specified separator and quote characters to separate cells and quote cell values that contain the separator or newline characters.

You can also use the default separator and quote characters by omitting the separator and quote parameters:

import { stringifyCsv } from './csv';

const csv = [
  ['name', 'email', 'age'],
  ['John Smith', 'john.smith@example.com', '30'],
  ['Jane Doe', 'jane.doe@example.com', '25'],
];

// Use the default separator (,) and quote (") characters
const csvString = stringifyCsv(csv);

console.log(csvString);

This code generates the following CSV data:

"name","email","age"
"John Smith","john.smith@example.com","30"
"Jane Doe","jane.doe@example.com","25"

Using property transformers to customize cell values¶

The propertyTransformer parameter allows you to provide custom functions that transform the values of certain properties before they are included in the CSV rows. The propertyTransformer parameter should be an object that maps property names to transformer functions. The transformer functions should take an object and a property name as arguments, and return the
transformed value for the property.

Here's an example of using the propertyTransformer parameter to customize the values of the name and email properties:

import { getCsvFromJsonObjectList } from './fi/hg/core/csv';

const list = [
    {
        id: 123,
        name: "John Smith",
        email: "john.smith@example.com"
    },
    {
        id: 456,
        name: "Jane Doe",
        email: "jane.doe@example.com"
    }
];

const propertyTransformer: CsvPropertyTransformerMap<typeof list[0]> = {
    name: (item: typeof list[0]) => `"${item.name}"`,
    email: (item: typeof list[0]) => item.email.toLowerCase()
};

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email'], true, propertyTransformer);
console.log(csv);
// Outputs:
// [
//   ["id", "name", "email"],
//   ["123", "\"John Smith\"", "john.smith@example.com"],
//   ["456", "\"Jane Doe\"", "jane.doe@example.com"]
// ]

In this example, the name property values are surrounded by double quotes, and the email property values are converted to lower case.

Handling line breaks in cell values¶

Line breaks within cell values can cause issues when generating and parsing CSV data, as they can disrupt the expected structure of the CSV data.

To handle line breaks in cell values, the HG CSV module provides the replaceLineBreak option, which allows you to specify a character to replace line breaks in cell values with. By default, this constant is set to a single space ().

You can use the stringifyCsv function to replace line breaks in cell values with the character specified by the replaceLineBreak option when stringifying CSV data.

Working with quoted line breaks¶

Sometimes, CSV cell values may contain line breaks that are intended to be
included as part of the cell value, rather than being used to indicate the end of a row. In such cases, the cell value may be quoted to indicate that the line breaks within it are part of the cell value, rather than being used to split the rows.

TODO

Examples¶

Here are some examples of using the various functions in the HG CSV module to parse and generate CSV data:

Example 1: Generating CSV data from a list of objects¶

import { getCsvFromJsonObjectLis, CsvPropertyTransformerMapt } from './fi/hg/core/csv';

const list = [
    {
        id: 123,
        name: "John Smith",
        email: "john.smith@example.com",
        description: "Lorem ipsum dolor sit amet,\nconsectetur adipiscing elit."
    },
    {
        id: 456,
        name: "Jane Doe",
        email: "jane.doe@example.com",
        description: "Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris."
    }
];

const propertyTransformer: CsvPropertyTransformerMap<typeof list[0]> = {
    description: (item: typeof list[0]) => item.description.toLowerCase()
};

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email', 'description'], true, propertyTransformer);
console.log(csv);
// Outputs:
// [
//   ["123", "John Smith", "john.smith@example.com", "lorem ipsum dolor sit amet, consectetur adipiscing elit."],
//   ["456", "Jane Doe", "jane.doe@example.com", "ut enim ad minim veniam, quis nostrud exercitation ullamco laboris."]
// ]

In this example, we use the getCsvFromJsonObjectList function to generate CSV data from a list of objects. We specify the properties to include in the CSV rows using the properties parameter, and we set the includeHeader parameter to true to include a header row with the property names.

We also use the propertyTransformer parameter to provide a custom function that replaces uppercase characters in the description property with the lowercase character.

Example 2: Parsing CSV data¶

import { parseCsvRow } from './fi/hg/core/csv';

const csv = [
    'id,name,email,description',
    '123,John Smith,john.smith@example.com,"Lorem ipsum dolor sit amet,\nconsectetur adipiscing elit."',
    '456,Jane Doe,jane.doe@example.com,"Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris."'
];

for (const row of csv) {
    let cells = parseCsvRow(row, ',', '"');
    console.log(cells);
}
// Outputs:
// ["id", "name", "email", "description"]
// ["123", "John Smith", "john.smith@example.com", "Lorem ipsum dolor sit amet,\nconsectetur adipiscing elit."]
// ["456", "Jane Doe", "jane.doe@example.com", "Ut enim ad minim veniam,\nquis nostrud exercitation ullamco laboris."]

Example 3: Generating CSV data¶

import { getCsvFromJsonObjectList } from './fi/hg/core/csv';

const list = [
    { id: 123, name: 'John Smith', email: 'john.smith@example.com', description: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.' },
    { id: 456, name: 'Jane Doe', email: 'jane.doe@example.com', description: 'Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.' }
];

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email', 'description'], true, undefined);
console.log(csv);
// Outputs:
// [
//  ["id", "name", "email", "description"],
//  ["123", "John Smith", "john.smith@example.com", "Lorem ipsum dolor sit amet, consectetur adipiscing elit."],
//  ["456", "Jane Doe", "jane.doe@example.com", "Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris."]
// ]

In this example, we use the getCsvFromJsonObjectList function to generate CSV data from a list of objects. We specify the properties to include in the CSV rows using the properties parameter, and we set the includeHeader parameter to true to include a header row with the property names.

Example 4: Using property transformers to customize cell values¶

import { getCsvFromJsonObjectList } from './fi/hg/core/csv';

const list = [
    { id: 123, name: 'John Smith', email: 'john.smith@example.com', description: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.' },
    { id: 456, name: 'Jane Doe', email: 'jane.doe@example.com', description: 'Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.' }
];

const propertyTransformer : CsvPropertyTransformerMap = {
    email: (item: ReadonlyJsonObject, key: string): string => {
        return `${key}: ${item[key]}`;
    }
};

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email', 'description'], true, propertyTransformer);
console.log(csv);
// Outputs:
// [
//  ["id", "name", "email", "description"],
//  ["123", "John Smith", "email: john.smith@example.com", "Lorem ipsum dolor sit amet, consectetur adipiscing elit."],
//  ["456", "Jane Doe", "email: jane.doe@example.com", "Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris."]
// ]

In this example, we use the propertyTransformer parameter to provide custom functions that transform the values of certain properties before they are included in the CSV rows.

We define a propertyTransformer object with a email property, which is a function that takes an object and a property key as its parameters and returns a string with the property key and value.

We pass the propertyTransformer object to the getCsvFromJsonObjectList function as the propertyTransformer parameter. This causes the email property values in the CSV rows to be transformed by the custom function before being included in the CSV data.

As a result, the email cell values in the CSV rows are modified to include the property key, resulting in cell values like "email: john.smith@example.com" instead of just "john.smith@example.com".

Example 5: Handling line breaks in cell values¶

import { getCsvFromJsonObjectList, parseCsvRow } from './fi/hg/core/csv';

const list = [
    { id: 123, name: 'John Smith', email: 'john.smith@example.com', description: 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.\nSuspendisse in ipsum quis nisl ultricies rhoncus at sit amet dolor.' },
    { id: 456, name: 'Jane Doe', email: 'jane.doe@example.com', description: 'Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.\nConsectetur adipiscing elit.' }
];

const csv = getCsvFromJsonObjectList(list, ['id', 'name', 'email', 'description'], true);
console.log(csv);
// Outputs:
// [
//  ["id", "name", "email", "description"],
//  ["123", "John Smith", "john.smith@example.com", "Lorem ipsum dolor sit amet, consectetur adipiscing elit.\nSuspendisse in ipsum quis nisl ultricies rhoncus at sit amet dolor."],
//  ["456", "Jane Doe", "jane.doe@example.com", "Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.\nConsectetur adipiscing elit."]
// ]

const csvString = stringifyCsv(csv, undefined, undefined, "\n");
console.log(csvString);
// Outputs:
// id,name,email,description
// 123,John Smith,john.smith@example.com,"Lorem ipsum dolor sit amet, consectetur adipiscing elit.\nSuspendisse in ipsum quis nisl ultricies rhoncus at sit amet dolor."
// 456,Jane Doe,jane.doe@example.com,"Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.\nConsectetur adipiscing elit."

const parsedCsv = parseCsv(csvString);
console.log(parsedCsv);
// Outputs:
// [
//  ["id", "name", "email", "description"],
//  ["123", "John Smith", "john.smith@example.com", "Lorem ipsum dolor sit amet, consectetur adipiscing elit.\nSuspendisse in ipsum quis nisl ultricies rhoncus at sit amet dolor."],
//  ["456", "Jane Doe", "jane.doe@example.com", "Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.\nConsectetur adipiscing elit."]
// ]

console.log(parsedCsv[1][3]);
// Outputs: "Lorem ipsum dolor sit amet, consectetur adipiscing elit.\nSuspendisse in ipsum quis nisl ultricies rhoncus at sit amet dolor."