CSV++ (CSV Plus Plus): Extension to RFC 4180 for Hierarchical Data

1.1. Motivation

Traditional CSV files represent flat, tabular data. However, real-world data often contains:¶

• Repeated values (e.g., multiple phone numbers for one person)¶
• Structured components (e.g., addresses with street, city, state, zip)¶
• Nested hierarchies (e.g., addresses with multiple address lines)¶

CSV++ addresses these limitations by introducing:¶

While formats like JSON, XML, and YAML excel at representing hierarchical data, they introduce complexity and redundancy that may not be warranted for moderately structured datasets. CSV++ occupies a middle ground by extending CSV's tabular simplicity with hierarchical capabilities, making it particularly suitable for:¶

• Data interchange where CSV is already established but structure is needed¶
• Spreadsheet applications where users expect tabular layouts¶
• Systems with existing CSV infrastructure that need enhanced capabilities¶
• Scenarios where human readability and editability in text editors is valued¶
• Applications requiring backward compatibility with legacy CSV parsers¶

CSV++ maintains CSV's core strengths - simple tooling, wide compatibility, and human-readable plain text - while addressing its limitations with hierarchical data through declarative header syntax.¶

1.2. When to Use CSV++

CSV++ is most appropriate for:¶

• Moderately structured data (1-3 levels of nesting)¶
• Environments where CSV is already the established interchange format¶
• Scenarios requiring backward compatibility with existing CSV infrastructure¶
• Applications that benefit from self-documenting tabular data with inline structure definitions¶
• Data that needs to be both machine-parseable and human-readable in plain text¶
• Large datasets where file size and bandwidth matter, as CSV's columnar format avoids repeating field names in every record (unlike JSON or XML)¶

For deeply nested hierarchical data (4+ levels), document-oriented formats like JSON or XML may provide better readability and tooling support. CSV++ aims to extend CSV's capabilities for moderately structured data while preserving its tabular nature, not to replace hierarchical data formats.¶

1.3. Design Principles

1. Backward Compatibility: Standard CSV parsers can read CSV++ files (though they won't interpret the enhanced structure)¶
2. Self-Documenting: Structure is defined in column headers¶
3. Tabular Readability: Data maintains a tabular layout suitable for spreadsheet viewing and editing, though deeply nested structures (3+ levels) may be more readable in hierarchical formats like JSON¶
4. Explicit Over Implicit: Delimiters are declared, not assumed¶
5. Recursively Composable: Structures can nest to any depth, though practical implementations SHOULD limit nesting to 3-4 levels for readability¶

1.4. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

4.1. Syntax

A field containing repeated values is declared in the header using square brackets:¶

column_name[delimiter]
column_name[]

Where:¶

• column_name - The name of the field¶
• [delimiter] - Optional: The character used to separate repeated values¶
• [] - Empty brackets use the default array delimiter¶

Delimiter Resolution:¶

1. If delimiter is specified: phone[|] uses |¶
2. If empty brackets: phone[] uses the tilde (~) as default delimiter¶

The tilde (~) is recommended as the default array delimiter to avoid conflicts with common data characters and the field separator.¶

4.2. Examples

id,name,phone[|],email[;]
1,John,555-1234|555-5678|555-9012,john@work.com;john@home.com
2,Jane,555-4444,jane@company.com

id,name,phone[],email[]
1,John,555-1234~555-5678~555-9012,john@work.com~john@home.com
2,Jane,555-4444,jane@company.com

4.3. Empty Values

Empty values in repetitions are represented by consecutive delimiters:¶

id,tags[|]
1,urgent||priority

This represents three tags: "urgent", "" (empty), "priority"¶

4.4. Escaping

If a repetition delimiter appears within the data itself, that individual value MUST be quoted per [RFC4180]. The repetition delimiter outside quotes still functions as a separator:¶

id,notes[|]
1,First note|Second note|"Third note contains | pipe character"
2,"Note with | pipe"|Another note|Final note

In the first row, there are three notes. The third note contains a literal pipe character. In the second row, the first note contains a literal pipe character.¶

5.1. Syntax

A field containing structured components is declared using parentheses:¶

column_name[repetition_delim]component_delim(
    comp1 component_delim comp2 ...)
column_name[]component_delim(comp1 component_delim comp2 ...)
column_name[](comp1 component_delim comp2 ...)
column_name(comp1 component_delim comp2 ...)

Component Delimiter Resolution:¶

1. If specified before (: address^(...) uses ^¶
2. If omitted: address(...) uses the caret (^) as default delimiter¶

The caret (^) is recommended as the default component delimiter to avoid conflicts with common data characters.¶

5.2. Examples

id,name,geo^(lat^lon)
1,Location A,34.0522^-118.2437
2,Location B,40.7128^-74.0060

id,name,address[~]^(street^city^state^zip)
1,John,123 Main St^Los Angeles^CA^90210~456 Oak Ave^New York^NY^10001
2,Jane,789 Pine St^Boston^MA^02101

6.1. Recursive Composition

Structures can nest arbitrarily deep. Component names can themselves be arrays or structures. Within component names in (...), array and structure syntax applies recursively.¶

6.2. Examples

id,name,address[~]^(type^lines[;]^city^state^zip)
1,John,home^123 Main;Apt 4^LA^CA^90210~work^456 Oak^NY^NY^10001

id,location^(name^coords:(lat:lon))
1,Office^34.05:-118.24
2,Home^40.71:-74.00

6.3. Delimiter Selection Guidelines

To maintain readability and parseability:¶

1. REQUIRED: Use different delimiters at each nesting level. Nested structures MUST use different component delimiters than their parent¶
2. Use visually distinct delimiters at each level¶
3. Recommended progression: ~ -> ^ -> ; -> :¶
4. Avoid using the field separator as a component delimiter¶
5. Document delimiter choices for complex schemas¶
6. Recommendation: Limit nesting to 3-4 levels maximum¶

8.1. Validation

Implementations SHOULD validate:¶

• Matching number of components across repeated structures¶
• Proper bracket nesting in headers¶
• Delimiter conflicts (same delimiter at multiple levels)¶
• MUST reject: Nested structures using the same component delimiter as their parent¶
• Reasonable nesting depth (recommend warning beyond 3-4 levels)¶

8.2. Limits

Implementations MAY impose reasonable limits on:¶

• Nesting depth (recommended minimum: 10 levels)¶
• Number of components per structure (recommended minimum: 100)¶
• Number of repetitions per array (recommended minimum: 1000)¶

9.1. MIME Type

CSV++ files use the text/csv media type defined in [RFC4180].¶

9.2. File Extensions

• .csv - Standard extension (recommended for compatibility)¶
• .csvpp - MAY be used to explicitly indicate CSV++ format¶
• .csvplus - Alternative explicit extension¶

10.1. Injection and Interpretation Risks

Malicious data may attempt to inject delimiters or structural markers to influence parsing behavior. Implementations MUST respect [RFC4180] quoting rules. Delimiters and structural markers appearing within quoted fields MUST be treated as literal values.¶

The default delimiters defined by this specification are intentionally chosen to be neutral and to avoid characters commonly associated with executable or control semantics. In addition, the explicit declaration of any non-default delimiters in the header allows parsers to establish expectations up front, reducing the likelihood of delimiter injection or ambiguous interpretation.¶

As with traditional CSV, some spreadsheet applications interpret certain values (e.g. those beginning with "=", "+", "-", or "@") as formulas. This specification does not attempt to redefine or mitigate spreadsheet formula evaluation; producers and consumers SHOULD continue to apply established best practices when targeting such environments.¶

10.2. Complexity and Resource Exhaustion

Deeply nested, malformed, or highly repetitive structures may lead to excessive CPU or memory consumption during parsing.¶

Implementations SHOULD:¶

• Enforce configurable limits on nesting depth and repetition¶
• Enforce reasonable limits on field sizes and record length¶
• Fail fast on structurally invalid input¶
• Prefer streaming or incremental parsing for large files¶
• Validate headers and structural definitions before processing data rows¶

10.3. Mixed-Tool Interoperability

CSV++ files may transit through tools unaware of the extended semantics, potentially resulting in loss of structure or unintended reinterpretation. Implementations used in security-sensitive pipelines SHOULD explicitly validate inputs and avoid implicit trust when moving between CSV-aware and CSV++-aware tools.¶

10.4. Encoding Issues

Files SHOULD use UTF-8 encoding. Implementations SHOULD detect and handle encoding errors. A BOM (Byte Order Mark) MAY be present.¶

10.5. IANA Considerations

This document has no IANA actions.¶

CSV++ files use the text/csv media type as defined in [RFC4180]. The format is fully backward compatible with standard CSV parsers; implementations unaware of the extensions defined in this document will process CSV++ files as conventional CSV, ignoring extended semantics.¶