· Development

CSV and RFC 4180: commas, quotes and line breaks inside fields

Why a comma or newline inside a value breaks CSV columns, per RFC 4180, the IANA text/csv registration, the W3C tabular data Recommendation and Python/Go docs, and what RFC 4180 leaves open.

DevelopmentData

CSV looks easy to produce by hand: text separated by commas. Then one value contains a comma or a line break, and columns shift or a row splits in two. The usual reference is RFC 4180, but what it does and doesn’t define is less widely known.

RFC 4180 is not an Internet standard

RFC 4180, published in October 2005, is an Informational RFC. Its status section says it “does not specify an Internet standard of any kind.” Section 2 notes that no formal specification exists and then documents “the format that seems to be followed by most implementations.” What the RFC formally does is register the text/csv media type with IANA.

In 2014, RFC 7111 updated RFC 4180 by defining URI fragment identifiers for rows, columns and cells in CSV. A revision intended to replace RFC 4180, draft-shafranovich-rfc4180-bis, is listed on the IETF Datatracker as expired after revision 07 (checked October 3, 2026).

Quoting: wrap the field and double the inner quote

The main rules in section 2 of RFC 4180 are:

  • Fields containing line breaks, double quotes or commas should be enclosed in double quotes.
  • Inside a quoted field, a double quote is escaped by writing it twice: "".
  • An unquoted field may not contain double quotes.
  • Spaces are part of a field and should not be ignored. The last field in a record must not be followed by a comma.
id,memo
1,"He said ""hi"", then left
line2"

This takes three lines on screen, but apart from the header it holds one record. The second field is quoted, so its comma and line break belong to the value.

Why you can’t split on newlines

RFC 4180 uses CRLF as the record separator and makes the line break after the final record optional. Its ABNF grammar allows CR and LF inside a quoted field. Splitting a file on line breaks therefore cuts a record at any newline inside quotes. A CSV reader has to track whether it is currently inside a quoted field.

Implementations also differ in how they handle line endings:

  • Python’s csv writer ends records with \r\n by default. Its reader recognizes both \r and \n as line endings regardless of the lineterminator setting. The documentation says to open files with newline=''; without it, newlines inside quoted fields may be misread, and an extra \r may be added on platforms that use \r\n.
  • Go’s encoding/csv supports the RFC 4180 format, except that its Writer uses LF rather than CRLF by default. Set UseCRLF to change that. Its Reader converts every \r\n to \n, including inside multiline fields.
  • The W3C’s 2015 Recommendation, “Model for Tabular Data and Metadata on the Web,” recommends CRLF at the end of a row but also permits LF. The document itself notes that this differs from text/csv as defined by RFC 4180.

Run with Python 3.14, the following code produces CRLF at the end of each record and a bare LF inside the field. The ABNF permits CR and LF within a quoted field, so both forms can occur in the same file.

import csv, io

buf = io.StringIO(newline="")
w = csv.writer(buf)
w.writerow(["id", "memo"])
w.writerow([1, 'He said "hi", then left\nline2'])
print(repr(buf.getvalue()))
# 'id,memo\r\n1,"He said ""hi"", then left\nline2"\r\n'

In the RFC 4180 ABNF, field content (TEXTDATA) is %x20-21 / %x23-2B / %x2D-7E: printable ASCII minus the comma and double quote. Accented letters, Korean and Japanese are all outside that grammar. The RFC says US-ASCII is common, but other character sets may be used with the charset parameter. The W3C Recommendation notes that the ASCII restriction is routinely ignored in practice and impractical on the international web.

The current IANA registration for text/csv, updated January 17, 2014, recommends using the charset parameter, and recommends assuming UTF-8 when it is absent. The W3C Recommendation also recommends UTF-8.

Programs don’t all treat a byte order mark at the start of a file the same way; that is covered in the UTF-8 BOM post. A CSV file encoded in EUC-KR or Shift_JIS can be converted to UTF-8 with the encoding converter.

Takeaways

  • RFC 4180 is Informational, not an Internet Standards Track specification. It supplies an ABNF grammar, but its lowercase “should” and “must” are ordinary prose, not RFC 2119 uppercase requirement keywords.
  • When writing CSV, quote fields that contain commas, double quotes or line breaks, and double every quote inside. A CSV library is safer than joining strings yourself.
  • When reading CSV, don’t split on newlines. The record ending may be CRLF or LF depending on the program that wrote the file.
  • The encoding and whether there is a header row may not be clear from the file alone. When serving CSV over HTTP, both can be declared in the media type: text/csv; charset=utf-8; header=present.

Specifications and documentation used

Links and content were checked on October 3, 2026. Draft status and language documentation can change.

Back to the blog