Japanese CSV files that open garbled in Excel
If your app exports a CSV in UTF-8, someone in Japan who opens it in Excel can see 蜷榊燕 where the header says 名前 (name). Excel on Japanese Windows reads a CSV without a byte order mark (BOM) as Shift_JIS. Start the file with the BOM, U+FEFF:
// csv: your rows, joined with "\r\n"
const blob = new Blob(
["\uFEFF" + csv],
{ type: "text/csv;charset=utf-8" });
// On a Node.js server
res.setHeader("Content-Type",
"text/csv; charset=utf-8");
res.end("\uFEFF" + csv);
In Python, the utf-8-sig encoding writes the BOM for you:
import csv
with open(path, "w", newline="",
encoding="utf-8-sig") as f:
csv.writer(f).writerows(rows)
The BOM is three bytes at the start of the file, EF BB BF. charset=utf-8 in the type doesn't do this job: it isn't saved in the downloaded file.
We opened the same three rows in Excel for Microsoft 365 (version 16.0, build 19127) on Japanese Windows 11. As UTF-8 without a BOM, the header came out as 蜷榊燕, and the comma after 山田 太郎 vanished, so the name and the next column landed in one cell. With the BOM, and as Shift_JIS, every cell was right, ① and ㈱ included. The files written by the code above, the Blob in Node.js 24, the server's response and Python 3.12's csv module, all opened correctly too.
What went wrong in real apps
- OpenMetadata #25764: Chinese text came out garbled in its CSV import and export. #28134, closed as a duplicate, adds that Excel on Japanese Windows falls back to CP932 (Shift-JIS) for a UTF-8 file without a BOM. The fix, merged in October 2026, starts every CSV download with a BOM rather than adding an opt-in setting (#35735).
- Eventra #6810: the attendee export made its Blob with
type: "text/csv;charset=utf-8;"and no BOM, and non-ASCII names came out garbled. The fix put"\uFEFF"in front (#6946). - eitango-quest #114: the other direction. The word import read the uploaded file with
readAsText(file, "UTF-8"), so a CSV saved from Japanese Excel came in full of U+FFFD, and nothing warned the user.
When people upload what Excel saved
People in Japan edit the file in Excel and upload it again. In our run, Excel's “CSV (Comma delimited)” saved Shift_JIS, and “CSV UTF-8” saved UTF-8 with a BOM. Read both:
async function readCsvText(file) {
const buf = await file.arrayBuffer();
try {
return new TextDecoder("utf-8",
{ fatal: true }).decode(buf);
} catch {
return new TextDecoder("shift_jis")
.decode(buf);
}
}
With fatal: true, the UTF-8 decoder throws on bytes that aren't UTF-8 instead of turning them into U+FFFD, and the code falls back to Shift_JIS. TextDecoder also drops the BOM. In the Encoding Standard, shift_jis also answers to windows-31j and ms932, the Windows form that Excel writes (Encoding Standard). We ran readCsvText() on Excel's two saved files and our Shift_JIS file, in Node.js 24 and Chromium, and all three read back exactly.
Two things to watch for on a server:
- Node.js's
fs.readFileSync(path, "utf8")keeps the BOM. The first header came out as"\uFEFF名前", so a lookup by"名前"found nothing. Decode withTextDecoderas above. - In Python, read with
"utf-8-sig"and fall back to"cp932", as Random-Presenter-Selector #13 did when files saved from Excel failed. Usecp932, notshift_jis: Python'sshift_jiscodec failed on the ① that Excel wrote.
The rest of a Japanese check
A CSV export is often what a team in Japan opens first, in Excel. The same pass catches Enter while typing Japanese (the fix), yen charged 100 times over (the fix), kanji drawn in their Chinese shapes (the fix), headings that break in the middle of a word (the fix), postal codes rejected when typed in Japanese (the fix), dates saved one day early in Japan (the fix), address forms in the American order (the fix), phone numbers sent to +81 with the 0 still in (the fix), PDFs with the Japanese missing (the fix), search that finds nothing in Japanese (the fix), previews cut down to “...” in Japanese (the fix), lists sorted in the wrong order (the fix), email subjects that arrive garbled (the fix), Japanese titles that give an empty URL slug (the fix), file names garbled on upload and download (the fix), dates in the Japanese era that come out wrong (the fix), full-width spaces that get past your checks (the fix), public holidays your app gets wrong (the fix), and vertical text that comes out sideways or upside down (the fix).
Lesstep translates web apps into Japanese and delivers them as one pull request. In Complete (US$249), we check every screen in Japanese, and we open the CSV your app exports in Excel on Japanese Windows. The first step is a free sample.
Microsoft, Excel and Windows are trademarks of the Microsoft group of companies; Python is a registered trademark of the Python Software Foundation; Node.js is a trademark of the OpenJS Foundation; Chromium is a trademark of Google LLC. This page is not affiliated with or endorsed by any of them.
More: Enter sends the message while typing Japanese · Stripe charges 100 times too much in yen · Kanji in their Chinese shapes · Japanese headings that break in the middle of a word · Forms that reject numbers typed in Japanese · Dates that come out wrong for Japanese users · Address forms in the Japanese order · Japanese phone numbers in forms · Japanese text missing from your PDFs · Search that finds nothing in Japanese · Length limits that break in Japanese · Japanese lists sorted in the wrong order · Japanese email subjects that arrive garbled · Japanese titles that give an empty URL slug · Japanese file names garbled on upload and download · Japanese era dates that come out wrong · Full-width spaces that get past your checks · Japanese public holidays your app gets wrong · Japanese vertical text that comes out sideways or upside down · Japanese localization checklist · Japanese localization for your web app