When importing JSON data as search terms in Amazon CloudSearch, some of the data generated the following error and couldn’t be imported.

Error Message

[*Deprecated*: Use the outer message field] Validation error for field 'title_ja': Invalid codepoint 1C

The problematic character was u001c, and by removing it, I was able to successfully load the JSON data.

This was a value saved to the database from form input, but I wonder how this character got in there…?

---

Reference Information

・AWS Developer Forums: Which codepoints are disallowed in text …

・Guide to Formatting Your Data in SDF for Amazon CloudSearch : Articles & Tutorials : Amazon Web Services

・Scalaで正規表現: ユニコードの空白文字29種類にマッチするか試してみた - Qiita

・これは備忘録　ASCII文字コード

That’s all from the Gemba.