RAG Chunker

RAG Chunker shows how a document will be split before you index it for retrieval-augmented generation (RAG) or embeddings. Choose fixed character or token windows, sentence, paragraph, recursive separator or Markdown-aware chunking, and see every chunk with its character range, exact token count, overlap with its neighbours and heading path. Statistics show the chunk count, total, average and median tokens, the smallest and largest chunk, and how much text overlap duplicates.

1. 1.Paste Markdown or plain text, import a .md or .txt file, or load an example.
2. 2.Choose a strategy and set the chunk size and overlap in characters or tokens.
3. 3.For token sizes, pick o200k_base or cl100k_base; counts are exact.
4. 4.Review the statistics and the chunk cards; highlighted text is shared with the neighbouring chunk.
5. 5.Under Advanced, edit recursive separators, set a minimum chunk size or keep the heading path.
6. 6.Copy or download the chunks as plain text, Markdown, JSON, JSONL or CSV.

Every chunk is an exact slice of your document, so its character range always points back to the original text. Token sizes and overlap are measured with the selected exact tokenizer, never estimated from characters; the Claude, Gemini and generic entries are estimates and are only offered for character sizes. A single block, word or token larger than the chunk size becomes its own chunk and is marked as over size. The tool does not do semantic or embedding-based chunking, ATX headings (lines starting with #) are recognised but underlined headings are not, and exports use CodingTool fields rather than a LangChain or LlamaIndex schema. Documents up to 5 MB are processed locally in a background worker.