FileConcat

Filter precedence

FileConcat decides which files end up in your output through a four-layer pipeline. Each layer runs in order and the next one only sees what survived the previous. Knowing the order makes it easier to predict why a file appears or disappears when you type a pattern.

The four layers

  1. Drop-time prune runs at ingestion.
  2. Validation cache runs once per file at ingestion.
  3. Pattern layer runs live every time you edit a textarea.
  4. Manual overrides are sticky and applied last.

The pipeline is deterministic. Same inputs produce the same included decision for every file.

Layer 1: drop-time prune

Two kinds of path are left out before a byte of them is read:

  • A directory the default ignore list names by itself: .git, node_modules, vendor, dist, build, target, __pycache__, .venv, coverage, .next and the rest of the plain names in that list. The walker that traverses a dropped folder never enters one, and the folder picker leaves their files out of its list.
  • A file whose extension never holds text: fonts, audio and video, compiled code and libraries (.o, .dll, .class, .pyc, .jar), databases, and data or model files. Images are not on this list: a dropped image gets the recognition offer instead. Neither is an extension a text format also uses (.dat, .mat, .nc, .obj, .orc, .pdb, .pkl, .pt, .res): those files are read, and their bytes decide whether they are text.

Until 2026-09-15 only .git and node_modules were pruned here, and everything else was read into memory first and hidden by the pattern layer. A file that never reaches the bundle still cost a read and a classification, and the largest drops were mostly such files.

The folder you drop is exempt. Drop dist by itself and it is read, whatever it is called, and the result's ledger says it was read because you chose it. Only what sits inside a drop is judged by name. The ledger names what this layer turned away (the folders, and the files by extension). A drop turned away whole says so on the empty screen.

An archive in the drop is opened, and its contents go through the same check with the archive as their root. build.zip is opened whatever it is called, and the fonts/ and node_modules/ inside it are turned away like the ones beside it. Until 2026-09-15 an archive's contents skipped this layer, so a zipped build tree was read in full while the unzipped one next to it was not.

What this layer leaves out cannot be turned back on from the drawer, because it never entered memory. Every glob and every plain file name in the default list (*.log, *.test.ts, package-lock.json) runs at filter time instead, and those can be. The test is prunedAtWalk in packages/core/src/path-utils/skip-paths.ts.

Layer 2: validation cache

When a file is ingested, FileConcat checks its content and records the result in a sticky validations map:

  • Is the content binary? A file with no text to recover (an executable, a font, an archive FileConcat cannot open) is left out for good and shows locked in the tree. Images and recordings are offered for text recognition or transcription, which runs only when you ask. Extractable documents (PDF, Word, Excel, PowerPoint and the rest of the formats list) are the exception. Their text is pulled out and included as text, in your browser, instead of the file being excluded. A video's own subtitle track is read the same way. Jupyter notebooks, .srt / .vtt caption tracks, saved .eml messages and web pages saved from a browser take the same path for the opposite reason. They are already text, but the text is a serialization, so what lands in the bundle is the document rather than the file. Scanned PDF pages are read by text recognition in your browser. It starts on its own for up to three such documents and 8 MB per drop, and is offered past that. A document that still yields no text (an encrypted PDF, say) is surfaced as excluded rather than silently left out.

There is no size check. A large file is read like any other, and the result screen reports how much of the context window the bundle fills.

This check runs once per file. Pattern edits do not retrigger it, which is why pattern changes feel instantaneous even on large drops.

Layer 3: pattern layer

Two textareas in the Patterns section of the Files pane drive this layer:

  • Include is a comma-separated list of glob patterns. Empty means every surviving file passes.
  • Ignore is a comma-separated list of glob patterns. Files matching any of them are excluded.

There is one rule that surprises people: if the Include textarea has any non-empty content, the Ignore textarea is skipped entirely. The include list is treated as an exclusive whitelist. To go back to "every file except these", you have to clear the Include textarea.

Pattern matching uses the pathMatches helper in packages/core/src/path-utils/glob-match.ts. Two semantics matter:

  • Bare directory or filename patterns match anywhere in the path. Typing node_modules matches apps/web/node_modules/react/index.js, not only a top-level node_modules. The same goes for *.log, matching every .log file at any depth.
  • Slashed patterns match the full path or any path suffix obtained by stripping leading directories. Typing src/**/*.ts matches src/app.tsx and also myrepo/src/app.tsx. This lets patterns work when you drop a folder whose name you do not want to type.

See File filtering for the pattern syntax in more detail.

Layer 4: manual overrides

Every checkbox in the pane's file tree writes to a sticky map called userToggled. An entry for a path can be "include" or "exclude", and either value wins over whatever the pattern layer decided.

The map only stores divergences. If you tick a checkbox so that the file's state matches the pattern decision anyway, the entry is removed instead of recorded. So the override map stays small and tracks only the deliberate exceptions.

The pane's bulk actions (Take out N, Keep only these N and the By type chips) write here too, one entry per file. Pattern edits never touch this map, so an include rewrite cannot accidentally lose your individual decisions. Reset to defaults clears it on purpose, with an Undo. Starting over clears the bundle and the overrides together.

Worked example

You drop a Node project into FileConcat:

  1. Drop-time prune removes .git/, node_modules/, dist/ and the fonts under public/. Everything else lands in memory.
  2. Validation marks any binaries (PNGs) as excluded.
  3. You type examples into the Ignore textarea. The file tree updates immediately. Every file under examples/ disappears.
  4. You manually re-check one file at examples/quickstart.ts because you want it in the context. userToggled["examples/quickstart.ts"] = "include".
  5. You edit the Ignore textarea to add docs. Pattern layer hides docs/* but the override for examples/quickstart.ts is untouched.

Every other interaction in FileConcat is a variation on this loop.