.. _file-attachments:

Attach one PNG or PDF per prepared request
==========================================

File attachments are an optional step for projects that already use the normal
text-based CSV workflow. ``init``, ``gui``, ``validate``, and ``prepare`` behave
as before. The attachment command makes no provider calls; ``submit`` is the
first paid step.

Arrange files in a directory such as ``data/attachments/`` and add a filename
column to the input CSV. Keep at least one ordinary text field mapped in
``input.fields_sent``. Do **not** map ``attachment_file`` in
``input.fields_sent``: it identifies a local file and is not part of the text
prompt.

.. code-block:: text

   my-project/
   ├── project.yaml
   └── data/
       ├── input-data.csv
       └── attachments/
           ├── grant-001.pdf
           └── grant-002.png

For example, the CSV could contain:

.. code-block:: text

   grant_id,project_title,abstract,attachment_file
   G001,Community research,Study summary,grant-001.pdf
   G002,Instrument design,Study summary,grant-002.png

Because the filename column is not sent, ``validate`` still treats rows whose
mapped text fields are identical as duplicates, even if their files differ.
Send a field that distinguishes each record, such as a title or caption.

Set ``task.rows_per_request: 1`` in ``project.yaml`` so each request has one
record and one file. Then validate and prepare normally. To test the actual
file-input workflow, attach files to a selected pilot *before* submitting it:

.. code-block:: console

   $ kllm-batch validate -c my-project/project.yaml
   $ kllm-batch prepare -c my-project/project.yaml --provider openai --sample-size 2
   $ kllm-batch attach-files PILOT_RUN_DIR --column attachment_file \
       --files-dir data/attachments --acknowledge-unestimated-cost
   $ kllm-batch submit PILOT_RUN_DIR

A relative ``--files-dir`` is resolved against the directory containing
``project.yaml``, not the current directory; use an absolute path if unsure.

For the full batch, run ``prepare`` without ``--sample-size``, then
``attach-files`` on that new run before ``submit``. The same command works with
Anthropic. Inspect ``REVIEW.md`` and ``api_requests/attached_segment_*.jsonl``
before submission. The original text-only request files remain intact; the
attached files are the ones executed.

.. warning::

   The cost estimate produced by ``prepare`` covers text, not image or PDF
   processing. ``attach-files`` therefore requires an explicit acknowledgment;
   the total cost is unknown until the provider returns usage. The configured
   estimated-cost budget does not cap the attached run's actual cost. Start
   with a small pilot and review provider pricing and file limits.

The command embeds file bytes in the prepared request artifacts. Treat the run
directory as sensitive if the source files are sensitive. The manifest records
the relative filenames and SHA-256 checksums, and submission rejects modified
prepared requests. Filenames must resolve inside ``--files-dir``; missing,
unsupported, or oversized files are rejected locally. Only PNG and PDF are
supported in this first version.

If an attached run has failures, ``kllm-batch retry RUN_DIR`` prepares a child
run but does not attach files automatically. Run ``attach-files`` on the child
with the same column and directory before submitting it. Submission refuses a
retry child until the required files are attached, and the files must match
their parent-run checksums. See :ref:`retries-and-reruns` for merging results.

Use the checked-in helper scripts
---------------------------------

The `job-post attachment example
<https://github.com/rs-kellogg/llm-batch_pipeline/tree/main/examples/job_post_attachments>`_
provides four small PDF/PNG inputs, a structured extraction schema, and helper
scripts for running the same smoke test with either provider. From the
repository root, prepare a new batch run with one provider:

.. code-block:: console

   $ examples/job_post_attachments/prepare.sh openai
   $ examples/job_post_attachments/prepare.sh anthropic

Run only one preparation command at a time and copy the complete run directory
it prints. Then attach, submit, and synchronize that run:

.. code-block:: console

   $ examples/job_post_attachments/attach.sh RUN_DIR
   $ examples/job_post_attachments/submit.sh RUN_DIR
   $ examples/job_post_attachments/sync.sh RUN_DIR

``prepare.sh`` validates the project and prepares a batch run. ``attach.sh``
uses the example's combined PDF/PNG directory and supplies the required
unestimated-file-cost acknowledgment. ``submit.sh`` retains the normal
interactive cost confirmation, and ``sync.sh`` watches the submitted batch
through processing and audit. The scripts resolve their project paths from
their own location, so they do not depend on the shell's current directory;
each paid step still requires the explicit ``RUN_DIR`` you reviewed.