DocsUseful Toolsread_file

read_file

One copyable tool that reads any file — text, images, PDF, PowerPoint, Word, audio, video — and hands the extracted content to the agent.

Download

Point it at a file path and get the content back — the agent decides what to do with it. Images even enter the model's vision, so it can see and describe them.

Quick Start

read_file is a copyable tool — it isn't imported from the package. Copy it into your project:

Terminalbash

$co copy read_file # → ./tools/read_file.py

Pass it to your agent. Add the image_result_formatter plugin so images reach the vision model:

main.py
python
from connectonion import Agent
from connectonion.useful_plugins import image_result_formatter
from tools.read_file import read_file

agent = Agent(
    "assistant",
    tools=[read_file],
    plugins=[image_result_formatter],   # required so images reach the vision model
)

agent.input("What's in report.pdf?")
agent.input("Describe diagram.png")     # the model actually sees the image

Supported Formats

One tool, dispatched by file extension:

Extension	Returns
.txt .md .csv .json .tex + code	the file's text, as-is (UTF-8, or GB18030 for Chinese)
.png .jpg .jpeg .gif .webp	a `data:image/…;base64,…` URL the vision model sees
.pdf	extracted text, one section per page
.pptx	the text of each slide
.docx	the document's text
.mp3 .wav .m4a … (audio)	a transcript (via `transcribe()`)
.mp4 .mov .mkv … (video)	a transcript of the audio track (ffmpeg → `transcribe()`)

Legacy binary .ppt / .doc, scanned (text-less) PDFs, and a missing ffmpeg return an actionable Error: … string. Video reads the audio only — a silent screen recording has nothing to transcribe.

Supersedes the built-in read_file

This tool shares the name read_file with the lightweight built-in text reader in FileTools (text + line numbers). Copy this one in when you need the heavier formats — and pass one or the other, not both.

How Images Reach the Vision Model

Reading an image is not the tool doing OCR or captioning. read_file only produces the image data; a separate plugin turns that into something the model can actually see:

code

text

read_file("diagram.png")
      │  returns  "data:image/png;base64,iVBORw0KGgo…"   (raw image bytes, base64)
      ▼
tool result  →  a normal text tool-message in the conversation
      ▼
image_result_formatter plugin        (runs on the after_tools event)
      │  1. detects the  data:image/…;base64,…  URL in the result
      │  2. uploads the bytes to oo-api → gets back a short /img URL
      │  3. replaces the tool message with a short placeholder
      │  4. inserts a user message: { "type": "image_url", "image_url": {"url": "…/img/…"} }
      ▼
next LLM call  →  the model SEES the image (vision), not base64 text

Only the ~70-byte /img URL enters the message history — never the base64 — so screenshots don't bloat the replayed context or session logs. The upload needs OPENONION_API_KEY (set up by co init); oo-api stores the bytes and materializes them per provider at call time (e.g. inlines them for Gemini, which can't fetch URLs).

Why a plugin, and not read_file itself? A tool's return value can only become a text tool-message. Putting an image into the conversation means mutating the message list, which is only safe on the after_tools event — so a plugin does it. This is the same path browser screenshots use.

Consequence: images only reach the model if the agent has the image_result_formatter plugin. The minimal template enables it by default; a bare Agent(...) does not — add plugins=[image_result_formatter].

Dependencies

Python packages ship with connectonion (no lazy imports, no "package missing" fallbacks — they're real dependencies):

•pypdf — PDF text
•python-pptx — PowerPoint slide text
•python-docx — Word document text
•charset-normalizer — text encoding detection

System tool: ffmpeg on PATH, for video only (not a pip package — brew install ffmpeg / apt install ffmpeg). Audio and video transcription use the Gemini-based transcribe(), which needs co auth (a managed key); audio needs no ffmpeg.