Image description, OCR, and screen capture

Use Arka’s vision skills to describe photos, extract text via OCR, analyze blueprints, and capture your screen with Gemini, Ollama llava, or vLLM backends.

Arka supports vision tasks: photo description, blueprint analysis, and screen capture.

Describe images

Two-layer analysis: OCR extracts exact text; vision describes layout and colors.

arka describe photo.jpg
arka describe ~/Downloads/chart.png

Backends (auto-selected): Gemini, Ollama (llava), vLLM.

DESCRIBE_IMAGE_BACKEND=auto   # auto | gemini | ollama | vllm
GEMINI_API_KEY=...
# DESCRIBE_IMAGE_TWO_LAYER=1
# DESCRIBE_IMAGE_OCR=1

pip install "arka-agent[vision]"
brew install tesseract    # OCR layer

Drawings and blueprints

Analyze floor plans, elevations, MEP schematics, and scanned contracts with Gemini vision:

arka analyze plan.pdf "extract door schedule and room dimensions"
arka analyze --pages 1-3 specs.pdf "summarize payment terms"

pip install "arka-agent[drawings]"

# DRAWING_MODEL=gemini-2.5-flash
# DRAWING_MAX_PAGES=8

Screen capture

10-second countdown, full-display capture, vision describe:

arka describe screen
arka describe my screen
arka describe screen "what app is in focus"

Describe videos

describe_video samples frames with ffmpeg and sends those frames through the configured vision backend. Use it for gameplay recordings, UI animation checks, demos, and people-location questions.

arka describe_video demo.mp4
arka describe_video demo.mp4 "find visual glitches and confusing UI states"
arka "describe video gameplay.mp4"
arka "who is in meeting.mp4 and where are they"

For people-focused prompts, Arka asks the vision model to identify visible people and approximate their screen positions. For general describe/analyze prompts, it describes subjects, actions, setting, text, framing, and visual issues.

brew install ffmpeg
pip install "arka-agent[vision]"