Pi Agent internal architecture

Why read this appendix?

To be clear, you will not need this appendix 99% of the time. Follow Chapter 2 to install Pi Agent and Chapter 3 to open it from the sidebar. If everything works, you no more need to understand the services behind it than you need to understand a compressor to use an air conditioner.

For the other 1%, the architecture becomes useful:

• The Logs tab shows an unfamiliar error. Messages such as “nginx: [emerg] host not found in upstream,” “s6-rc: fatal: unable to start service pi-web,” or “video-tools-init exited with code 1” may not have Pi Agent-specific search results. This appendix explains what each service does and what its failure means.
• You want to debug the problem yourself. Before entering the add-on container over SSH, you need to know its directories, processes, and logs.
• You want to customize or contribute. Anyone forking Pi Agent, contributing upstream, or building a companion add-on needs to understand the complete pipeline.
• You are simply curious. If you are the kind of person who opens a new device to see how it works, read on.

Architecture overview: the apartment-building analogy

A building provides a useful mental model for the layers of the system:

Keep two design principles in mind:

1. Ingress is the only entry point

   Your browser cannot connect directly to Pi Agent's internal services. Every request passes through HA Ingress, which confirms that you are signed in as an HA administrator before forwarding the request to Pi Agent. This is why Chapter 3 emphasizes the automatically created sidebar button: that button opens the Ingress route.

2. The container has several layers

   The Pi Agent add-on container does not run just one process. nginx receives and forwards traffic, pi-web is the main application, and video-tools-init is a one-time initialization worker. s6-overlay coordinates them in the background.

The s6-overlay boot sequence

When HA Supervisor starts Pi Agent, events inside the container follow a defined sequence. s6-overlay (often shortened to s6) is the process supervisor that starts and manages the container's services.

There is an important source-level detail behind that numbered overview: the video-tools-init oneshot is not listed in nginx or pi-web's dependencies.d. s6-overlay therefore starts all three services in parallel. While video-tools-init downloads 720 MB of dependencies, nginx and pi-web can already accept requests. During the initial 3–8 minutes, you can use chat even though the video pipeline is not yet ready. A comment at the start of rootfs/etc/s6-overlay/scripts/video-tools-init calls this “Parallel with pi-web.” The deliberate trade-off is an immediately available chat UI rather than making the whole interface wait for the video tools.

Supervisor also sends the Watchdog probe described below. If pi-web becomes stuck or too slow to respond, Supervisor restarts the entire container. s6-overlay then performs the same initialization again. Stage 2 normally exits in under a second, but only when both the sentinel and executable ${VENV}/bin/python3 are present. This double check prevents an interrupted first installation from being mistaken for a complete one.

nginx: the translator for HA Ingress

Why put nginx in front of a web application? In this container, nginx is both subtle and essential: it translates between HA Ingress and pi-web.

HA gives each add-on a URL like this:

https://homeassistant.local:8123/hassio/ingress/woow_ha_pi_agent/

Internally, pi-web understands only the root path (/); it does not know that HA has placed it under /hassio/ingress/woow_ha_pi_agent/. Without translation, a pi-web link such as /settings would send the browser to https://homeassistant.local:8123/settings, outside HA's route, and return a 404.

nginx performs that translation in several stages:

1. Receive the request from HA Ingress

   HA sends the request to port 30142 in the Pi Agent container (ingress_port: 30142). Because nginx listens on that port, it receives the request first.

2. Read the Ingress path from the request header

   HA supplies an HTTP header named X-Ingress-Path, with a value such as /api/hassio_ingress/A1b2C3d4e5F6g7H8. Its random identifier is 16–128 characters long and changes whenever HA restarts.

3. Forward the request to pi-web

   nginx proxies the request to http://127.0.0.1:30141, pi-web's internal port, and receives pi-web's HTML response.

4. Validate the header against an allowlist

   To prevent a forged X-Ingress-Path from injecting arbitrary content, the map $http_x_ingress_path $safe_ingress_path block validates it against ^/api/hassio_ingress/[A-Za-z0-9_-]{16,128}$. This expression comes directly from nginx.conf; it is not a guess. A nonmatching value becomes an empty string, making subsequent body rewrites no-ops and preventing XSS through a forged header. This defense was added in v0.7.0 for security-audit finding F-01.

5. Rewrite absolute paths with sub_filter

   nginx uses sub_filter to rewrite outgoing HTML, CSS, and JavaScript byte by byte. It prefixes the Ingress path to href="/_next/, src="/_next/, href="/manifest, href="/icons/, CSS references such as url(/_next/, and the JSON-escaped \"/_next/ references in RSC flight payloads.

6. Patch runtime JavaScript that sub_filter cannot reach

   This is the most intricate part of the configuration. The pi-web front-end bundle makes 40+ absolute-path /api/* calls through fetch, EventSource, and XMLHttpRequest. It also uses Next.js App Router RSC prefetches (?_rsc=<token>), history.pushState/replaceState, and links inserted at runtime by ReactDOM.preinit()/next/font as <link> elements. Static sub_filter rewriting cannot intercept these operations. nginx therefore injects JavaScript before </head> that monkey-patches window.fetch, EventSource, XMLHttpRequest.prototype.open, history.pushState/replaceState, Element.prototype.setAttribute, and the HTMLLinkElement/HTMLScriptElement/HTMLImageElement href/src setters, adding window.__INGRESS_PATH__ where needed. It also stubs navigator.serviceWorker.register with an empty registration because a PWA service worker is not useful under HA Ingress and would generate console errors. The history in CHANGELOG.md v0.5.0–v0.10.4 shows how difficult this compatibility layer has been.

Together, these layers make it possible for pi-web to remain completely unaware of HA Ingress. pi-web behaves like an ordinary Next.js application; nginx combines header rewriting, sub_filter path rewriting, and a client-side shim to fit it into HA's routing system. This design is called an Ingress shim: an adapter between incompatible interfaces.

/data/pi-agent/ directory map

Chapter 20 provides a short summary. The full map below shows what you may find after entering the add-on container:

Remember two principles:

• Irreplaceable data lives under /data/pi-agent/. Sessions, models.json, Skills, and the rclone configuration form the core of an HA backup. Because key and token files contain plaintext secrets, protect exported backups accordingly.
• Large, reproducible data such as venv and playwright-cache is deliberately excluded. Including the virtual environment would make each backup at least 500 MB larger. Instead, video-tools-init downloads these resources again after restoration.

How the Watchdog probe checks Pi Agent

As a long-running service, Pi Agent can become unresponsive because of a memory leak, infinite loop, or stalled external API. HA Supervisor's Watchdog detects services that still have a running process but no longer function correctly.

Watchdog is a double-edged sword. Automatically recovering a stuck service is useful, but a healthy add-on that merely responds slowly—for example, while processing a very long conversation history—can be mistaken for a hang and restarted. In that situation, increase the timeout in config.yaml or temporarily disable Watchdog with the switch on the HA page.

How Skill discovery works

Chapter 14 describes a Skill as a work manual for the AI. This section explains how the AI may discover those manuals—and why a Skill installed with pi install may not appear until you start a new Session.

The likely process occurs whenever you select “New Conversation”:

1. Create the Session

   pi-web creates a new Session file under sessions/.../*.jsonl.

2. Scan the Skills directory

   pi-web scans /data/pi-agent/skills/*/SKILL.md and reads the SKILL.md file in each subdirectory.

3. Parse the YAML frontmatter

   Each SKILL.md begins with YAML frontmatter like this:

   ---
   name: fridge_inventory
   description: Help the user inventory refrigerator items and identify food that is about to expire
   ---
   
   # Fridge inventory skill
   
   ...Details...

   pi-web reads only name and description at this point. It does not place the entire SKILL.md body into the AI's context; it initially supplies only the summary.

4. Build an <available_skills> block

   The names and descriptions are combined into a list wrapped in XML tags:

   <available_skills>
   <skill name="fridge_inventory">
   Help the user inventory refrigerator items and identify food that is about to expire
   </skill>
   <skill name="pitch_video">
   Script-to-YouTube Uploadable Video Pipeline
   </skill>
   </available_skills>

5. Add the block to the system prompt

   The <available_skills> block is placed near the start of the AI's system prompt, giving it a menu of capabilities available on this machine.

6. Let the AI select a Skill when relevant

   For each request, the AI compares the task with that list. When it chooses a Skill, it uses a tool call to read the complete SKILL.md and follow the detailed instructions.

This explains why installing a new Skill requires a new Session: an existing Session still has the old system prompt. It also explains why the description in SKILL.md is so important. A vague description makes the Skill hard for the AI to select; a clear one makes it useful at the right time.

The pi-web technology stack

pi-web, a Node.js application, provides the entire browser interface: the message composer, conversation list, model selector, and Skills panel. Its main components are:

Several technical details are worth noting:

• The pi-coding-agent SDK runs in-process. If pi-web crashes, the agent crashes with it. Agent tool calls, such as file writes, use pi-web's permissions. Omitting a separate daemon avoids another layer of IPC.
• There is no database. Conversations use JSONL files, settings use JSON files, and all state lives in the filesystem. This intentional simplification also makes backups straightforward.
• Chat streams over SSE (Server-Sent Events). SSE delivers each part of an AI response as it is generated. This is why the nginx configuration must disable proxy_buffering; otherwise nginx would hold the response until generation finished and then deliver it all at once.

Why v0.13.0 moved API keys into the pi-web UI

v0.13.0 removed the API-key fields from the add-on Configuration page. If you upgrade from an older release, those fields disappear; Chapter 21 covers the procedure. This section explains the architectural reason.

Old architecture (through v0.12.x):

1. Enter keys on the add-on Configuration page

   The form contained fields such as anthropic_api_key, openai_api_key, and glm_api_key.

2. Supervisor writes /data/options.json

   When you saved, Supervisor serialized the form as JSON inside the container at /data/options.json.

3. The startup script exports environment variables

   The s6-overlay startup script read /data/options.json, converted each field into an environment variable such as ANTHROPIC_API_KEY=..., and then executed pi-web.

4. pi-web reads the environment

   At startup, pi-web obtained the key from values such as process.env.ANTHROPIC_API_KEY.

Problems with the old architecture:

• Adding a provider required a new config.yaml schema and a new release; providers could not be added dynamically.
• One provider could not have two keys, such as separate work and personal credentials, because each field held one value.
• There was no Test button. A bad key was not discovered until the first real request returned 401.
• Changing a key required restarting the add-on before the new environment variable took effect.

New architecture (v0.13.0 and later):

1. Open pi-web's Models panel

   Provider configuration now lives in the browser interface, not on the add-on Configuration page.

2. Select Add Provider, complete the form, and select Test

   Follow the procedure in Chapter 6. pi-web sends a real API request to validate the key and saves it only after a successful test.

3. pi-web writes /data/pi-agent/models.json

   This structured JSON supports many providers, multiple keys per provider, and multiple models per key.

4. The setting takes effect without an add-on restart

   The next Session uses the new configuration because pi-web rereads models.json whenever it opens a Session.

Benefits of the new architecture:

• Providers, keys, and models can all be managed dynamically.
• The Test action validates credentials in advance.
• No add-on restart is required.
• The UI can enable only selected models for a provider.

Migration path—hard cut, no auto-migration. v0.13.0 does not seed old keys into models.json. Both CHANGELOG.md and DOCS.md explicitly say “No auto-import”: there is no automatic migration. After upgrading, re-enter every provider key in the pi-web Models panel and select Test. Supervisor discards the old fields in options.json under the new schema. Existing models.json placeholders such as $GLM_API_KEY resolve to empty strings, so requests return 401 until you supply the keys again. This is the expected transitional state, not an automatic migration in progress.

Advanced debugging inside the container

The add-on Logs tab is enough for routine debugging. For deeper investigation, you may need to enter the container itself. These are the most useful techniques:

1. Enable debug logging

   On the add-on Configuration page, change log_level from info to debug. Appendix A describes the levels. Saving restarts the add-on, after which the Logs tab becomes much more verbose. Return the setting to info when you finish.

2. Discover and enter the add-on container

   Install SSH & Web Terminal or use Advanced SSH. First list the running containers and capture the exact Pi Agent name reported by your host. Repository hashes and slugs vary, so never hard-code a Docker or Podman container name:

   CONTAINER_NAME="$(docker ps --format '{{.Names}}' | grep pi_agent | head -n 1)"
   printf '%s\n' "$CONTAINER_NAME"
   docker exec -it "$CONTAINER_NAME" bash

   Confirm that the printed value is the intended Pi Agent container before executing the final command. Once inside, use ls /data/pi-agent, cat /etc/nginx/nginx.conf, and ps aux to inspect its state. Treat the container filesystem as read-only unless you deliberately need to change persistent content under /data. Changes elsewhere disappear on restart.

3. Inspect the nginx access log

   Inside the container:

   tail -f /var/log/nginx/access.log

   This shows each request's URL, status code, and response time. It is the quickest way to investigate whether HA is sending the expected X-Ingress-Path header.

4. Test the edge-tts connection to Microsoft

   Inside the container:

   /data/pi-agent/venv/bin/python -m edge_tts --list-voices | head

   If this lists voices, the network connection works and the virtual environment is intact. If it does not, treat the video-tool installation as incomplete and use the reset procedure in Chapter 18.

5. Test rclone access to Google Drive

   Inside the container:

   rclone --config /data/pi-agent/rclone/rclone.conf ls gdrive: | head

   If this lists Google Drive content, the authorization still works. An error such as token expired means you need to run rclone config reconnect, as described at the end of Chapter 19.

Architecture-level troubleshooting

This section does not repeat the user-facing symptoms in Chapter 22. It covers cases that require an understanding of the architecture:

1. You cannot find /data/pi-agent/ on the HA host

   Symptom: you want to use File Editor or SSH to vi models.json on the HA host, but cannot find the path.
   Cause: an add-on's internal /data is not mounted as host /data. The add-on source specifies only a per-add-on mount and does not confirm a host path. Candidates include /mnt/data/supervisor/addon_configs/<slug>/ on HA OS and /usr/share/hassio/addon_configs/<slug>/ on Supervised installations, but Supervisor controls this location and it can change.
   Solution: if you must work from the host, use ls to check both candidate directories. A safer approach is to discover the container name first. Do not copy docker exec -it addon_<hash>_woow_ha_pi_agent bash literally; use the discovery procedure above, verify the printed name, enter that container, and then vi /data/pi-agent/models.json. Its internal /data is the add-on's mounted storage. An add-on restart is not normally required because pi-web rereads the file, although restarting after a consequential manual edit is safer.

2. Ingress fails: nginx repeatedly returns 403

   Symptom: selecting Pi Agent in the sidebar produces a blank page or nginx's 403 page.
   Cause: nginx's allowlist rejected the X-Ingress-Path header. An HA update may have changed the format—for example, by adding a character not covered by _—or HA may not be sending the header.
   Solution: inspect tail -f /var/log/nginx/access.log inside the container. If the observed format has clearly changed, open an issue in Woow_ha_pi_agent_add_on and include the relevant details. As a temporary measure, restart HA so Ingress generates a new identifier.

3. pi-web hangs: Watchdog keeps restarting the add-on

   Symptom: the Logs tab shows s6-rc-init started every few minutes, indicating repeated Supervisor restarts.
   Cause: pi-web's /api/home endpoint did not return 200 within 60 seconds. CPU exhaustion, insufficient memory, or a synchronous operation blocking Node's event loop may be responsible.
   Solution: (a) check whether the HA host's CPU or RAM is saturated—Pi 3 hardware and older mini PCs are especially vulnerable; (b) temporarily disable Watchdog on the HA add-on page to observe whether pi-web recovers; (c) enable log_level: debug and capture the final activity before each restart; and (d) if the cause remains unclear, open a GitHub issue with the logs.

4. Video tools are incomplete even though the sentinel exists

   Symptom: the video pipeline fails because edge-tts or Playwright cannot be found.
   Cause: an earlier video-tools initialization was interrupted, leaving state that needs to be reset.
   Solution: on the add-on Configuration page, set reset_video_tools: true and save. The reset removes the sentinel so initialization can run again. This is a one-shot trigger: the add-on automatically returns the option to false after handling it, preventing a new 720 MB download at every start. See Chapter 18 for details.

5. A Skill is installed, but the AI cannot see it

   Symptom: pi install reports success, but the AI never selects the Skill.
   Cause: the Skill discovery process runs when a new Session starts. The existing Session's system prompt does not include the new Skill.
   Solution: select “New Conversation.” If the Skill still does not appear, run ls /data/pi-agent/skills/ to confirm that its directory exists, then cat SKILL.md and verify that the frontmatter contains both name and description. A YAML error, including incorrect indentation, can cause the entire Skill to be skipped.

Frequently asked questions