Frigate Installation Project Plan#

codex resume 01a010cc-f385-7fb1-bda6-375f3a85e303

UniFi Protect remains the primary NVR. Frigate will be an additional detection and classification layer. No installation or configuration should begin until the read-only validation phase is complete.

1. Project Objectives#

1.1 Functional Objectives#

The system will detect animals using Frigate motion detection, object detection, tracking, and zones.

It will distinguish coyotes from chickens, dogs, cats, deer, raccoons, foxes, and unknown animals. The initial tuning priority is avoiding missed coyote detections; false positives can be reduced afterward.

Home Assistant will handle final automation and notifications. Processing will remain local wherever practical.

1.2 Operating Constraints#

UniFi Protect remains responsible for normal camera recording and will not be replaced or disabled.

The project uses existing hardware only. Networking, firewall rules, Home Assistant, and existing Ollama configuration remain unchanged unless explicitly approved during the relevant phase.

Frigate will not store full-quality continuous recordings initially.

2. Proposed Target Architecture#

2.1 Processing Flow#

UniFi cameras / Protect stream
        -> Frigate on HP1GPU
        -> motion, object detection, tracking, zones
        -> animal event and snapshots
        -> Ollama vision classification
        -> structured animal classification
        -> MQTT
        -> Home Assistant
        -> notification or automation

2.2 Initial Component Placement#

Frigate will run on the HP1GPU Ubuntu VM at root@10.1.2.231, using /home/marc/docker-compose/frigate as its Docker Compose project directory.

The proposed GPU allocation is the Tesla P4 for Frigate detection and video decoding, with the Tesla T4 reserved for Ollama vision inference. This allocation remains subject to validation.

Frigate will use lower-resolution camera substreams for detection. Full-resolution snapshots or event crops will be used only when higher-level classification is required.

The existing MQTT broker will be used once its location and access are verified. Home Assistant will remain the final orchestration and notification layer.

3. Phase 1 — Read-Only Infrastructure Validation#

3.1 HP1GPU Hardware Validation#

  • 3.1.1 Confirm that HP1GPU exists as a Proxmox VM.
  • 3.1.2 Confirm HP1GPU has 24 vCPUs.
  • 3.1.3 Confirm HP1GPU has 128 GiB RAM.
  • 3.1.4 Confirm GPU 0 is an NVIDIA Tesla T4.
  • 3.1.5 Confirm GPU 1 is an NVIDIA Tesla P4.
  • 3.1.6 Confirm NVIDIA driver version 580.173.02.
  • 3.1.7 Confirm CUDA version 13.0.
  • 3.1.8 Confirm both GPUs are currently idle.
  • 3.1.9 Confirm NVIDIA container runtime availability on HP1GPU. NVIDIA container runtime is already working for Immich, OpenWebUI, and Plex.
  • 3.1.10 Confirm Docker access on HP1GPU. Docker GPU access is confirmed by Immich, OpenWebUI, and Plex using the NVIDIA GPUs.
  • 3.1.11 Confirm NVDEC support for the camera stream codecs.
  • 3.1.12 Document existing GPU-consuming workloads. Immich, OpenWebUI, and Plex currently use the NVIDIA GPUs.

3.2 HP1GPU Operating-System Validation#

  • 3.2.1 Confirm the installed Ubuntu version and kernel.
  • 3.2.2 Confirm available filesystem space for Frigate configuration, cache, and event snapshots. An 8 GB VM disk expansion was applied to the root LVM filesystem. HP1GPU now has a 69 GB root filesystem with approximately 15 GB available at 78% usage. A separate 32 GB scsi2 disk is now formatted as ext4 and mounted at /mnt/frigate, with approximately 30 GB available for Frigate configuration, storage, snapshots, and future event data. Root and Docker usage still require monitoring.
  • 3.2.3 Confirm Docker Compose availability. Docker Compose v5.4.0 is available on HP1GPU.
  • 3.2.4 Confirm the Docker storage location and available space. Docker stores images under /var/lib/docker on the root filesystem. The root filesystem has approximately 15 GB available after expansion; Docker currently uses approximately 19 GB and requires ongoing monitoring.
  • 3.2.5 Confirm the VM has reliable startup behavior after host reboot.
  • 3.2.6 Confirm time synchronization.
  • 3.2.7 Confirm the VM has stable network connectivity to the Home Assistant host. HP1GPU successfully reached Home Assistant at 10.1.2.200:8123.
  • 3.2.8 Confirm the VM has stable network connectivity to the intended MQTT broker. HP1GPU successfully reached 10.1.2.200 on MQTT ports 1883 and 8883; the active broker implementation and intended port remain to be confirmed in Section 3.4.

3.3 Network and Camera-Stream Validation#

  • 3.3.2 Test whether HP1GPU can reach UniFi Protect or a Protect-provided RTSP/RTSPS restream. The temporary firewall rule permits TCP traffic from HP1GPU (10.1.2.231) to the UNVR (10.1.0.2). ICMP ping is blocked, but TCP 7447 (RTSP) and TCP 7441 (RTSPS) are reachable. Restrict the temporary all-TCP rule to the final stream port after the authenticated stream is confirmed.
  • 3.3.3 Confirm whether direct camera access from the Lab VM network is blocked. Camera 10.1.4.2 is directly reachable from HP1GPU: ICMP succeeds and TCP 443 is open. TCP 554 refused the connection, so direct network access is available but the standard direct-camera RTSP port is not accepting connections.
  • 3.3.4 Confirm whether a Protect restream is available without changing Protect recording behavior. UniFi Protect exposes an RTSPS Share Livestream URL for the camera through the UNVR on TCP 7441; Protect remains responsible for recording.
  • 3.3.5 Confirm the stream URL format without exposing credentials in documentation. UniFi Protect provides an rtsps://<UNVR-IP>:7441/<stream-token>?enableSrtp URL, but Frigate requires the stored runtime value to use the rtspx:// scheme with the ?enableSrtp suffix removed. The transformed URL is stored in the Frigate 1Password item under FRONT_CAMERA_10_1_4_2_RTSPS_URL; the value must not be written to this plan.
  • 3.3.1 Identify the exact permitted path from HP1GPU to the camera video streams. Frigate will use the UniFi Protect RTSPS restream through UNVR 10.1.0.2:7441; direct camera RTSP access is not used.
  • 3.3.11 Record the temporary firewall access used for pilot testing. HP1GPU can reach the UNVR over TCP; restrict the temporary all-TCP rule to TCP 7441 after the RTSPS stream is validated.
  • 3.3.12 Record the Protect stream change used for pilot testing. The Protect Share Livestream RTSPS stream was enabled without replacing Protect recording.
  • 3.3.6 Create the minimal Frigate config.yml for the front-camera pilot. The configuration uses the protected RTSPS URL from the FRONT_CAMERA_10_1_4_2_RTSPS_URL 1Password field, enables only detection for front_camera, and keeps MQTT disabled for the initial stream test. The URL value is not written to this plan.
  • 3.3.7 Validate the generated Frigate configuration and Compose file without starting the service. Compose validation passed, and Frigate accepted the generated configuration during startup.
  • 3.3.8 Deploy the Frigate container image without starting the service. The stable-tensorrt image was downloaded and the container was created.
  • 3.3.9 Start Frigate for the one-camera pilot. Frigate is running healthy on HP1GPU.
  • 5.3.4 Confirm Frigate is receiving the intended stream and identify its resolution, frame rate, bitrate, and video codec. The active front-camera stream is H.264 Main at 2688×1512, 30 FPS, with AAC-LC mono audio at 16 kHz. Measured receive rate was approximately 4.25 Mbps during a 10-second sample. Frigate processes the detection path at 5 FPS.
  • 5.3.5 Confirm hardware video decoding is active. Frigate automatically detected NVIDIA hardware acceleration, and the container can see GPU 1 as the Tesla P4. The front-camera capture process started successfully.
  • 3.3.14 Confirm whether all seven cameras can eventually be accessed by Frigate.

3.4 Existing Service Validation#

  • 3.4.1 Identify the active Home Assistant host and confirm its reachable address.
  • 3.4.2 Identify the active MQTT broker and confirm its reachable address and port.
  • 3.4.3 Confirm whether the MQTT broker is Mosquitto or another implementation.
  • 3.4.4 Confirm that MQTT credentials can be provided securely without storing secrets in this note.
  • 3.4.5 Identify all existing Ollama endpoints.
  • 3.4.6 Confirm the Ollama endpoint intended for vision inference.
  • 3.4.7 Confirm available vision-capable Ollama models.
  • 3.4.8 Record baseline Ollama response latency using a representative test image.
  • 3.4.9 Confirm whether the Mac Studio should remain available as a fallback Ollama host.

4. Phase 2 — Frigate Design and Resource Budget#

4.1 Camera Scope#

  • 4.1.1 Select one outdoor camera for the initial pilot.
  • 4.1.2 Select the initial detection zone for that camera.
  • 4.1.3 Define which areas should be excluded from detection.
  • 4.1.4 Define whether a camera should use one or multiple substreams.
  • 4.1.5 Define the target detection frame rate.
  • 4.1.6 Define the maximum acceptable event-to-classification latency.
  • 4.1.7 Define the maximum number of frames sent to Ollama per event.

4.2 Frigate Resource Design#

  • 4.2.1 Use substreams for continuous detection processing.
  • 4.2.2 Keep Protect responsible for full-quality recording.
  • 4.2.3 Enable Frigate event snapshots only as required.
  • 4.2.4 Estimate CPU usage for one camera.
  • 4.2.5 Estimate GPU usage for one camera.
  • 4.2.6 Estimate memory usage for one camera.
  • 4.2.7 Estimate network bandwidth for one camera.
  • 4.2.8 Project CPU, GPU, memory, and bandwidth requirements for all seven cameras.
  • 4.2.9 Confirm that the Frigate configuration leaves adequate resources for Ollama and existing HP1GPU services.
  • 4.2.10 Define retention limits for Frigate event snapshots and metadata.

4.3 Detector and Decoder Selection#

  • 4.3.1 Test NVIDIA ONNX detection using the P4.
  • 4.3.2 Test NVIDIA hardware video decoding.
  • 4.3.3 Compare P4 detection performance with CPU fallback.
  • 4.3.4 Confirm whether the T4 should remain reserved for Ollama.
  • 4.3.5 Define a safe CPU fallback configuration for detector outages.
  • 4.3.6 Confirm that no Coral TPU or unowned accelerator is assumed.

5. Phase 3 — Controlled Frigate Installation#

5.1 Preparation#

  • 5.1.1 Obtain approval to begin installation. Approval was given to proceed with the one-camera pilot container setup.
  • 5.1.2 Document the selected deployment host: root@10.1.2.231.
  • 5.1.3 Document the selected Frigate project directory: /home/marc/docker-compose/frigate.
  • 5.1.4 Create and update the initial /home/marc/docker-compose/frigate/docker-compose.yaml file. The persistent /config and /media/frigate mounts now use /mnt/frigate/config and /mnt/frigate/storage on the dedicated disk. The server-side .env contains only op://Homelab/Frigate/... references for the UNVR credentials, and Compose consumes them through environment substitution. Compose syntax was revalidated without starting Frigate.
  • 5.1.5 Configure the initial Compose file to target NVIDIA GPU index 1 (Tesla P4) for Frigate.
  • 5.1.6 Validate the Compose file syntax without starting the service.
  • 5.1.7 Confirm the selected HP1GPU storage paths for Frigate configuration, cache, and event snapshots. Frigate persistent data will use /mnt/frigate/config and /mnt/frigate/storage on the dedicated 32 GB scsi2 disk.
  • 5.1.8 Create a backup location for Frigate configuration.
  • 5.1.9 Record current HP1GPU resource usage before changes.
  • 5.1.10 Record current NVIDIA GPU usage before changes.
  • 5.1.11 Confirm that installation will not stop existing services.
  • 5.1.12 Confirm the rollback procedure.

5.2 Initial Deployment#

The one-camera pilot startup sequence is tracked in Section 3.3 so the tasks are completed in dependency order. Configuration, hardware acceleration, MQTT, and multi-camera expansion remain follow-up work after the pilot container is running.

5.3 Initial Verification#

  • 5.3.1 Confirm Frigate starts without configuration errors. Frigate 0.17.2 is running healthy; database migrations completed and the API started.
  • 5.3.2 Confirm the camera stream remains available in UniFi Protect. The front-camera feed is visible in the Frigate UI through the UniFi Protect RTSPS restream, while Protect continues to provide the source stream.
  • 5.3.3 Confirm Protect continues recording normally.
  • 5.3.6 Confirm the selected GPU is processing detector workloads.
  • 5.3.7 Confirm which GPU index is used by each existing Ollama service.
  • 5.3.8 Confirm CPU usage is within the agreed budget.
  • 5.3.9 Confirm event snapshots are generated correctly.
  • 5.3.10 Confirm MQTT events are published.
  • 5.3.11 Confirm no unintended continuous recording is enabled.

6. Phase 4 — Local Vision Classification#

6.1 Ollama Model Selection#

  • 6.1.1 Select a local vision-capable Ollama model.
  • 6.1.2 Confirm the model can accept event images.
  • 6.1.3 Benchmark classification latency on the T4.
  • 6.1.4 Benchmark classification latency on the Mac Studio.
  • 6.1.5 Compare accuracy and consistency between the two hosts.
  • 6.1.6 Select the primary Ollama endpoint.
  • 6.1.7 Select a fallback Ollama endpoint if practical.

6.2 Classification Contract#

  • 6.2.1 Define the allowed labels: coyote, chicken, dog, cat, deer, raccoon, fox, other, and unknown.
  • 6.2.2 Require structured JSON output.
  • 6.2.3 Include a confidence value.
  • 6.2.4 Include an unknown result when the image is ambiguous.
  • 6.2.5 Include an optional explanation for troubleshooting.
  • 6.2.6 Require the model to distinguish visible evidence from uncertainty.
  • 6.2.7 Require conservative coyote classification during the initial recall-focused phase.
  • 6.2.8 Define behavior when multiple animals are present.
  • 6.2.9 Define behavior when the animal is partially occluded or too distant.

6.3 Event-Orchestration Design#

  • 6.3.1 Decide whether orchestration runs in Frigate, a dedicated service, n8n, or Home Assistant.
  • 6.3.2 Forward only relevant animal events to Ollama.
  • 6.3.3 Debounce repeated detections for the same tracked animal.
  • 6.3.4 Send multiple event frames when necessary to improve recall.
  • 6.3.5 Correlate classification results with the original camera and event ID.
  • 6.3.6 Prevent duplicate notifications for a single event.
  • 6.3.7 Record classification results for later tuning without storing unnecessary video.
  • 6.3.8 Define behavior when Ollama is unavailable.

7. Phase 5 — Home Assistant Integration#

7.1 Integration Design#

  • 7.1.1 Confirm the preferred MQTT integration path.
  • 7.1.2 Confirm the Home Assistant entity/event naming convention.
  • 7.1.3 Decide whether classifications become sensors, events, or both.
  • 7.1.4 Preserve existing UniFi Protect entities and automations.
  • 7.1.5 Keep Frigate events separate from Protect-native events.

7.2 Notification Logic#

  • 7.2.1 Create a coyote notification automation only after classification testing.
  • 7.2.2 Include camera name and event timestamp.
  • 7.2.3 Include a snapshot or event reference where appropriate.
  • 7.2.4 Include classification confidence.
  • 7.2.5 Avoid notifications for low-confidence classifications during the initial phase.
  • 7.2.6 Add a cooldown for repeated alerts.
  • 7.2.7 Add an explicit unknown/needs-review path.
  • 7.2.8 Test notification delivery without affecting existing security automations.

8. Phase 6 — Accuracy and False-Positive Tuning#

8.1 Test Dataset#

  • 8.1.1 Collect representative chicken events.
  • 8.1.2 Collect representative dog events.
  • 8.1.3 Collect representative cat events.
  • 8.1.4 Collect representative deer events.
  • 8.1.5 Collect representative raccoon and fox events where available.
  • 8.1.6 Collect confirmed coyote examples if available.
  • 8.1.7 Collect empty-scene and ambiguous examples.
  • 8.1.8 Keep test images local.
  • 8.1.9 Avoid sending test images to external services.

8.2 Evaluation#

  • 8.2.1 Measure coyote recall.
  • 8.2.2 Measure false-positive rate for chickens.
  • 8.2.3 Measure false-positive rate for dogs and foxes.
  • 8.2.4 Measure unknown/ambiguous classification frequency.
  • 8.2.5 Measure event-to-notification latency.
  • 8.2.6 Record missed detections and their cause.
  • 8.2.7 Tune zones, minimum object size, motion thresholds, and debounce intervals.
  • 8.2.8 Tune Ollama prompts and confidence thresholds.
  • 8.2.9 Re-test after each significant tuning change.

9. Phase 7 — Expansion to Additional Cameras#

9.1 Staged Expansion#

  • 9.1.1 Add the second outdoor camera.
  • 9.1.2 Verify resource usage and event latency.
  • 9.1.3 Add remaining outdoor cameras one at a time.
  • 9.1.4 Verify each camera’s stream, zones, and tracking.
  • 9.1.5 Add indoor cameras only if there is a defined detection purpose.
  • 9.1.6 Recalculate total GPU, CPU, RAM, storage, and network usage.

9.2 Seven-Camera Readiness#

  • 9.2.1 Confirm all seven camera substreams are reachable.
  • 9.2.2 Confirm the P4 can sustain the intended detector load.
  • 9.2.3 Confirm video decoding remains hardware accelerated.
  • 9.2.4 Confirm Ollama classification does not starve Frigate.
  • 9.2.5 Confirm MQTT and Home Assistant remain responsive.
  • 9.2.6 Confirm Protect recording remains unaffected.
  • 9.2.7 Confirm Frigate event retention remains within the storage budget.

10. Operations, Monitoring, and Recovery#

10.1 Monitoring#

  • 10.1.1 Monitor Frigate detector FPS and processing backlog.
  • 10.1.2 Monitor GPU utilization and memory usage.
  • 10.1.3 Monitor CPU and RAM usage on HP1GPU.
  • 10.1.4 Monitor Ollama latency and failures.
  • 10.1.5 Monitor MQTT connectivity.
  • 10.1.6 Monitor Home Assistant event and notification delivery.
  • 10.1.7 Monitor storage usage.
  • 10.1.8 Monitor camera stream failures.

10.2 Recovery#

  • 10.2.1 Document how to stop Frigate without affecting Protect.
  • 10.2.2 Document how to restore the previous Frigate configuration.
  • 10.2.3 Document how to disable coyote notifications independently.
  • 10.2.4 Document Ollama fallback behavior.
  • 10.2.5 Document recovery after HP1GPU reboot.
  • 10.2.6 Document recovery after Docker or VM restart.
  • 10.2.7 Confirm Protect continues operating if Frigate is offline.

11. Security and Privacy#

  • 11.1 Keep camera streams on the local network.
  • 11.2 Keep Ollama requests local.
  • 11.3 Do not expose Frigate, MQTT, or Ollama directly to the public internet.
  • 11.4 Store credentials outside this note.
  • 11.5 Use least-privilege credentials where supported.
  • 11.6 Limit Frigate access to required camera, MQTT, Home Assistant, and Ollama endpoints.
  • 11.7 Review event snapshot retention.
  • 11.8 Review access to stored snapshots and classifications.
  • 11.9 Confirm that no cloud vision API is used.

12. Current Open Questions#

  • 12.1 Can HP1GPU access the camera streams directly?
  • 12.2 Is a UniFi Protect restream available to HP1GPU?
  • 12.3 Which MQTT broker is active?
  • 12.4 Which Home Assistant host is active?
  • 12.5 Which Ollama service currently owns each GPU?
  • 12.6 Is the P4 sufficient for the final seven-camera detector load?
  • 12.7 Which cameras and zones should receive coyote classification first?
  • 12.8 Should the Mac Studio remain a fallback vision-inference host?

13. Completion Criteria#

  • 13.1 UniFi Protect continues recording normally.
  • 13.2 Frigate processes the approved camera streams reliably.
  • 13.3 Hardware video decoding is confirmed.
  • 13.4 GPU-accelerated detection is confirmed.
  • 13.5 Coyote classification is performed locally.
  • 13.6 Home Assistant receives structured classifications.
  • 13.7 Coyote notifications are reliable and deduplicated.
  • 13.8 False positives are measurable and tunable.
  • 13.9 The system has documented rollback and recovery procedures.
  • 13.10 All seven cameras can be added without exceeding the resource budget.
Backward Backup Strategies Immich & Photo Management Guide Forward