MeteorCloud

Deploy and operate

Development Guide

Prerequisites

  • Docker and Docker Compose v2.24+
  • Optional, for Kubernetes: k3d, kubectl, helm
  • Python 3.13+
  • Node.js 22+
  • Make

First-time setup

cp deploy/compose/.env.example deploy/compose/.env

python -m venv .venv
source .venv/bin/activate

make install

make install installs control-plane, data-plane, installer, agent, and console dependencies.

For a product walkthrough (console + first device), start with Getting started.

Running the stack

make dev                 # all Compose files (shared network meteorcloud)
make dev-control-plane   # bundled infra (postgres/redis/minio profiles) + control-plane API
make dev-data-plane      # EMQX + data-plane MQTT gateway
make dev-console         # operator console

Together this starts:

ServiceURL
Consolehttp://localhost:5173
Control planehttp://localhost:8000
Data plane healthhttp://localhost:8081/health
OpenAPIhttp://localhost:8000/docs
PostgreSQLlocalhost:5432
MQTT TLSmqtts://localhost:8883
EMQX dashboardhttp://localhost:18083 (development only)
Redisinternal Docker network

Compose files:

FileServices
deploy/compose/control-plane.ymlbackend; profiles postgres, redis, minio
deploy/compose/data-plane.ymlprofiles mqtt (data-plane), emqx
deploy/compose/console.ymlconsole (Vite dev server)
deploy/compose/docker-compose.ymlinclude: of those three
deploy/compose/docker-compose.dev.ymlSource mounts and hot reload (make dev)
deploy/compose/docker-compose.prod.ymlProduction overlay: Traefik, restart policies, no internal ports

COMPOSE_PROFILES in deploy/compose/.env selects the bundled services. Remove a profile and point the matching setting (DATABASE_URL, REDIS_URL, OBJECT_STORAGE_ENDPOINT_URL, MQTT_BROKER_HOST) at an external service instead.

Same machine stacks must use network name meteorcloud.

  • Control plane alone: API works; MQTT/ping do not; console can still attach if VITE_API_BASE_URL points at that API.
  • Data plane alone: broker listens; device connect fails until control plane auth is reachable.
  • Console alone: static UI; unusable until a control plane is reachable at the configured API base URL.
  • Together: DATA_PLANE_URL and CONTROL_PLANE_URL use Compose service names, not localhost.
  • Console: VITE_API_BASE_URL=http://localhost:8000 in the local browser. Never a data-plane host.
  • Host-only (no Compose between them): DATA_PLANE_URL=http://127.0.0.1:8081 and CONTROL_PLANE_URL=http://127.0.0.1:8000.

Telemetry is PostgreSQL last-value on Device (TELEMETRY_PROVIDER=postgresql). Timescale and ClickHouse are reserved names and fail fast.

Stop with:

make stop
make stop-control-plane
make stop-data-plane
make stop-console

View logs:

make logs

Running services without Docker (optional)

Control plane

cd src/control-plane
export DATABASE_URL=postgresql+psycopg://edge:edge@localhost:5432/edge_platform
export DATA_PLANE_URL=http://127.0.0.1:8081
alembic upgrade head
uvicorn app.main:app --reload

Data plane

cd src/data-plane
export CONTROL_PLANE_URL=http://127.0.0.1:8000
export MQTT_BROKER_HOST=127.0.0.1
uvicorn data_plane.main:app --host 0.0.0.0 --port 8081 --reload

Console

cd console
npm run dev

The browser talks only to the control-plane API (VITE_API_BASE_URL). See Console.

Installer

cd infrastructure/installer
edge-installer validate --config config/examples/installation.yaml

Testing

make test
make backend-test
make data-plane-test
make console-test

Deployment checks (the same ones CI runs):

make compose-config    # docker compose config for base, dev, prod, observability
make compose-smoke     # minimal production stack + scripts/smoke_test.py (make images first)
make helm-lint         # helm lint + template for every values file
make terraform-check   # fmt, init -backend=false, validate (no credentials)
make ansible-check ansible-lint
./scripts/k8s-smoke.sh # throwaway k3d cluster: helm install, test, uninstall, delete

make test runs:

  1. Installer Pytest suite
  2. Control-plane Pytest suite (requires PostgreSQL; use make dev first)
  3. Data-plane Pytest suite
  4. Agent Pytest suite
  5. Console Vitest suite (make install-console first)

Control-plane Pytest uses a sibling *_test database, never DATABASE_URL / edge_platform.

Migrations and seed data

make migrate
make seed

See Identity and organizations for auth and tenant details.

Lint and format

make lint
make format

Environment variables

Copy deploy/compose/.env.example to deploy/compose/.env and adjust as needed. Values are written from the containers' point of view (service names, not localhost). Important values:

VariablePurpose
DATABASE_URLSQLAlchemy connection string for the running app
TEST_DATABASE_URLOptional pytest database (must end in _test)
JWT_SECRET_KEYSigning key for JWT utilities
BACKEND_CORS_ORIGINSAllowed browser origins
VITE_API_BASE_URLConsole → control-plane origin
VITE_DOCS_BASE_URLConsole → product docs URL
DATA_PLANE_URLControl plane → data-plane HTTP API
CONTROL_PLANE_URLData plane → control-plane ingest/auth origin
TELEMETRY_PROVIDERLast-value store (postgresql only; timescale/clickhouse reserved)
DATABASE_PROVIDERPersistence adapter (postgresql only)
CACHE_PROVIDERRate-limit adapter: redis (shared) or memory (per process)
OBJECT_STORAGE_PROVIDERArtifacts: s3 (any S3-compatible API) or filesystem (OBJECT_STORAGE_PATH)
MQTT_PROVIDERMQTT adapter (emqx only)
DEPLOYMENT_PROVIDERWho executes Deploys: native (default), none, or mender (stub until adapter exists). Releases/UI stay in MeteorCloud
OTA_PROVIDERDeprecated; prefer DEPLOYMENT_PROVIDER
IDENTITY_PROVIDERMember sign-in account directory (local only: PostgreSQL users)

HTTP JSON between planes is documented in contracts/mqtt-http.md.

Adding a backend module later

  1. Create src/control-plane/app/<name>/
  2. Keep routers thin; put domain logic beside the module
  3. Register routers from app/main.py
  4. Add Alembic migrations for new tables
  5. Add focused tests under src/control-plane/tests/
  6. New infrastructure vendors implement a port in app/ports/ — do not import vendor SDKs from domain services

Adding an installer component later

  1. Implement PlatformComponent in infrastructure/installer/ as needed
  2. Register it in the installer service registry
  3. Wire enablement through configuration models

Local Kubernetes

make k8s-up        # k3d cluster (development tool only)
make k8s-deploy    # build images, import, helm upgrade --install with values-dev.yaml
make k8s-status
make k8s-test
make k8s-down

The console and API are at http://localhost:8088, MQTT TLS at localhost:18883 (CA in .k8s/certs/ca.crt). See Kubernetes.

Coding standards

  • Type hints on all Python public functions
  • Prefer small, explicit modules
  • Raise NotImplementedError or print a friendly CLI message for unfinished work
  • Avoid generic repository / CQRS / event-sourcing patterns unless a real product need appears
Development Guide · MeteorCloud