PLATFORM OVERVIEW

How NoryxLab works.

NoryxLab gives data scientists self-service compute, data and deployment on Kubernetes, and gives the team that runs the cluster explicit control over identity, resources and egress. This page describes the platform as it is built: the components, the objects, the API and what an operator inherits.

Community Edition · MPL-2.0Enterprise Edition · commercialKubernetes-nativeS3-native

What NoryxLab is

A control plane over a Kubernetes cluster you already operate. It turns four things into self-service, without asking a user to write a manifest:

  • Compute — interactive workspaces (JupyterLab, VS Code, RStudio), batch jobs, scheduled jobs and long-running applications, each a plain Kubernetes workload.
  • Data — datasets held in S3-compatible object storage, mounted into workloads, and database services provisioned beside them.
  • Environments — curated Python and R images, extended per project, built in-cluster and pinned by digest.
  • Access — projects and organizations resolved against your own OIDC identity provider, with per-resource sharing.

What it deliberately is not: a data plane. Your datasets stay in your object storage, your images in your registry, your identities in your directory. The platform's own database holds metadata — who owns what, what is running, what happened — and nothing that could not be rebuilt from your systems and a backup.

Design constraint

Every capability described here is one the platform can demonstrate on a running installation. Where something is an early implementation, this page says so rather than describing the destination.

Architecture

Three processes and a database, beside the cluster's own services. The control plane is small on purpose: it records intent and asks Kubernetes to do the work, so an operator debugging a workspace is debugging a pod, with the tools they already have.

CONTROL PLANE · namespace noryx
noryx-uiReact 19 · TypeScript
→
noryx-apiGo 1.25 · OpenAPI 3.0.3
→
PostgreSQL 16Metadata only
KUBERNETES · workload namespace
PodsWorkspaces
Jobs · CronJobsBatch
DeploymentsApps & data services
S3 / MinIODatasets, artefacts, backups
Keycloak / OIDCIdentity & organizations
RegistryHarbor · images by digest

What each component owns

ComponentResponsibilityFailure it survives
noryx-apiAuthentication, authorization, the object model, and the translation of intent into Kubernetes objects.Stateless: restart or replace it and running workloads are untouched.
noryx-uiThe console, served as static assets with runtime configuration.Independent of the API's version at build time.
PostgreSQLMetadata: projects, grants, workload records, audit events, usage samples.Restored from backup; workloads keep running while it is down.
Object storageDatasets, build contexts, backups. Reached with S3 credentials the platform never stores in clear.External to the cluster in a production installation.
Identity providerUsers, credentials, organizations. Keycloak by default, any OIDC provider in principle.Sessions already issued keep working for their lifetime.

Connections inside the cluster are encrypted where they cross a trust boundary: the database with TLS, the registry with its own certificate verified rather than skipped. Both are settings an installation turns on with a script, and the platform reports which state it is in on its health screen — a platform that silently talks to its database in clear is the failure this replaces.

The object model

Everything is addressed by a small vocabulary. Learn these fifteen nouns and the API holds no surprises, because the console uses exactly the same ones.

ObjectWhat it isWhat it becomes
ProjectThe unit of collaboration, ownership and access. Everything else belongs to one.A grant set, a quota, a namespace scope
OrganizationA group from your directory. Access can be granted to one rather than to twelve people.A Keycloak organization
WorkspaceAn interactive session — JupyterLab, VS Code or RStudio — with persistent storage.A Pod and a PVC
JobA batch execution with logs and a status.A Kubernetes Job
CronJobThe same, on a schedule.A Kubernetes CronJob
AppA long-running application published from project code, with revisions and rollback.A Deployment and a route
BuildAn image built in-cluster from a Dockerfile or a project environment.An image in your registry
EnvironmentA curated base image plus what a project adds to it, revision by revision.An image reference, pinned by digest
DatasetA named collection of objects in S3, shared with users or organizations.A mount, or a presigned URL
DatasourceA database or data service reachable from workloads, with managed credentials.A Deployment and a Secret
RepositoryA Git remote attached to a project, cloned into workloads.Credentials and a clone
SecretA project value encrypted at rest, injected into workloads by name.An environment variable
Semantic catalogAn inferred description of what a dataset contains. Profile-based, early: it reads object metadata, never object content.A stored manifest, queryable
Hardware tierThe named sizes a user may choose from, CPU or GPU, defined by the administrator.Resource requests and limits
QuotaWhat a project may hold at once: vCPU, memory, workspaces, jobs.A refusal that names the limit

Build, run, deploy

The same project, data and environment carry through from exploration to a deployed application. No export, no second platform, no separate data plane.

01

Build

Start from a curated Python or R environment. Add packages or a Dockerfile; the image is built in the cluster and pushed to your registry. The workload then runs the digest, not the tag, so a rebuilt tag cannot silently change what is running.

environment → build → image@sha256:…
02

Run

Launch a workspace for interactive work, or a job for something repeatable, on a hardware tier the administrator has published. Datasets mount, repositories clone, secrets are injected. Idle workspaces are reclaimed rather than left burning a GPU over a weekend.

workspace · job · cronjob → pod
03

Deploy

Publish project code as an application with its own revisions, logs and rollback, reachable through the platform's ingress and subject to the same access rules as the project it came from.

app → deployment → route
Half of this is real, and we say which half

A job records what it read: the datasets it had mounted when it started — identifier, bucket, prefix, and whether it could write — alongside the image digest that ran. A job that had nothing attached records an empty list rather than nothing at all, so “read no data” stays distinguishable from “nobody wrote it down”. Nothing is backfilled: runs from before this cannot be reconstructed, and pretending otherwise would be worse than an absence that says so.

What is not there is the other end — the artefact a run produced and the model version that names it. A model whose output becomes a regulated device raises the question “which version, fed by which snapshot, produced this”, and answering it is a deliberate piece of work, not a field to add quietly.

The API

The console is a client of the same API you get. Every endpoint it calls, you can call.

162supported operations
123documented paths
3.0.3OpenAPI version
/api/v1versioned base path

Each installation serves its own specification and a Swagger UI at /swagger, from the platform itself — no request leaves the installation to render it, which matters when the cluster has no route to the internet.

Supported and internal

Two documents are published from one source:

  • /swagger/openapi.public.yaml — the supported API: the operations an integration may build on. Every one of them declares what it returns.
  • /swagger/openapi.yaml — everything the platform serves, the console's own endpoints included, each marked x-noryx-internal: true.

The distinction is enforced rather than documented: a build fails when a served route is missing from the specification, when a supported operation answers without describing its body, or when either document drifts from the router. An endpoint that only the console calls stays out of the contract, so the console can keep changing its own screens.

Authenticating a caller

Two paths, both carrying the same identity and the same authorization: an OIDC bearer token for a person, and a project-scoped API token for a script or a CI pipeline. There is no third, undocumented way in.

Identity and access

NoryxLab does not own your users. It reads them from your OIDC provider and resolves what they may do against its own grants.

  • What a role may create — twelve functions in rows, roles in columns, a box in each cell: create a project, launch a workspace or a job, deploy an application, build an environment, register regulated data, administer an organization. Three roles ship — an administrator acts on what is not theirs, a user on their own, a consultant reads what is shared and creates nothing — and an installation ticks their boxes itself, or writes rows of its own. One role per person, given when the account is created. A refusal names the role and the function, so somebody knows what to ask for.
  • Three words on the data — owner administers, writes and shares; write and read are grants on the object itself, to a person, a team or an organization. Belonging to an organization gives nothing by itself: what it gives its members is a grant, decided once in its settings or object by object. Transferring ownership leaves the former owner a write grant, written in the list rather than implied.
  • Project roles — viewer, editor, admin. A role is granted to a person, a team or a whole organization, and it says what somebody may do in the project: read it, change it, attach data to it, launch work. It confers no right on the data itself, and attaching does not grant one. Grants add up: the strongest applies, so removing somebody from an organization never takes away access they were given personally.
  • Roles of your own — Enterprise installations describe their own roles in a matrix and hand them out like any other. Each one declares which built-in role it inherits from, and that inheritance is both its fallback and its ceiling: a role based on viewer can be described as doing anything and, until the matrix enforces the row, it views. Enterprise Edition.
  • Resource sharing — datasets, ontologies, extracts and data sources carry their own grants, independent of project membership, on one screen that reads the same on all four. A frozen extract is read, never written; a connector's definition belongs to whoever may change where it points, because that redirects everybody who mounts it.
  • Platform administration — a separate capability, not a project role, covering organizations, hardware tiers, quotas, settings and the audit trail.
  • Audit — who did what, on which object, when, and whether it succeeded. Append-only, exportable as CSV.

Two matrices, and the distinction is the point. One says what a role may create, everywhere, with no object named — a box. The other says who may reach this object — an access list, which must never become a box, because the moment it does it stops being able to name a subject. Asking the platform what you may do is a single call, answered once per session: the interface hides what you may not do and never re-implements the rule, because two implementations of one rule always diverge and the user is the one who finds out.

One column of the project-role matrix is deliberately refused: a role is held inside a project, and the administration screens are not inside any project — so no project role can grant them. A matrix that could promote its own editor to platform administrator would be a matrix anybody reaching that screen could use to take the installation. Who administers the platform stays with your identity provider.

The platform checks that your identity provider actually issues tokens it will accept, and says so on its health screen when it does not — the alternative being a user who signs in successfully and then has every request refused, with nothing anywhere explaining why.

Agents, and what they are for

The agent and workflow model is public, but the scheduler, runner, tool endpoints and screens are Enterprise Edition capabilities. The question worth asking is not whether one can be deployed, but what happens when your risk function asks what they are allowed to do — and nobody in the room can answer.

NoryxLab draws the line before the demonstration:

  • If you can write it down, it is a job. Automation is a job with a schedule, a log and a result. An agent that runs a procedure somebody already specified is a worse job.
  • An agent is for what you cannot specify — looking at a situation every hour and saying whether anything deserves attention.
  • If nobody can verify the answer, it is neither. A judgement nobody checks is not a capability; it is an opinion with a schedule.

The constraints are the product

▤

A closed list of actions

An agent's mission is free text and is never parsed for instructions. What it may do comes from a closed list: restart a stopped application or call the platform API as its owner. Prose cannot widen it.

◱

A role is a ceiling

Observer, operator, lead. The role caps what the actions can be, so promoting an agent can never grant more than its rank allows.

✎

Written in advance

Teams and mandates record who may ask what of whom before the fact, in a form a person can read. They are stored and governed now; delegation execution is not yet enabled.

⊙

Inside a project

An agent belongs to a project. Its read tools enforce that scope. The broad call_api grant currently reaches every project its owner can reach; this documented limitation is not presented as containment.

✓

Recorded, not claimed

What an agent did is read from the actions the platform performed, never from its own report. A report is text a model produced; it can claim an action that never happened.

⌸

One page, one export

Every agent, its project, its role, what it may do, who may ask it what, and what it actually did — on one administration page and as a CSV, because the answer has to leave the platform to reach a committee. Enterprise Edition.

What agents can reach today

With call_api, an agent can call the platform API as its owner, including dataset objects that owner may read. It is not yet narrowed to the agent's project and has no per-path guard rail, so this grant is deliberately broader than restart_app and must not be used as a substitute for a narrow policy.

There is no verb for external web access, connectors or feeds. Model access is metered and narrowed by the gateway policy of the project named by the run; work marked as staying inside the perimeter is pinned to on-premise inference or refused, and the refusal is visible.

Isolation and governance

⊞

Workload isolation

Workloads run in their own namespace, never beside the control plane, with network policies restricting what they may reach — including the platform's own database.

↗

Controlled egress

A project asks for outbound access; an administrator grants or refuses it against named profiles. Requests and decisions are recorded. Enterprise Edition.

◈

Quotas

Per-project ceilings on vCPU, memory, running workspaces and jobs. A launch that would exceed one is refused with the limit, the current usage and what was asked.

∿

Usage accounting

Consumption is sampled every five minutes and totalled as vCPU-hours and GiB-hours per project, with a CSV export. Measured, not estimated from a price list.

🔒

Secrets

Project secrets are encrypted at rest with a key held outside the database, and injected by name. The API returns them to those allowed to read them, never in listings.

HDS

Regulated datasets

Datasets can be marked as regulated, which restricts direct download and forces access through the platform's audited paths. Enterprise Edition.

Operating the platform

Deployment is a set of Kubernetes manifests and one script, not a Helm chart with three hundred values. An installation is described by a handful of decisions: which namespace, which domain, which registry, whether TLS terminates at the edge proxy or at Traefik, and which storage class carries the volumes.

  • Drift is checked, not assumed. The deployment compares what is running against what the manifests declare, and refuses to silently drop a variable an installation depends on.
  • Smoke tests on every deployment. The version served, the identity provider's discovery document, the certificate's remaining life, the background workers actually running — asserted after the rollout, not eyeballed.
  • Continuous validation. An Enterprise module runs suites against the live platform on a schedule and reports what broke.
  • Health as a first-class screen. The platform reports its own conditions — registry unreachable, backup target missing, identity misconfigured — and can push them to a webhook. A condition nobody is told about is a condition nobody acts on.

Backup and recovery

A backup is only real if a restore has been rehearsed, and a run that omits part of its contract is reported as degraded — never as a success. That rule exists because the opposite was observed: months of green backups holding 2.3 kB of JSON.

What is protectedHowWhere it lands
Platform metadataScheduled run: projects, datasets, grants, workloads, plus the audit trail as compressed NDJSON.Your S3 target, off-site
Accounts and credentialsThe identity provider's own export, password hashes included, encrypted before it leaves the cluster.Your S3 target, encrypted
The keys themselvesA sealed bundle of the platform's secrets, so a lost cluster is recoverable from two items in a vault.Your S3 target, encrypted
DatasetsNot copied: they live in object storage protected by its own replication, and a manifest is not a backup of a terabyte.Stated, not implied

Restoring is deliberately reluctant: the default mode reports what is missing and writes nothing, so a restore cannot be fired by accident against a live platform.

What the editions change

The Community Edition is the whole core workflow, published under MPL-2.0 on GitHub. The Enterprise Edition is an overlay compiled into the binary: an enterprise capability is absent from the Community binary rather than present and disabled — nothing to unlock, nothing to bypass, and no telemetry deciding what you may run.

CapabilityCommunityEnterprise
Projects, organizations, rolesIncludedIncluded
Workspaces, jobs, scheduled jobs, appsIncludedIncluded
Agent and workflow domain modelReadable source and testsIncluded
Agent runtime, tools and workflow scheduler—Included
Agent governance and CSV export—Installation-wide governance and export
Environments and in-cluster buildsIncludedIncluded
S3 datasets, data services, repositories, secretsIncludedIncluded
Quotas and usage accountingIncludedIncluded
The full API and its Swagger UIIncludedIncluded
Audit trailPlatform eventsAdvanced audit, CSV export
Access modelFixed roles: viewer, editor, adminCustom RBAC matrix
Outbound network accessCluster policyControlled egress, requested and decided
Regulated (HDS) datasets—Restricted download, audited paths
Policy engine—Included
Backups, restore and API tokens—Scheduled, off-site, rehearsed
Continuous platform validation—Scheduled suites and alerting
In-platform assistant—Included, OpenAI-compatible API
SupportIssues on GitHubCommercial agreement

Both editions run the same control plane, the same object model and the same API. Moving from one to the other replaces an image; it does not migrate data or re-teach a workflow.

Requirements

RequirementWhat is needed
KubernetesA cluster you operate. Tested on k3s; nothing in the platform depends on a distribution.
StorageA default storage class for volumes, and an S3-compatible endpoint for datasets and backups.
IdentityAn OIDC provider. Keycloak is deployed with the platform if you have none.
RegistryA container registry the cluster can pull from and the platform can push builds to.
IngressOne hostname, terminated at Traefik or at an edge proxy forwarding by SNI.

Install the Community Edition

No account, no licence server, no vendor endpoint to reach. You need a cluster, a registry and a hostname; the rest is in the repository — github.com/Noryxlab/NoryxLab-CE.

  1. Get the source

    git clone https://github.com/Noryxlab/NoryxLab-CE.git
    cd NoryxLab-CE
  2. Put the runtime images in your registry

    The cluster pulls everything from your own registry, so nothing depends on a public one being reachable — or on it still hosting a tag next year.

    ./scripts/ops/sync-images-to-harbor.sh   # postgres, keycloak, minio
    docker build -t $REGISTRY/noryx-ce/noryx-backend:$TAG backend
    docker build -t $REGISTRY/noryx-ce/noryx-frontend:$TAG frontend
    docker push $REGISTRY/noryx-ce/noryx-backend:$TAG
    docker push $REGISTRY/noryx-ce/noryx-frontend:$TAG
  3. Apply the manifests

    Plain Kubernetes objects under deploy/k8s/base: namespaces, the control plane, PostgreSQL, Keycloak, MinIO, network policies and the ingress route.

    kubectl apply --server-side --force-conflicts -k deploy/k8s/base
  4. Create the realm, then harden it

    The first script creates the realm, the clients and the audience the platform requires. The second is the one an installation actually goes to production with: brute-force protection, a password policy, and the API client reduced to an audience with no login flow of its own.

    scripts/keycloak/bootstrap-realm.sh
    scripts/keycloak/harden-realm.sh
  5. Turn on what production needs

    Encryption for the database connection, a verified registry certificate, and somewhere for backups to go. Each is one script, and the platform's health screen tells you which are still missing.

    scripts/ops/enable-postgres-tls.sh
    scripts/ops/configure-backup-target.sh    # Enterprise Edition

Sign in through Keycloak and the first authenticated call creates your project. From there: an environment, a workspace, a dataset.

Honest about the effort

This is a platform for a team that already operates Kubernetes, and the install assumes it: an afternoon for someone who runs clusters, not fifteen minutes for someone who does not. There is no hosted option to fall back on, by design — that is the same reason your data never leaves your infrastructure.

Talk to us, or try it on ours

Three ways in, depending on what you want to find out.

SELF-HOSTED

Install the Community Edition

The whole core workflow, on your cluster, with no conversation required. Read the source before you run it.

github.com/Noryxlab/NoryxLab-CE →
GUIDED TRIAL

A sandbox project on our platform

A time-boxed project on a NoryxLab installation we operate, with its own quota and its own data. Enough to judge the workflow without provisioning anything.

Ask for a trial →
ENTERPRISE

The governed edition

Audit, RBAC, controlled egress, backups and validation, with a commercial agreement behind them.

Request a quotation →
BUILD · RUN · DEPLOY

Read the source, or ask for the Enterprise Edition.