PLATFORM OVERVIEW

How NoryxLab works.

NoryxLab gives data scientists self-service compute, data and deployment on Kubernetes, and gives the team that runs the cluster explicit control over identity, resources and egress. This page describes the platform as it is built: the components, the objects, the API and what an operator inherits.

Community Edition · MPL-2.0Enterprise Edition · commercialKubernetes-nativeS3-native

What NoryxLab is

A control plane over a Kubernetes cluster you already operate. It turns four things into self-service, without asking a user to write a manifest:

  • Compute — interactive workspaces (JupyterLab, VS Code, RStudio), batch jobs, scheduled jobs and long-running applications, each a plain Kubernetes workload.
  • Data — datasets held in S3-compatible object storage, mounted into workloads, and database services provisioned beside them.
  • Environments — curated Python and R images, extended per project, built in-cluster and pinned by digest.
  • Access — projects and organizations resolved against your own OIDC identity provider, with per-resource sharing.

What it deliberately is not: a data plane. Your datasets stay in your object storage, your images in your registry, your identities in your directory. The platform's own database holds metadata — who owns what, what is running, what happened — and nothing that could not be rebuilt from your systems and a backup.

Design constraint

Every capability described here is one the platform can demonstrate on a running installation. Where something is an early implementation, this page says so rather than describing the destination.

Architecture

Three processes and a database, beside the cluster's own services. The control plane is small on purpose: it records intent and asks Kubernetes to do the work, so an operator debugging a workspace is debugging a pod, with the tools they already have.

CONTROL PLANE · namespace noryx
noryx-uiReact 19 · TypeScript
noryx-apiGo 1.25 · OpenAPI 3.0.3
PostgreSQL 16Metadata only
KUBERNETES · workload namespace
PodsWorkspaces
Jobs · CronJobsBatch
DeploymentsApps & data services
S3 / MinIODatasets, artefacts, backups
Keycloak / OIDCIdentity & organizations
RegistryHarbor · images by digest

What each component owns

ComponentResponsibilityFailure it survives
noryx-apiAuthentication, authorization, the object model, and the translation of intent into Kubernetes objects.Stateless: restart or replace it and running workloads are untouched.
noryx-uiThe console, served as static assets with runtime configuration.Independent of the API's version at build time.
PostgreSQLMetadata: projects, grants, workload records, audit events, usage samples.Restored from backup; workloads keep running while it is down.
Object storageDatasets, build contexts, backups. Reached with S3 credentials the platform never stores in clear.External to the cluster in a production installation.
Identity providerUsers, credentials, organizations. Keycloak by default, any OIDC provider in principle.Sessions already issued keep working for their lifetime.

Connections inside the cluster are encrypted where they cross a trust boundary: the database with TLS, the registry with its own certificate verified rather than skipped. Both are settings an installation turns on with a script, and the platform reports which state it is in on its health screen — a platform that silently talks to its database in clear is the failure this replaces.

The object model

Everything is addressed by a small vocabulary. Learn these fifteen nouns and the API holds no surprises, because the console uses exactly the same ones.

ObjectWhat it isWhat it becomes
ProjectThe unit of collaboration, ownership and access. Everything else belongs to one.A grant set, a quota, a namespace scope
OrganizationA group from your directory. Access can be granted to one rather than to twelve people.A Keycloak organization
WorkspaceAn interactive session — JupyterLab, VS Code or RStudio — with persistent storage.A Pod and a PVC
JobA batch execution with logs and a status.A Kubernetes Job
CronJobThe same, on a schedule.A Kubernetes CronJob
AppA long-running application published from project code, with revisions and rollback.A Deployment and a route
BuildAn image built in-cluster from a Dockerfile or a project environment.An image in your registry
EnvironmentA curated base image plus what a project adds to it, revision by revision.An image reference, pinned by digest
DatasetA named collection of objects in S3, shared with users or organizations.A mount, or a presigned URL
DatasourceA database or data service reachable from workloads, with managed credentials.A Deployment and a Secret
RepositoryA Git remote attached to a project, cloned into workloads.Credentials and a clone
SecretA project value encrypted at rest, injected into workloads by name.An environment variable
Semantic catalogAn inferred description of what a dataset contains. Profile-based, early: it reads object metadata, never object content.A stored manifest, queryable
Hardware tierThe named sizes a user may choose from, CPU or GPU, defined by the administrator.Resource requests and limits
QuotaWhat a project may hold at once: vCPU, memory, workspaces, jobs.A refusal that names the limit

Build, run, deploy

The same project, data and environment carry through from exploration to a deployed application. No export, no second platform, no separate data plane.

01

Build

Start from a curated Python or R environment. Add packages or a Dockerfile; the image is built in the cluster and pushed to your registry. The workload then runs the digest, not the tag, so a rebuilt tag cannot silently change what is running.

environment → build → image@sha256:…
02

Run

Launch a workspace for interactive work, or a job for something repeatable, on a hardware tier the administrator has published. Datasets mount, repositories clone, secrets are injected. Idle workspaces are reclaimed rather than left burning a GPU over a weekend.

workspace · job · cronjob → pod
03

Deploy

Publish project code as an application with its own revisions, logs and rollback, reachable through the platform's ingress and subject to the same access rules as the project it came from.

app → deployment → route
Not yet, and said plainly

An execution does not yet freeze an immutable record of the commit, environment revision, dataset state and parameters that produced it. The pieces exist; binding them into a run that can be reopened months later is the next object on the roadmap, not a capability to claim today.

The API

The console is a client of the same API you get. Every endpoint it calls, you can call.

162supported operations
123documented paths
3.0.3OpenAPI version
/api/v1versioned base path

Each installation serves its own specification and a Swagger UI at /swagger, from the platform itself — no request leaves the installation to render it, which matters when the cluster has no route to the internet.

Supported and internal

Two documents are published from one source:

  • /swagger/openapi.public.yaml — the supported API: the operations an integration may build on. Every one of them declares what it returns.
  • /swagger/openapi.yaml — everything the platform serves, the console's own endpoints included, each marked x-noryx-internal: true.

The distinction is enforced rather than documented: a build fails when a served route is missing from the specification, when a supported operation answers without describing its body, or when either document drifts from the router. An endpoint that only the console calls stays out of the contract, so the console can keep changing its own screens.

Authenticating a caller

Two paths, both carrying the same identity and the same authorization: an OIDC bearer token for a person, and a project-scoped API token for a script or a CI pipeline. There is no third, undocumented way in.

Identity and access

NoryxLab does not own your users. It reads them from your OIDC provider and resolves what they may do against its own grants.

  • Project roles — viewer, editor, admin. A role is granted to a person or to a whole organization.
  • Resource sharing — datasets and catalogs carry their own grants, by user or by organization, independent of project membership. Ownership can be transferred.
  • Platform administration — a separate capability, not a project role, covering organizations, hardware tiers, quotas, settings and the audit trail.
  • Audit — who did what, on which object, when, and whether it succeeded. Append-only, exportable as CSV.

The platform checks that your identity provider actually issues tokens it will accept, and says so on its health screen when it does not — the alternative being a user who signs in successfully and then has every request refused, with nothing anywhere explaining why.

Isolation and governance

Workload isolation

Workloads run in their own namespace, never beside the control plane, with network policies restricting what they may reach — including the platform's own database.

Controlled egress

A project asks for outbound access; an administrator grants or refuses it against named profiles. Requests and decisions are recorded. Enterprise Edition.

Quotas

Per-project ceilings on vCPU, memory, running workspaces and jobs. A launch that would exceed one is refused with the limit, the current usage and what was asked.

Usage accounting

Consumption is sampled every five minutes and totalled as vCPU-hours and GiB-hours per project, with a CSV export. Measured, not estimated from a price list.

🔒

Secrets

Project secrets are encrypted at rest with a key held outside the database, and injected by name. The API returns them to those allowed to read them, never in listings.

HDS

Regulated datasets

Datasets can be marked as regulated, which restricts direct download and forces access through the platform's audited paths. Enterprise Edition.

Operating the platform

Deployment is a set of Kubernetes manifests and one script, not a Helm chart with three hundred values. An installation is described by a handful of decisions: which namespace, which domain, which registry, whether TLS terminates at the edge proxy or at Traefik, and which storage class carries the volumes.

  • Drift is checked, not assumed. The deployment compares what is running against what the manifests declare, and refuses to silently drop a variable an installation depends on.
  • Smoke tests on every deployment. The version served, the identity provider's discovery document, the certificate's remaining life, the background workers actually running — asserted after the rollout, not eyeballed.
  • Continuous validation. An Enterprise module runs suites against the live platform on a schedule and reports what broke.
  • Health as a first-class screen. The platform reports its own conditions — registry unreachable, backup target missing, identity misconfigured — and can push them to a webhook. A condition nobody is told about is a condition nobody acts on.

Backup and recovery

A backup is only real if a restore has been rehearsed, and a run that omits part of its contract is reported as degraded — never as a success. That rule exists because the opposite was observed: months of green backups holding 2.3 kB of JSON.

What is protectedHowWhere it lands
Platform metadataScheduled run: projects, datasets, grants, workloads, plus the audit trail as compressed NDJSON.Your S3 target, off-site
Accounts and credentialsThe identity provider's own export, password hashes included, encrypted before it leaves the cluster.Your S3 target, encrypted
The keys themselvesA sealed bundle of the platform's secrets, so a lost cluster is recoverable from two items in a vault.Your S3 target, encrypted
DatasetsNot copied: they live in object storage protected by its own replication, and a manifest is not a backup of a terabyte.Stated, not implied

Restoring is deliberately reluctant: the default mode reports what is missing and writes nothing, so a restore cannot be fired by accident against a live platform.

What the editions change

The Community Edition is the whole core workflow, published under MPL-2.0 on GitHub. The Enterprise Edition is an overlay compiled into the binary: an enterprise capability is absent from the Community binary rather than present and disabled — nothing to unlock, nothing to bypass, and no telemetry deciding what you may run.

CapabilityCommunityEnterprise
Projects, organizations, rolesIncludedIncluded
Workspaces, jobs, scheduled jobs, appsIncludedIncluded
Environments and in-cluster buildsIncludedIncluded
S3 datasets, data services, repositories, secretsIncludedIncluded
Quotas and usage accountingIncludedIncluded
The full API and its Swagger UIIncludedIncluded
Audit trailPlatform eventsAdvanced audit, CSV export
Access modelFixed roles: viewer, editor, adminCustom RBAC matrix
Outbound network accessCluster policyControlled egress, requested and decided
Regulated (HDS) datasetsRestricted download, audited paths
Policy engineIncluded
Backups, restore and API tokensScheduled, off-site, rehearsed
Continuous platform validationScheduled suites and alerting
In-platform assistantIncluded, OpenAI-compatible API
SupportIssues on GitHubCommercial agreement

Both editions run the same control plane, the same object model and the same API. Moving from one to the other replaces an image; it does not migrate data or re-teach a workflow.

Requirements

RequirementWhat is needed
KubernetesA cluster you operate. Tested on k3s; nothing in the platform depends on a distribution.
StorageA default storage class for volumes, and an S3-compatible endpoint for datasets and backups.
IdentityAn OIDC provider. Keycloak is deployed with the platform if you have none.
RegistryA container registry the cluster can pull from and the platform can push builds to.
IngressOne hostname, terminated at Traefik or at an edge proxy forwarding by SNI.

Install the Community Edition

No account, no licence server, no vendor endpoint to reach. You need a cluster, a registry and a hostname; the rest is in the repository — github.com/Noryxlab/NoryxLab-CE.

  1. Get the source

    git clone https://github.com/Noryxlab/NoryxLab-CE.git
    cd NoryxLab-CE
  2. Put the runtime images in your registry

    The cluster pulls everything from your own registry, so nothing depends on a public one being reachable — or on it still hosting a tag next year.

    ./scripts/ops/sync-images-to-harbor.sh   # postgres, keycloak, minio
    docker build -t $REGISTRY/noryx-ce/noryx-backend:$TAG backend
    docker build -t $REGISTRY/noryx-ce/noryx-frontend:$TAG frontend
    docker push $REGISTRY/noryx-ce/noryx-backend:$TAG
    docker push $REGISTRY/noryx-ce/noryx-frontend:$TAG
  3. Apply the manifests

    Plain Kubernetes objects under deploy/k8s/base: namespaces, the control plane, PostgreSQL, Keycloak, MinIO, network policies and the ingress route.

    kubectl apply --server-side --force-conflicts -k deploy/k8s/base
  4. Create the realm, then harden it

    The first script creates the realm, the clients and the audience the platform requires. The second is the one an installation actually goes to production with: brute-force protection, a password policy, and the API client reduced to an audience with no login flow of its own.

    scripts/keycloak/bootstrap-realm.sh
    scripts/keycloak/harden-realm.sh
  5. Turn on what production needs

    Encryption for the database connection, a verified registry certificate, and somewhere for backups to go. Each is one script, and the platform's health screen tells you which are still missing.

    scripts/ops/enable-postgres-tls.sh
    scripts/ops/configure-backup-target.sh    # Enterprise Edition

Sign in through Keycloak and the first authenticated call creates your project. From there: an environment, a workspace, a dataset.

Honest about the effort

This is a platform for a team that already operates Kubernetes, and the install assumes it: an afternoon for someone who runs clusters, not fifteen minutes for someone who does not. There is no hosted option to fall back on, by design — that is the same reason your data never leaves your infrastructure.

Talk to us, or try it on ours

Three ways in, depending on what you want to find out.

SELF-HOSTED

Install the Community Edition

The whole core workflow, on your cluster, with no conversation required. Read the source before you run it.

github.com/Noryxlab/NoryxLab-CE →
GUIDED TRIAL

A sandbox project on our platform

A time-boxed project on a NoryxLab installation we operate, with its own quota and its own data. Enough to judge the workflow without provisioning anything.

Ask for a trial →
ENTERPRISE

The governed edition

Audit, RBAC, controlled egress, backups and validation, with a commercial agreement behind them.

Request a quotation →
BUILD · RUN · DEPLOY

Read the source, or ask for the Enterprise Edition.