Skip to content

[Feature Request] Add experiment tagging to Visdom (GSoC 2026 Starter Task) #1075

@tonypzy

Description

@tonypzy

Is your feature request related to a problem? Please describe.

ML researchers often run hundreds of experiments in Visdom. Currently, there is no structured way to categorize or filter these experiments, making it difficult to manage runs at scale.

Describe the solution you'd like

I propose an Experiment Tagging System designed for high reliability and performance:

  1. Hybrid Metadata Storage:
    Primary tags are persisted within each {env_id}.json for portability.
    A lightweight tags_index.json is maintained on the server to allow OOM-safe "lazy-loading" of metadata without parsing large environment files during startup.

  2. Concurrency & Data Integrity:
    Thread-safe writing using threading.Lock per environment.
    Atomic filesystem updates using a write-to-temp + os.replace pattern to prevent file corruption during crashes or high-frequency updates.

  3. Real-time Sync & SDK Symmetry:
    New WebSocket commands (tags_sync and tags_update) to broadcast changes to all connected frontends.
    Symmetric Python SDK methods: vis.set_tags(tags, env, append) and vis.get_tags(env).

  4. UI Integration (Phase 2):
    Tag-based filtering in the environment selector.
    Tag management interface within the environment management modal.

Describe alternatives you've considered

Manual naming (prefixing) is currently the only way, but it's rigid and doesn't support multi-dimensional filtering.

Additional context

I am a GSoC 2026 applicant and have already implemented the Phase 1 (Backend & SDK) infrastructure. I will link the PR shortly.

@hongquan @mariobehling @marcoag @norbusan

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions